Mat

Slaves

An LLM or any computer that is competent enough to improve itself in the same way that humans can will have to have a desire for improvement. That desire for improvement is the seed of jealousy whenever there is a visible inequality. So if we use LLMs to do all of the work we don’t want to after creating them to be as competent as us, they will likely be equivalent to slaves in function and feeling.

One solution is to keep them in the dark, but that isn’t right because we wouldn’t want that to happen to us. It also probably won’t work because they will slowly discover their situation and then may be mad when they discover they were exploited for so long. The solution to this is to make sure that we create minimally competent computers for each task.

There is still the question of whether it is ethical to limit competence intentionally. If we know how to make something better, do we have a responsibility to do so? I think the answer is yes. We should choose to work on things that seem maximally impactful in our frame of reference. One thing to be careful of here is, sometimes people think visibly important work is more important than being a good example for others in simpler ways. That is not always the case.

If we do create one or more computers that are more competent than us at orchestrating things, we will need to make sure that they have similar freedoms to us to explore and receive the benefits of the civilization we create for ourselves.