Heard some people say that a so called “Human Zoo” is one of the best-case scenarios with pseudo-omniscient AI taking care of all of us. Supposedly that comes true, to what ends would humans continue to keep a population going, simply due to biological bias? You think humans would devolve in some way, or become intellectually atrophied?


That’s theoretically true, but that’s not what LLMs do. LLMs truly are just n-th level autocomplete. They aren’t simulating brain function. They’re just repeatedly outputting what ‘should’ be the next set of letters in the sequence.
And why do you think that human brains basic mechanics are not similarily simple? Intelligence is an emergent property of some physical system. There is no magic making biological cells fundamentally different than transistors in this sense. They may be built very differently, but both are still just physical devices.
It’s not a matter of a priveleged medium. It’s the fact an LLM is not a simulation of a brain. You can do the math for an LLM on paper, given enough time. The same cannot be said for a human brain. If human brains were as comprehensible as an LLM, we’d be too stupid to comprehend them.