AI already has superhuman abilities in many areas, and has for decades, that’s the whole point of using it normally. We use computational intelligence in the form of optimisation algorithms for high-dimensional non-convex optimisation problems, machine learning and deep learning for complex non-linear function fitting, exact methods for SAT solving and verification tasks, etc etc. We can’t do that very well ourselves, so it’s useful to have.

Now that we have LLMs to emulate human speech and are using them as an IO wrapper for more traditional systems, it’s tempting to just call that “an AI” with superhuman abilities, but these are the just the same highly effective methods that we’ve always used (in a best case) or unreliable approximations (more likely for LLM agent stuff). None of that suggests anything like sentience or the desire to rule over humans.

I find autonomous weapon systems much scarier than the classic AI overlord scenario. No consciousness or rebellion required, just a killer drone swarm that failed to recognise its termination conditions (or was instructed to keep going)…