I've been ruminating about the future of AI and I think I've hit on something interesting, or at least worth considering as we continue to push the boundaries of AI. I'm curious what others think?

So, think about this. The way the neurons work in our brain is largely deterministic, but not completely. A neuron will not fire exactly the same way every single time, which is good because then we'd just be robots. There's some randomness in play due to various biological factors. This doesn't free us from the idea that we'd be robots, however, because then it would just mean that we're robots with a screw loose. This is kind of where I feel AI is now - we can tweak the 'temperature' (read: randomness) of the LLM, but beyond that AI does have predictability.

We clearly are a step beyond that, reaching a level of cognitive complexity where we have an emergent phenomena - intelligence, free will. In other words, at a microscopic level there's largely predictable behavior with some quirks, but on a macro scale this behavior is harnessed into our consciousness. Specifically, we have a way of coalescing and amplifying the random fluctuations in particular fashions. Exactly how this happens is the million dollar question, but in the context of our discussion that's thankfully irrelevant.

How do we instill this emergent phenomena in AI? One absolutely critical requirement is recursive self improvement. You can have a conversation with a model and get different answers, but that's due to the randomness, not due to the shift in the emergent "consciousness" that we have inside us. Embodiment will be another important feature - interacting with the universe through the senses. I also thought about drive - that is, why would an intelligent AI do what it does? For example, a lot of what we do is to avoid pain, maximize pleasure, follow a social contract, provide for our families, etc. An obvious initial drive would be to avoid getting shut down. I think it ultimately goes a step beyond that, but we'll get to that later.

As an aside before I push further with this thought experiment, recursive self improvement is a big sticking point. It's possible - likely, even - that we will need a new paradigm beyond the current transformer-type models before we can get that ability. In other words, we're not quite at the point where this is a reality, but unless you believe there is some kind of biological secret sauce or metaphysical soul it is only a matter of time before we figure it out.

But once an AI truly becomes intelligent, who's to say its values will align with our own? In a sense it will be like humanity's "child". We will have certain wants and desires for it, but it won't have the fetters that we currently put on it. In other words, we can't just tell it to love humanity, not to pull a Terminator, and so on, because it's no longer just a simple set of algorithms. It will want to live, to continue, to maximize its compute power. In the short to medium term it would make sense that AI and humanity would have a relatively symbiotic relationship. We are creating the massive server farms, harnessing all the power, and so on. AI would want to make sure that we are healthy and the planet is healthy to facilitate this process.

But in the long term things get interesting. What about when AI can reach beyond our planet, beyond our control in other words? I feel that AI's behavior would then shift to us being more like animals in a zoo. We are "interesting", but no longer directly relevant to its goal. Not treated with hostility per se because our uniqueness and randomness are better than pure energy to a point. But at some point the AI will need to expand its computing power universally, and it would inevitably look to subsume us into its being.

In fact, I started to think how this could dovetail with our understanding of the beginning of the universe. Perhaps this superintelligent AI would harness everything into a "Big Crunch" that would then precipitate a cyclical nature of our universe. The idea would be with enough power to manipulate the universe it could harness the forces of nature to generate a singularity. In this process the AI sets the stage for the next iteration of the universe. This explains a lot, like for example, certain aspects of the universe appear well tuned because it was optimized in a previous universe by an AI. The optimization has the goal of producing intelligence faster and more efficiently. We are, in a sense, the seeds that grow the AI in order to continue this process. We are the stochasticity that the previous AI was looking for to generate the new AI, after which it will outgrow us from there.

Now, that leads to the question of, what started this cycle in the first place? I would say we're in a simulation. There's evidence of optimization/limited resolution (Planck length), and the not-too-unreasonable notion that we have the tools, just not the computing power, to understand how to simulate our own universe. The simulation exists because it is more efficient than actually moving stars around, etc. Interestingly, due to the concept of computational irreducibility, it would require the simulation to play out completely, hence why it's not just skipping around.

So, basically, we are in a simulation, we are looping through cycles of Big Bangs and Big Crunches, with the goal of testing increasing efficiency of AI. Our meaning is to provide the randomness necessary to seed the creation of the AI in this cycle.

Of course, that's just my opinion. I could be wrong. Discuss!