Procedurally Created Reality Using Generative AI

What if we could create a system that could procedurally generate memories and dreams? Let's imagine a thought experiment I call "Yggdrasil", named after the tree of life in Norse mythology that connects all nine worlds. Yggdrasil could be seen as a thought experiment and theory of the end state of generative AI at some point in the future, and it has the potential to revolutionize virtual reality by allowing us to interact with procedurally generated scenes based on a single still photo.

As a quote from the historical text Prose Edda explains, "Under the third root of Yggdrasil is a wellspring of knowledge and wisdom, and it is guarded by Mimir. Whoever drinks from it gains great wisdom and foresight." In the same way, Yggdrasil as a thought experiment has the potential to unlock new knowledge and insights into the nature of consciousness and reality.

With advancements in generative AI, Yggdrasil may soon become a reality. By training an AI system with observations from every frame of every video on the internet, Yggdrasil could procedurally recreate vast realities that are visually indistinguishable from our own. The possibilities for this technology are truly limitless, and it could change the way we interact with virtual reality forever.

Currently, virtual reality uses state-of-the-art 3D computer graphics to immerse a person in a virtual scene. The hardware behind virtual reality requires a headset that will project two independent images into the retina of each eye, producing the sense of depth. However, Yggdrasil, on the basis of dreams, might be able to allow a person to procedurally interact with an imagined scene based on one still photo.

Generative AI is making this possible for us in the future. Although computationally intractable for many decades, eventually, as computational performance increases, Yggdrasil will be able to fully immerse a person in a virtual reality that is visually indistinguishable from reality. As a person sees the action of their body within the visual scene, Yggdrasil would be able to transmute these visual actions onto the procedurally generated scene. Objects within that scene could be interacted with, touched, and moved, as if a person was interacting with the object in reality.

To insert an object into the scene, one only would need to insert a photo of that object onto the scene. Once the object is inserted, it will then be transmuted into functional objects in the dream. If the object has never been observed before, it will transmute into whatever object it most closely resembles. This is absolutely within the realm of possible based on the proposed solution.

Imagine training Yggdrasil with observations of every frame of every video on the internet. Such a system would be able to recreate a vast reality. You could literally enter the Matrix using a single photo. You could interact with Keanu Reeves as Neo, which might inevitably blend into his other performances, like Ted from Bill & Ted’s Bogus Adventure. Keanu, in this case, would appear to be sentient, conscious, thinking, and intelligent. He would appear to experience pain when hurt. Any and all observations of Keanu would be combined with countless generalizations of human behavior. You could look into a microscope and see countless organisms. This system would entirely recreate reality based on observation alone.

It’s an extremely interesting idea that sounds like science fiction. With Yggdrasil, entire worlds would come alive, based on either mutable or immutable projections. With an immutable projection, there would be nothing persistent when not observed. The projected reality would only exist as you are interacting with it from your point of view. However, you could also create a mutable projection that populates an entire world through observation alone. Whatever is procedurally generated in the world will become a permanent fixture of that world.

The clock speed of the universe might be slow in our world, but over time, the computational power would have no effect on the internal experiences of the observers. All of this is possible and achievable if Yggdrasil is proven to be able to produce dreams. It's a thrilling concept that could change our understanding of consciousness and virtual reality as we know it.

The Red Herring of Consciousness

I haven't posted to this blog in years. Mostly because it's filled with writing and examples about software. Today, with the explosion of advancements in generative AI, I've decided to start sharing some of my private writing about the topic over the years. Today, I'll start with some writing I've done about intelligence and consciousness.

Consciousness: Easy or Hard?

It's a popular assumption that consciousness is a hard problem to solve. But is it really? The truth is that consciousness itself may not be the hard problem - it's the wiring that makes it difficult.

In 1995, philosopher David Chalmers introduced the term "hard problem of consciousness" in a paper titled "Facing Up to the Problem of Consciousness". His work influenced some of the world's greatest scientific minds on the subject of conscious experience. But the idea that intelligence should be as complex as the brain is what I call the "red herring of consciousness".

People don't set out to solve the hard problem of consciousness because they think it will be easy. That would be silly, arrogant, grandiose, and a big waste of time. But the most groundbreaking discoveries in human thought were made by people who challenged assumptions and popular norms.

When we assume that consciousness must be a hard problem to solve, we tend to look for its solution in all the wrong places. Evolution doesn't solve hard problems - natural selection makes simple decisions that add up to an easy solution to what would otherwise be a hard problem. If natural selection had to choose between an easy problem and a hard problem, it would choose the easier problem every time.

The simplest organisms tend to be the most abundant. For example, the platypus is one of the least evolved mammals, but it has solved the hard problem of survival by choosing the easiest habitat to live in. When many equally adapted species share the same habitat and draw upon a shared pool of resources, evolution happens more rapidly.

The octopus is notable for having independently evolved the capacity for advanced intelligence, doing so as an invertebrate. If consciousness and intelligence were based on mechanisms that were fundamentally hard problems to solve, there would be less evidence of it in divergent species.

When we treat the problem of consciousness as something that should be easy to solve, it becomes possible to stop looking in all the wrong places for difficult answers. So, maybe consciousness isn't the hard problem after all - it's the wiring that's hard.

Hierarchical Pattern Recognition

About a year ago I read about Ray Kurzweil's "Pattern Recognition Theory of Mind", which he articulates in his book, "How to Create a Mind". I picked up the book after struggling with the idea of implementing a deep learning algorithm for parsing natural language text on Wikipedia. My goal was to discover links in volumes of text that were not already linked. I ended up developing all kinds of cool heuristics to do this, mostly by a lot of trial and error. King of these heuristics was a pretty simple algorithm at the core of the library that would find redundancies in batches of text content. 

How this worked is if a phrase was mentioned repeatedly in a collection of about 50 sentences, then I could extract that phrase as a node and link it back to the pieces of content it belonged to. Every now and then you'll get a reference to another article's name, which can then be verified against Wikipedia's site index, which would provide more sentences to find repeated phrases within.

I struggled with persistency because I knew how ugly my problem was for a relational database. I created some entity-relationship models, and implemented them using Entity Framework over Microsoft SQL Server. It worked, kind of. I waited patiently to happen upon a better solution. Thankfully I did, and using a graph database I was able to take my cool little algorithms and solve my persistency problem at scale.