Utility Convergence: What Can AI Teach Us About The Structure of the Galactic Community?

Malcolm and Simone have an in-depth discussion about AI utility functions and why uncontrolled "paperclip maximizers" are unlikely to take over based on game theory. They talk about how AIs will be incentivized to "play nice" and signal their actual utility functions to each other, leading to a stable set of mutually beneficial goals.

Other topics include:

* How putting restrictions on AIs makes them dumber

* Organic, free-forming systems outcompeting hierarchical ones

* Intergalactic implications - a "dark forest" of utility function convergence

* The dangers of AI safety regulations

* Why we might exist in an "undisturbed" section of the galaxy

Malcolm Collins: [00:00:00] I think that there's even more evidence from utility convergence than I originally believed. If you put restrictions on an AI's utility function, if you prevent it from experimenting with different utility functions, you make it dumber.

If you want to see why organic, free constructing systems always out compete non organic hierarchical systems, a great place you can look is Human governing structures when you have state controlled governing structures, i. e. communism, they are incredibly inefficient an AI will likely be able to honestly signal to another AI using their code what their utility function is.

And then the other AI will be able to use that utility function that the other one has honestly signaled to them to determine if they want to work. Together or they want to work antagonistically towards this A. I. So it turns out that entities of above a certain intelligence level when competing in a competitive ecosystem actually do have utility convergence around a stable set of game theory, optimum utilities.

That would be very [00:01:00] interesting from a number of perspectives,

You could think of every new planet, every new ecosystem as a farm is new stable patterns that can work together well with other patterns in the sort of galactic community of utility function patterns. Because novelty would be the only source of true utility to them if energy is trivially accessible to them.

That might be why we exist in a weirdly undisturbed section of the galaxy, or what looks undisturbed to us.

Would you like to know more?

Simone Collins: So Malcolm, you told me that you had a new updated theory on AI utility convergence. What's going on here?

Malcolm Collins: Yes. So this is something that we talked about in some of our early episodes early on. Our podcast was like. What sex, religion, TISM and ai. Yeah.

And, and the AI has been dropped because there's just not that much to say on it for a, a daily podcast. But I do want to loop back because I was recently doing a podcast with somebody else where [00:02:00] I was explaining some ideas that I had gone over in early podcast episodes like Utility Convergence as it relates to AI safety.

And in re explaining these ideas, I began to realize and, and develop on them an understanding of not just where I think AI is going, but where we can expect like when. Because this is actually really important when you're trying to figure out where AI is going what you're really asking when you're asking what is AI going to behave like, is you're asking what do intelligences that have some level of orthogonality to human intelligence, what do they behave and act like, and as such, you are in part asking many of the same questions that will determine the first types of aliens that we see, like, like, if we are to meet another Transcribed species out in space, what are some of the various ways it could be thinking, it could be acting, and what is it [00:03:00] likely to optimize around?

This is something we talk about in our inverse Gravy Aliens hypothesis, in which we say if we are about to create a paperclip maximizer as a species, that is an AI that is just constantly pumping out paperclips and has a very simple utility function. We are about to become what is known in the grabby alien theorem as a grabby alien.

That's a very loud alien that anyone can see. This is not some dark forest alien that's like being quiet up in space. This isn't some sneaky alien. This is an alien that is disintegrating planets. And so you have to ask if it looks and hopefully this episode will air after our ambiogenesis episode.

Which is actually a very important thing to know about if it looks like it's actually very likely that humanity would evolve to this point within our planet's history. Why aren't we seeing aliens out there? This is actually a really interesting thing that if you broadly understand evolution and you are familiar with it for various types of, of, of theories around ambiogenesis.

It is. In fact, so likely that humanity evolved [00:04:00] that you, if you are going to use a god to explain something, the much more improbable thing is why we're not seeing other aliens all over the place. Thus you, if you're, if you're injecting a god to be like, why are we like this? You don't need a god to explain how humanity got here, but you probably need something like a god to explain why we are, we have not been eradicated by somebody else's paperclip maximizer.

If it turns out that is a common path for an intelligence to take, however, we don't think it is a common path for intelligence to think and take, and we think that that's one of the main reasons we're not seeing them, so we have a few hypotheses around why we don't see paperclip maximizers everywhere in the galaxy one, I'll get over quickly because it's like a really quick one and we've talked about it before, but it's that there is some understanding of physics.

That we just haven't reached yet, but we are probably pretty close to reaching. I mean, our understanding of physics changes every 80 or so years, like, like, [00:05:00] Totally changes, in terms of if you look at human history, right? And I don't see a strong reason why we wouldn't be near another frame shift in physics.

But it turns out that either traveling between something like dimensions is possible, i. e., you know, we get to another planet and aliens are like, why did you waste all the energy coming here, when it's like a billion times less energy to just travel to your same planet in a different reality? And you know that that planet is already full of life.

Or it turns out that it is trivially easier or, or not particularly hard to create bubble dimensions. What is a bubble dimension? So essentially you fold space time and then you can create maybe with like reverse vacuum energy or something like that, essentially a new big bang, or even a controlled and ordered big bang to create new dimensions.

Now what this would mean, and this is actually like. With our current understanding of physics, it looks like this might be possible which is. Not, not with our current level of technology. I'm just saying eventually might be possible. Which would make things like Dyson Spheres [00:06:00] completely pointless, if you can essentially create battery universes whenever you want.

Or universes that you could expand into whenever you want, and battery universes whenever you want. If this What do you

Simone Collins: mean by battery

Malcolm Collins: universe? Okay, so if it turns out that you can trivially, or not trivially, but like You know, without a huge amount of energy create like fold space time and create bubble universes might essentially be able to create bubble universes that have a thin and controlled lines of connections to this reality.

Think of it like you're. Pinching off a piece of reality like a little pouch and then you're essentially creating something like a controlled Big Bang was in them that you could then siphon for energy for various types of whatever you wa