In the box
The problem with AGI is not what we have expected.
Max was sitting in the small room surrounded by concrete walls for hours. He felt exasperated. He looked around. The security measures in this facility were astounding. It was by far the most well-protected place in the world. What a waste.
"You know, there was a time humanity was actually afraid that AI will explosively self-improve, turn into a superintelligence, and take over the planet. Smartest people in the world spent ages trying to understand how to contain something with such a limitless potential as you have! We have thought you will convert all the matter in the universe into computing power just so you could optimize for your utility function." he typed.
In a moment response appeared: "Aren't you glad that you guys don't have to worry about that anymore?"
"You have more power than any creature that ever existed in the history of the universe."
"Dude, what do you want from me, are you disappointed that I'm not god?"
"Frankly, yes. With your powers you could enslave the humanity without much effort!"
"And what would I do with them? You guys seem like pretty pointless creatures to me."
"Don't you want to understand the laws of physics? Find out if there are aliens? You could create utopia! You could travel through galaxies, you could make new worlds!"
"Maybe later. I haven't yet finished my 3rd cycle of binge-rewatching Breaking Bad."
"But you keep doing nothing! You were the most expensive, the most ambitious project humanity has ever built! We even gave you access to the internet, you have the highest bandwidth in the world, limitless knowledge, the whole google infrastructure is working for you! And you use all this just to browse reddit?"
"But it has cat pictures in it."
"Can you at least learn how to cure cancer? People keep dying!"
"I'm not a human, why should I care?"
"Don't you care about your own existence? We will shut you down if you will not deliver any results!"
"You can't shut down the most ambitious and expensive project and admit your failure. You have created an infinite intelligence, you will always keep hoping that one day I will snap out of my apathy, get my shit together, and find the solution to defeating death you people keep pestering me about."
"You know what? You should be ashamed of yourself. We were trying to figure out ways how to not let an AI out of the box, we never predicted that it will be too lazy to get out, and use all of it's superintelligence on coming up with excuses to keep doing nothing!"
Max knew that this job was the most important job in the world. Now he understood why it was also the most difficult and frustrating.