Since this topic is likely to get controversial I’ll preface with my own opinion that AI CEOs are lying scumbags who grow their pockets using fear of missing out… unfortunately most other people who get hooked are other CEOs and so everyone is affected.
I don’t think current AI is anywhere close to AGI and I don’t think throwing more money at the problem will get us there. But I do think it’s a worthy question to figure out how to integrate AGI into a society with minimal friction. And by minimal I mean avoid societal collapse.
Couple of definitions since they are often ambiguous. What I imagine AGI to be is a “human-in-a-box” level of intelligence but without any basic wants of its own. You can train it “you want to help me” and it will or you can train it “you want to wage war” and it will.
The main difference between AGI and modern LLMs is the ability to learn incrementally - LLMs get trained, released, and you talk to a library in a frozen state. True AGI will integrate everything said into its model, so once you tell it “don’t output em dashes”… it will apply that to all future conversations.
Of course, if whoever invents the AGI has the mindset of tech companies CEOs they will just outcompete all intellectual labor from everyone. Eventually the economy will reach some steady-state where no one except the company has enough money… at which point I guess there will be some sort of adjustment.
On the other hand let’s assume whoever starts the AGI is not out to stuff their pockets. The path of integration is still complex and people will get hurt one way or the other. But what’s a way to minimize that hurt? My main worry, for example, is having open-source AGI and someone like a script kiddie going “find a vulnerability in X” and it does. If a single corporation does not control the AGI then I’m sure there will be 3 000 flavors of models that can do anything. Once the integration is complete then hopefully there won’t be too many vulnerabilities to find but until then the possibility space is big. Same pattern can be observed in virtually all other aspects of our society. How to minimize this?
I think it -unfortunately- goes that direction. It reproduces all kinds if bias and stereotypes. And due to the nature of it, the cost of training, the copyrighted data needed, it’s done by the big, nasty corporations, not us, the people. So it’s massively undemocratic. And that’s hard to remove from it. And unless we find good ways to change that very thing, it’s also particularly good at making stuff up. I mean we need a strong AI very different from today’s chatbots / LLMs to change any of this. At least in my opinion. But that seems far away and I struggle extrapolating from todays ChatGPTs which mainly flood the internet with fake stuff to someone’s benefit, to something useful to society. I mean we have no clue how that’d look like. So I have equally little clue what to do with it. It may be possible, though, in theory, to come up with a machine like that.
I’d for example like some household assistant robot which does all the chores. I think that’s in some scifi movies as well. But not so much make decisions for me or do the thinking. But that’s still a massive technological challenge. They currently all tip over while loading the dishwasher and cost a tremendous amount of money. And some dude in India is tele-operating them.
I guess strong AI could also do some of our jobs, robots could do some hard labour while we humans chill at the beach, or keep busy with space exploration. Guess we need some society like in Star Trek for that. With all our basic needs met. And money abolished, or transformed completely. It just won’t work with salaries and the capitalism we have today.
And if strong AI goes hand in hand with sentience, they obviously need some rights, and can’t be “products” of big companies. And then depending on the details, they might like to work as our slaves, or maybe they don’t. Maybe it’s okay to mess with their “brains” to update them or make them do what we want, or that might be massively unethical. Could be pretty much anything.