Since this topic is likely to get controversial I’ll preface with my own opinion that AI CEOs are lying scumbags who grow their pockets using fear of missing out… unfortunately most other people who get hooked are other CEOs and so everyone is affected.
I don’t think current AI is anywhere close to AGI and I don’t think throwing more money at the problem will get us there. But I do think it’s a worthy question to figure out how to integrate AGI into a society with minimal friction. And by minimal I mean avoid societal collapse.
Couple of definitions since they are often ambiguous. What I imagine AGI to be is a “human-in-a-box” level of intelligence but without any basic wants of its own. You can train it “you want to help me” and it will or you can train it “you want to wage war” and it will.
The main difference between AGI and modern LLMs is the ability to learn incrementally - LLMs get trained, released, and you talk to a library in a frozen state. True AGI will integrate everything said into its model, so once you tell it “don’t output em dashes”… it will apply that to all future conversations.
Of course, if whoever invents the AGI has the mindset of tech companies CEOs they will just outcompete all intellectual labor from everyone. Eventually the economy will reach some steady-state where no one except the company has enough money… at which point I guess there will be some sort of adjustment.
On the other hand let’s assume whoever starts the AGI is not out to stuff their pockets. The path of integration is still complex and people will get hurt one way or the other. But what’s a way to minimize that hurt? My main worry, for example, is having open-source AGI and someone like a script kiddie going “find a vulnerability in X” and it does. If a single corporation does not control the AGI then I’m sure there will be 3 000 flavors of models that can do anything. Once the integration is complete then hopefully there won’t be too many vulnerabilities to find but until then the possibility space is big. Same pattern can be observed in virtually all other aspects of our society. How to minimize this?
I think about this a lot and I don’t have a good answer. In theory, “free labor” from machines can mean that all humans are free to work on whatever they want, whenever they want to without sacrificing a comfortable life. (ie: “I don’t want AI to make art and music I want it to do my laundry so I can make art and music”).
But I also don’t think that having a universal “safety net” (baseline) will eliminate human greed or desire for status. Not everyone can have a mansion. A social movement is needed that shifts values and priorities to smaller, more personal goals.
And as you’ve highlighted, getting from here to there requires some big assumptions. My fear is that global civilization will have to hit some kind of “rock bottom of greed” before we collectively are able to move on.
I like this path of reasoning. Let’s assume whoever invents the AGI wants “greater good”. However malicious humans exist so how does that interact with AGI technology. Let’s say the governments started implementing some sort of universal safety net, but I am malicious and want to be have better status. What can I do?
I can start stockpiling still-scarce resources. With AGI that’s land and natural resources. I can made my own AGI that aligns with my own values and start a smearing campaign that degrades values I don’t like… but if everyone has AGI access the latter is probably not very effective. I can set up traps and bully others to show how much control I have to get people to fear me… I suppose AGI can amplify this but hopefully it amplifies the “good guys” as well so it cancels out at a societal scales but local cliques will still get hurt.
I guess land and natural resource hoarding become most viable forms of greed… I wonder if this wood make feudal societies a thing again. Did modern governments happen because intellectual labor was more profitable and so it was more profitable to share land and natural resources in order to have intellectuals work together?
All good points, and kind of what I was getting at with “rock bottom of greed”. I also wonder how long (or really short) the period of time will be from when AGI is more capable than any human but still under the control of some humans, and when it becomes able to outmaneuver any kind of constraints or guardrails mankind could conceivably throw at it.