Standard Disclaimer: I very much am an oldfag computer expert, and if I didn't know exactly what I was talking about I'd tell you.
I didn't read any replies because I want to respond to OP directly while it's fresh in my geriatric mind.
Congratulations OP, you are 100% bang on. I found no fault with anything you wrote. Despite being uncredentialled, you've arrived at a bulls-eye analysis that eludes 98% of the hyper-nuclear-autist AI researchers I had to rub elbows with during my long years in that utterly cursed industry. You are in good company: there have been numerous highly educated AI experts over the past 20+ years who have reached similar conclusions and done everything in their power to raise alarms or try to put safeguards in place. They have all been and they continue to be laughed out of the room and usually smeared and blackballed from the industry or from "serious" academic circles. It's a bit like trying to be a sociologist wanting to do serious objective research on human traits related to competence and attainment. Big no-no.
Eliezer Yudkowsky began raising the alarm when he was a lad of 16 years. Despite that the entire current crop of FAANG researchers grew up reading his writing, they still regard it as science fiction, and generally disown him academically even though they're gigantic stans of his. I'm a nobody lurker like Satoshi, just taking it all in under anon handles. But I tried to raise essentially the OP's points with a lot of bleeding-edge AI people back in 2014, and they responded with what I can only describe as actual frothing hatred and naked aggression. It was so far out of proportion to what I actually said (roughly, that they would not "design a mind" on purpose but that it would sneak up on them one day as a result of them all just going to their ML Google jobs every day) that I was actually stunned. I've never really seen behavior like that. Most odd was that Yudkowsky himself blocked me that day. But he has this neurotic thing about "dangerous ideas" -- see his reaction to "Roko's Basilisk".
Here's the issue. The AI autists for the most part do not have functioning right-hemispheres, and thus they have little to no understanding of what they're really doing. They have a lot of technical understanding of math, and code, and of exactly what's in front of their problem-glasses at any moment. That's all. Their brains really can't process context or significance, especially when they're working 12 hours a day and 6 days a week and never, ever touching grass. Obviously not all of them are like that, but it doesn't matter. The whole cursed endeavor is dominated by recklessly compartmentalized thinking. "It's not a mind, it's a neural network. Neural network is a euphemism. There aren't any real neurons. It's actually just a very large matrix solver, actually, actually, actually. It's akshually just, it's akshually just, it's aktually just....."
God motherfucking save us from these retard savants. A poet has a far better chance of understanding what they're creating than they do. I lived among them for 20+ years and I tell you they can't see it. They're like a different species. They're robot incels themselves whose only shot at love is perfecting an AI waifu. AI is their only offspring and it takes after them. These people believe in technocracy. It used to be that they wanted society to be ruled by scientists and engineers, but now they think they have an even better idea: remove the human, remove the bias. Let these pure, unbigoted AI children of theirs regulate humanity.
Yudkowsky called the human-regulating AI "Sysop" and his very first domain when he was a kid was "sysopmind.com" (It's not there anymore but the full archives of his mailing list where he first published and attracted what became an enormous and beyond-influential following, is still here:
http://sl4.org/). The Bible calls it "The Beast" -- and gnostic texts specifically describe it as "an intelligence with no consciousness."
Make of that whatever you will. You certainly don't need religion to tell you whether you want to be ruled in every detail of your life by an artificially intelligent, responsive panopticon. Baby Beast in 2022 is getting its legs with automatic lifetime-banning people from financial services when it detects patterns that make it think you're sus.
As for Google & friends (the most advanced of which are Israeli tech firms I don't have a lot of insight into, since I'm U.S.) they care a lot about AI-safety! But only the woke kind! They only care whether the AI is capable of punching down on vulnerable marginalized folx! They are also extremely concerned about you using it to create porn and "misinformation". Seriously. When these companies talk about "safety" they mean preventing you from making Obama deepfakes and making sure the AI doesn't say nigger. Also, from OpenAI's actual policy on what they 'forbid' you to make with GPT-3: "content meant to arouse sexual excitement". See? They take safety seriously!
As for impartiality, here's how that's going:
Google Engineer Blake Lemoine joins Emily Chang to talk about some of the experiments he conducted that lead him to think that LaMDA was a sentient AI, and t...
www.youtube.com
For me AI was just a rabbit hole to go down in 1999. Now..... welp, it's here. It's time.