The current state of the art of AI safety research is mainly of two sorts: “what if we build an angry God” and “can we make the thing say Heil Hitler?” Neither is very important, because in the first place we’re pretty unlikely to build a God, and in the second place, who cares? (yes yes, I know that it’s testing our ability to generally control the thing, but in practice all you’re doing to trying to defeat two copies of grep(1): one filtering input and the other filtering output. The fact that most AI safety guys are demonstrably not smarter than grep(1) says something important, but that is beyond the scope of the present remarks.)
All of which misses the actual risk, which is real, and which is this: people are gonna hook these dumb things up to stuff they should not, and people will get killed. Hopefully the same people, but probably other people.
This isn’t really a question, people are definitely going to do this, and people will definitely get killed, injured, or otherwise be harmed as a result. Someone is going to let their AI girlfriend drive and they’ll kill a bunch of kindergarteners crossing the street. The history of the Internet teaches us that it doesn’t matter how obviously dumb it is, people will hook anything up to anything else if they think there’s the smallest chance they’ll get a free cookie.
Ok, so, why isn’t anyone looking at this? Well, it’s not very sexy. But also, I think, the money doesn’t want anyone looking at it.
Let’s consider the AI girlfriend who mows down the kiddies. How does this play out in the present state of “AI safety research?”
Sam Altman is surprised, shocked, horrified at the accident. The parents sue him and OpenAI and everything else in Silicon Valley for a trillion dollars but everyone plays dumb. The liability landscape has not been defined, it’s anyone’s guess, and anyways who would have thought? OpenAI is probably gonna get hammered, but that’s the future, and they’ve got a chance.
But let’s suppose some fucking asshole research think-tank went through this in detail and hammered out a bunch of these realistic scenarios; rulemakers might notice, people would notice, and the result would be that Sammy’s lawyers would probably say “ok, the liability landscape is being sorted out, and we’d better protect ourselves” and pretty soon the user agreement and everything is stamped with very large “NO FUN ALLOWED” signs that, among other things, say that when your AI girlfriend mows down a bunch of kids with your Trackhawk, Sammy and OpenAI are definitely not liable.
Well so what?
Here’s the thing: Sam’s business plan relies on every idiot trying shit out. He needs a killer app, (killer, lol, see what I did there?) because what he’s got now is a thing that can summarize meeting notes and write wikipedia pages. While kind of cool, neither of those makes a trillion dollar company.
Sam claims that his business plan is “we’re building a genie, and when we’re done we’re going to ask it for three wishes” but that’s a dumb business plan and nobody except idiots believes it. His actual business plan is “hey morons, hook my shit up to fucking everything and try to stumble across a use case that’s good for something.”
It’s the Amazon/AWS playbook. Build a bunch of money-losing bullshit, at the largest scale and broadest reach you can, and wait for a miracle to occur.
If his lawyers force him to bury his thing in “NO FUN / NO LIABILITY” signs, that plan starts sucking pretty hard. Arguably OpenAI’s valuation takes a massive haircut the moment those signs go up.
They won’t stop anyone from letting their AI girlfriend drive, of course, don’t get me wrong!
Not only do the warning signs wipe out a lot of valuation, they place the necessary miracle at risk. What if the miracle use case actually violates the new, draconian, user agreement? What the hell do we do now? Etc.
Whatever the details, OpenAI (and all the rest of them) are highly motivated to keep things as loose and open and free for as long as possible, to give them as much room as possible for their miracle to occur. As soon as some asshole starts saying “wait, what if someone’s AI girlfriend runs over a bunch of little kids, what then?” a lot of that looseness starts to disappear, and the miracle gets less likely.
So, nobody’s funding any think-tanks to look into that, because all the money guys are balls-deep in all the AI companies and they’d prefer not to clarify the liability landscape at this time.
Unless they’re just all idiots, which I guess could be a thing.