AI bots make stuff up. This is not new; in fact, it’s so commonplace that people aren’t even phased by it. Rather than question the value of the bullshit robot, we’ve started to doubt the existence of non-bullshit.
Unfortunately, this problem isn’t going away. And while there’s a weird consumer religion out there that machine learning products somehow fix themselves (“sure, it’s bad now, but every mistake is just the model getting better!”), that’s not going to happen when bots learn from the internet and the internet is increasingly just content from robots that learned how to write content by reading the same internet.
Sometimes the product IS the ethics
Generative AI skeptics have been pretty clear that there’s an alarming correlation between ethics teams being dissolved and tons of new generative products coming to market. You don’t have to be a conspiracy theorist to see that obviously the ethical quandaries of bullshit robots and idea-smushing-together algorithms are significant, and that fully staffed ethics teams were probably putting a pretty high burden on getting stuff out the door.
Well, they’re largely gone now, so here come the products. They are very ambitious, and the main ethical effort made on most of them is to say “don’t use this for anything” somewhere in the terms of service. That’s pretty gross from a personal/moral/ethical standpoint, but in this particular case, it’s more than that. The ethical objections to this were partially about poisoning people’s minds with plausible fiction generated at scale with little effort. But some of them were also about destroying the content soul of the internet that these very products depend on to function.
We’ve blown right past that! Chatbot Tech is now a tourist beach resort that dumps its sewage directly into the ocean, and we’re barely out of the gate with this stuff.
This could have been
If this was a serious product, to me, you’d reverse the stack here. Output tools are not that interesting, because the output isn’t particularly good. At best, it’s fine and extremely cheap. But the input — that’s the good stuff. That’s how you’d deal with things like misinformation, “hallucinations”, and things like that. I don’t want tools to generate output that’s been trained on a bunch of garbage. If something is going to feign understanding, at least feign the understanding I want and don’t start mansplaining to me about things I know you don’t know.
But my guess has always been that LLMs need that first “L” to be so large that restricting the dataset to trusted info was always going to be a problem. That you could theoretically do it, and that you can probably do it today with some tools, but that either (a) with a smaller, more restrictive data set you can’t actually generate anything as human-ish as what the general purpose chatbots are barfing up and delighting everyone with, or (b) to work at all it would still need to reference other training, and there’s no viable way to say “look at this data to learn English, and this data to learn facts”, because there are no facts, we’re just spitting out tokens here.
Seriously, I’m guessing on this. I’m not a linguist OR an ML engineer (clearly), but I do know how people bring products to market when they want to make money, and one clear sign that something is impossible to do well is when the guy winning the race concedes a necessary portion of his/her market to a “network of partners” or developers and says “you guys figure this out, we’re just here to support you”. Best case scenario, you get the iOS App Store (which is still hell for a lot of developers, but we did get a lot of great apps along with limitless terrible ones). Worst case, you get Salesforce, where the last mile is a death march you pay a consultant $5,000 an hour to slog through with you and literally no one is happy, ever.
Ethically, owning the model should suck, because the model is what makes everything terrible. Unless, that is, you can convince everyone that (a) the model is a child so it both can’t be controlled AND you should feel bad about yelling at it, or (b) it’s actually the apps ON the model that need to take responsibility, even if that is impossible, or more likely, just not important to said app-makers.
I think this is where OpenAI and the likes are headed. They can’t control this, and that’s fine right now because they don’t need to. The magical, self-healing lie non-experts like me tell ourselves about ML will buy them plenty of time, and by the time we realize this will always be bad OpenAI will have shifted to “we enable the developers” and “tools are tools, they can be used for good or evil and we think everyone should use them for good” with absolutely no blame-able hand on the steering wheel.
Plus, all the current execs and employees will leave and sit around being rich and blaming 2028’s management team for losing the “mission” of the company.