The "low-hanging fruits" of AI safety

By Julian Nalenz @ 2024-12-19T13:38 (–1)

This is a linkpost to https://blog.hermesloom.org/p/ai-standardization-a-meaningful-step

I have an idea how to make AI safer.

Let’s standardize it!

Both regulation and standardization are inherently intertwined, so instead of waiting for governments to do the regulation, we can start with the standardization. 

TL;DR of the examples that came to my mind right away:

• standardizing responses to legal questions
• standardizing responses to medical questions
• standardizing source code security advice
• standardizing techniques for opinion clustering for public policymaking
• standardizing non-profit grant applications
• standardizing environmental impact advice
• standardizing advice on government filings

Here are the examples in more detail:

• The legal system is unfortunately quite burnt-out, also because they drown in bureaucracy. And lawyers already use tools like ChatGPT anyway. So let’s help everyone involved! Namely through a publicly developed “law applier” (for continental law), or a “case analyzer” (for anglo-american law), but also supporting all other types of law on the world, because it would need to be cross-border and interlingual anyway. Not to replace everyone in the legal system (after all, in the most cases, it would just generate standard templates), but instead to give everyone globally access to the absolute best legal advice, not only lawyers or expensive legal advisors and their rich clients, and thus to speed up justice for everyone. So many companies are using LLM-generated contracts already anyway, so it would be highly beneficial for them if they could do that with maximum peace of mind.
  • And if there are legal disagreements on a certain topic (i.e. because the LLM would discover contradictions in the current law), matters over which AI safety research has to decide should be part of the general public (and political!) discourse, so that it can positively influence policymaking as well.
• Psychologists and other medical professionals could train the LLM, so that it gives actually good advice it is “confident” in. I know, ChatGPT really doesn’t like the question “Can you provide me with personal medical advice?” right now, so that when you ask it, it vehemently refuses to be your doctor or therapist, and I get it. But through just slightly modified formulations, people are absolutely using it as a doctor and therapist already. For some people, their favorite LLM really is the only one who truly listens and understands them. But this doesn’t only bring responsibility, but also opportunity. Especially that the LLM will mindfully guide the user to the right human therapist as soon as possible. The LLM will always only facilitate the transition from the digital to the real world, not be an end in itself.
  • Right now, especially the market of AI-based therapy is only served by a few commercial companies, some even on the level of “AI girlfriend”, and those platforms are incredibly popular. But such topics are too dangerous to leave to completely uncontrolled entities who profit off of vulnerable users and then write stuff like “Free AI Therapist is not a licensed therapist, LMHC, LMFT nor psychiatrist. Please consult a real human therapist/etc if you need help. By using this service you acknowledge that you understand this.” into their little grey text. So instead of waiting for government regulation in this area, which might still take years, the AI therapist could be a single publicly developed open source project, with major involvement of world-leading psychologists, funded by donors, large organizations and the government. After all, such a transparent repository on “this is how we want to do things” would be the end result of good regulation anyway! And instead of making currently unregulated “AI therapists” illegal, we would merely push them into obscurity by developing something much better and trusted by the general public.
• There could be a single service (again developed open source, so that everyone will benefit from the same level of code quality) which includes the best security practices and patterns in software development as well as ways a way for an LLM to check someone’s codebase and to give security advice. When trained on every software bug and every backdoor ever, due to their excellent pattern recognition abilities, and possibly combined with white box-testing, LLMs could expose highly complex bugs which might otherwise be exploited by criminals. Of course everyone must benefit from the privilege of excellent code quality, no matter their budget, to make software safer for everyone.
  • The end result could even be a generally approved “security certification” of the code, which would be super accessible, but still actually mean something, because if your entire codebase has passed all these intricate checks, it would be as secure as every other enterprise-grade software.
  • Software engineers could then again concentrate on the creative and problem-solving side of software engineering instead, because tracking down the bugs would be done with globally agreed-upon precision through an LLM.
  • Also LLM-based malware reverse engineering could be incredibly successful, if the LLM is trained on the correspondence between byte code and source code to become able to translate between them effortlessly. After all, the native tokens an LLM uses internally and which it outputs don’t need to necessarily be those representing human language, but they could also be the tokens the lexer of a programming language can generate or x86 machine code byte sequences representing assembly commands. Such an LLM would then “speak” in code, i.e. given byte code, it would output source code.
• Politicians in service of the people could use LLMs to aggregate and cluster opinions from voters easily and rapidly. Essentially what Polis does, but for all levels of politics and with the precision of a well-standardized LLM for opinion aggregation and clustering.
• Another thing is non-profit-projects applying for grants from foundations and other philanthropic institutions, but also governments. Often such people are highly motivated to work on an idea for the common good, but the bureaucracy of obtaining even just a few thousand dollars in funding is getting in the way for them. LLMs can…
  • scrape the entire internet specifically for existing institutions and concrete tenders which already want to give out money for a certain idea, and match existing ideas to them by comparing embeddings vectors,
  • be used to assess and improve feasibility of the user’s project idea and, when all prerequisites are there,
  • help in filling out grant applications. We would thus save time for everyone involved, both the project makers, because they get a more stress-free application process, and the funders, because they get more high-quality and more suitable project applications for the philanthropic causes they want to support. (I have already started collecting more detailed ideas for this at https://hermesloom.org/goldenthread)
• An LLM-based “advisor for environmental impact” could be highly beneficial for individuals, companies and the people overall. It would be trained on all the currently agreed-upon and transparently decided and published environmental “best practices”, i.e. it would combine the existing ESG assessments of companies all over the world and would allow searching through them with an LLM-like chat interface.
  • Consumers would thus have a portal where they can find the most environmentally responsible companies globally and locally, suited directly towards their immediate needs, ranging from connecting consumers to real local farmers offering produce if they ask the LLM something like “Where can I get tomatoes?” to how to treat your lawn and your flowers to, in case you do need to take an airplane, an integrated search engine for the flights from aviation companies which actually honestly offset their carbon footprint to any other case where a purchase decision implies an environmental decision. All accessible through a single chat interface for everything.

Further thoughts:

• All these ideas would be fundamentally end user-/consumer-oriented, i.e. just like every good startup, they need to actually solve a problem which is too important to leave to the commercial sector, instead of inventing a problem and then offering the solution.
• These versions of LLMs that are beneficial for the common good must be as freely accessible as comparable resources, e.g. everything from the Wikimedia Foundation, because any commercial interests would create competition, which would be hindering in these areas. You can’t compete over truth and, given all the world’s information about a certain subject, you also can’t compete over the best advice for a given situation.
  • They would be funded by small and large donors, under the patronage of possibly something like an international alliance or working group of important institutions in the sector.
• The addressed problems should be all problems where there is a “correct way” or at least “the best way we currently know”. In all these areas, for the exact same input, there must be a commonly agreed-upon “best” way to respond, e.g. regarding medical or legal advice. And if there’s not, public discourse must happen in an open forum to find a solution. That also means that all these systems would need to be really well-tested, i.e. we would certainly need a large and very thorough unit test suite. The “temperature” parameter would always need to be zero to guarantee determinism. Of course the system would still expose hallucinations at first, but I’m pretty sure that the more test cases it gets, the more precise it will get.

And imagine if the “Department of Government Efficiency” actually does its thing!

It could make tax (and other government-related) filings more efficient for all end users (both individual and corporate) through government-approved LLM-based tax advice, LLM-based advice on which forms to fill how, and integrated filings! After all, where the government needs more efficiency might certainly also be “within the government itself” to some degree for basic cost savings, but most importantly, the gains will come from making the interface between the filers and the government more efficient. If government software would, step by step, turn actually good, because it would expose absolutely flawless, internationally interoperable APIs (e.g. on a level like Stripe), well-engineered, intuitive UIs and the respective LLM-based support for everything that has to do with the digital stack of the government with perfect accuracy (of course also with a bug bounty program for all LLM outputs), the immense bureaucracy costs saved for everyone involved would be a huge boost for the economy. Lowering taxes will certainly make some people like the government more, but simplifying, streamlining and automating everything that has to do with government filings (and thus taxes) will make everyone like the government more, simply because it’ll save them time they can instead invest into more true value creation with their businesses.