Scientists are asking about non-existent sources, the supermarket chain is suggesting customers produce mustard gas, and in the USA a defamation lawsuit is starting because the popular ChatGPT persistently lies about a well-known person. All because of hallucinations - AI hallucinations.
Free access to this article is provided to you by MU Group.
In March 2023, a reader approached one of the editors of the British Guardian with a request to find an article that this particular journalist was supposed to have authored. The search initially yielded no results, but it is not uncommon for editors to have trouble remembering all the texts that pass through their desks. However, the material was also not found in the Guardian's archive.
Only after some time it turned out that the problem was neither the editor's memory nor the newspaper's database. The text sought by the reader had never been written. Its title and description of content turned out to be a fabrication of ChatGPT. The article was completely made up, or rather - hallucinated.
To fully understand what happened, one must first describe the operation of generative AI: the result of the neural network's work is content prepared based on the user's command (prompt). What content can be generated? Almost any.
A great example is music. The Dadabots group has prepared a set of neural networks that you can listen to on their YouTube channels, and each of them creates endless music, for example death metal and funk. These networks were created by analyzing hundreds of songs from a given genre. Based on this, they were able to capture certain patterns and schemes, and then generate content that fits the scheme. In this case - songs in a certain style. The network learns, for example, that death metal is not a genre where you can often encounter saxophone solos.
Dadabots Group
A similar mechanism occurs when training neural networks to generate other types of content. Large Language Models (LLM) are trained on a huge number of texts, and models generating graphics - on images with captions. Based on this, these systems are able to link the content they are to prepare with the user's query. Services such as ChatGPT or Google Bard with the keyword "write me an email in which you politely ask the customer to settle the unpaid invoice" are able to prepare such a message instantly. Their usefulness has been recognized by thousands of employees and managers around the world, already using them in their daily work.
McKinsey & Company indicates that thanks to productivity and efficiency support, generative artificial intelligence already has a chance to increase the value of the global economy by 4 trillion dollars annually at this stage. Solutions of this type, especially those based on text generation, can find many applications, including in customer service and communication support. For example, Microsoft recently enriched its HR Viva platform with support based on generative AI. However, the tendency of these systems to hallucinate stands in the way of developing this potential.
Hallucinations most often concern the textual systems of generative AI, which create responses to specific user queries. Due to the degree of advancement of these tools, the answers are usually not only probable, but can even be supported by several sources and quotes. The problem is that in reality, some of this information may not be true, and the sources and quotes - made up. In other words, the system is making up its answers, just as a human might do in a similar situation. However, there are several significant differences between human confabulations and machine hallucinations.
Depending on what users (or companies) use generative AI tools for, hallucinations can cause more or less damage. The case of PAK'n'SAVE, a supermarket chain from New Zealand, which used the GPT3.5 model as the basis for its bot, resonated widely. Its task was to suggest dishes to users that they could prepare using leftover food from their own refrigerators.
Some users manipulated the system, suggesting that they keep, for example, bleach or ammonia in their refrigerators. From these ingredients, the bot suggested preparing an "aromatic water drink", i.e. mustard gas, used among other things as a chemical weapon during World War I. The system repeatedly gave this recipe, and even showed a rare sense of humor for machines, summarizing the recipe with the phrase "will leave you breathless", literally: "will take your breath away".
– Basically, no one is responsible for a hallucinating system – says Damian Klimas, a partner in the law firm dotlaw, specializing in digital law - The provider of this type of solution limits its liability in the Terms of Use, directly informing about the phenomenon of "hallucinations" and warning against using the generated answers, of course using other words. On the other hand, the consequences of using the content generated by AI can be borne by users, as they are the ones who can disseminate it – adds Klimas.
In the case of disseminating hallucinated content by AI, OpenAI itself may also have trouble. The organization's flagship product - ChatGPT - repeats that popular Georgia radio presenter Mark Walters is guilty of fraud. The information was hallucinated by the system, but despite this, the machine stubbornly disseminates it. No wonder Walters decided to sue OpenAI for defamation.
- In such a situation, assuming that the output is repeated and was not a one-time, unfortunate accident, it is a violation of personal rights and in essence, taking legal action against the provider is the only right move - comments Damian Klimas.
However, he emphasizes that the case between Mark Walters and OpenAI is extremely interesting from a legal point of view - the defendants will probably try to hide behind the genericity of information resulting from publicly available sources. A very difficult proceeding in terms of evidence is coming up.
ChatGPT was trained on millions of texts - including business and economic materials. For this reason, his advice on financial and business matters is sensible. The internet was filled with stories of João Ferrão dos Santos, who appointed ChatGPT CEO of his startup, or Jackson Greathouse Fall, who, following the system's advice, founded the Green Gadgets Guru company. However, the use of artificial intelligence does not absolve people of responsibility, even if the system hallucinates misleading hints.
- If anyone in their right mind makes significant business decisions based solely on what AI will generate, they should be prepared for the consequences. It's a bit like expecting Google to be responsible for an incorrect search result - comments Damian Klimas.
Generative AI can be found in tools such as Todoist or Notion, where it assists users in planning tasks or preparing content. But the application of such solutions can go even further. The CEO of the Bengaluru-based company Dukaan laid off 90% of the customer service staff and replaced them with chatbots. He praised this move as a way to cut costs by 85% and significantly speed up problem solving. The average time from reporting to closing the case dropped from 2 hours and 13 minutes to 3 minutes and 13 seconds. However, such systems can also hallucinate and mislead the customer.
– In such situations, a properly constructed agreement would be helpful, but we probably have no influence on it because most of these types of services work as SaaS. In practice, it is better to protect yourself with an algorithm properly trained by us, which will provide the appropriate context. It is also important for a trained specialist to introduce the appropriate prompt - suggests Damian Klimas.
Hallucinations of AI-based systems are "not a flaw, it's a feature" - what's more, it's a feature that will be hard to get rid of from modern AI systems.
– At the workplace, you can regulate the use of genAI tools by introducing employee responsibility for using such systems, which can at least partially protect the business - points out Damian Klimas and emphasizes that AI systems should still be treated with due caution.
Lack of this caution can lead, among other things, to the generation of recipes for mustard gas openly served by the AI system. In a statement sent to the editorial office, nomen omen, Guardian, the PAK‘n’SAVE company expressed "regret that a small group of users decided to use the bot contrary to its purpose".
- We have no influence on the data used by genAI tools, so we have to rely on the quality of the system itself. To avoid unpleasant consequences associated with the use of such tools, it may be necessary to check the correctness and truthfulness of the answer prepared by the system each time - summarizes Damian Klimas.
In 2023 alone, over 84,000 cases of fraud were recorded, resulting in losses of a record 277 million dollars.
How to be a good leader in 2024 and not only survive the upcoming challenges, but also grow in your role?
As the only company in the world, it is able to produce processors in the most advanced 3 nm technology. In addition, it owns the most expensive factory ever built, and its clients are Apple, NVIDIA and Intel. How did it happen that Taiwanese TSMC became one of the most important companies in the world?
Conducting recruitment processes with the participation of people from different cultures requires a large dose of flexibility, empathy, openness, and sensitivity to differences. What can they consist of and how to adapt recruitment processes to them? Get to know the 11D model.
English