South Korean researchers have developed a new method that enables AI models to acknowledge their unfamiliarity with certain topics, mimicking human-like behavior. This innovation could significantly enhance the reliability of AI systems used in critical areas such as autonomous driving and medicine, according to scientists from the Korea Advanced Institute of Science and Technology.
Previous studies have highlighted the issue of AI overconfidence as a major risk when these tools are used for decision-making, particularly in fields like medical diagnosis. Common AI models, such as OpenAI’s ChatGPT, have been shown to “hallucinate” or fabricate facts because they are designed to make guesses rather than admit when they lack knowledge.
Now, researchers have created a technique that allows AI to recognize situations where it lacks familiarity or has not encountered specific information. This advancement could improve the overall reliability of chatbots and other AI applications.
The primary cause of AI overconfidence is how these models learn from initial data using artificial neural networks, which form the foundation of their infrastructure. Small errors during this phase can accumulate and lead to significant mistakes if not addressed.
Researchers discovered that when random data was introduced into a neural network during the initialization stage, the model displayed high confidence despite having learned nothing. This resulted in “hallucination.”
To tackle this issue, researchers looked to the way the human brain handles uncertainty. In humans, brain signals are generated without external input even before birth, helping manage unknowns. Inspired by this, scientists developed a system where the neural network backbone of an AI model undergoes brief pre-training with random noise inputs before actual learning begins.
This warm-up process helps AI establish a baseline for itself by adjusting its own uncertainty before starting data learning. The warm-up can help an AI model set its initial confidence to a low level close to chance, significantly reducing its overconfidence bias.
In other words, the method helps models first understand the state of “I don’t know anything yet.” Researchers explained that while conventional models tend to give incorrect answers with high confidence for data they haven’t encountered during training, models with warm-up training showed a clear improvement in their ability to lower confidence and recognize that they “do not know.”
This capability can help AI distinguish between what it knows and what it does not know. Se-Bum Paik, an author of the study published in the journal Nature Machine Intelligence, said, “This study demonstrates that by incorporating key principles of brain development, AI can recognize its own knowledge state in a way that is more similar to humans. This is important because it helps AI understand when it is uncertain or might be mistaken, not just improve how often it gives the right answer.”
Independent readers are independent-minded global citizens. They are not defined by traditional demographics or profiles, but by their attitudes. In today’s increasingly fragmented world, communities value real facts and frank opinions delivered first-hand from a non-biased news brand that they can trust. Armed with information and inspiration, Independent readers are empowered and equipped to take a stand for the things they believe in.
