Appier (TSE: 4180), an AI-native company delivering Agentic AI as a Service (AaaS), continues to advance AI innovation and research while expanding the frontiers of marketing and advertising technology. As Agentic AI becomes increasingly embedded in core enterprise operations, Appier’s AI Research team has published two papers examining how large language models (LLMs) can recognize when retrieved information does not support a valid answer and how selecting the appropriate reasoning language can better serve users across linguistic backgrounds — setting a more rigorous benchmark for the trustworthiness and global deployment of enterprise AI.
Both capabilities are increasingly important in real-world business scenarios. Consider an e-commerce service agent: if a shopper asks about a product not covered by an existing return policy, an AI that applies rules for a similar product could provide misleading information and create disputes. Likewise, a game studio expanding overseas may rely on AI to understand local players, but linguistic nuances can shape market preferences and risks. If the AI does not reason in the local language, it may overlook insights that a native speaker would recognize. To support enterprise decision-making effectively, Agentic AI must be able to manage uncertainty and select the reasoning approach and language best suited to each task.
To complete tasks autonomously, Agentic AI typically retrieves information from enterprise knowledge bases, documents, or external sources before reasoning and responding. Retrieval-Augmented Generation (RAG) is a common architecture for this process. However, when retrieval fails to provide sufficient information, the model’s ability to recognize that gap directly affects the reliability of any subsequent decision.
In “None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering,” Appier’s AI Research team used “None of the Above” (NA) options to simulate scenarios in which no valid answer was available. The team tested 28 leading LLMs of varying sizes and found that model accuracy fell by 30% to 50% when “none of the above” was the correct response. The findings show that even when models possess relevant knowledge, they tend to select suboptimal or incorrect options rather than proactively flagging insufficient information.
This capacity to “know what you don’t know” matters most in tasks like business ethics, where models must assess several plausible options holistically, a more complex task than solving problems with directly verifiable answers, such as mathematics.
Appier’s AI Research team applied two training methods—Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO)—to teach models to recognize “none of the above” scenarios. SFT trains models using correct examples, while DPO exposes them to both correct and incorrect responses so they can learn to distinguish between them. DPO improved model accuracy in identifying questions with no correct answer by nearly 30 percentage points, showing that targeted training can strengthen this capability.
However, “none of the above” is not suitable for every question. It is most effective when answers are clearly defined and the options are mutually independent, suggesting that an AI’s ability to withhold an answer should be trained for specific tasks. In enterprise knowledge scenarios, this means improving retrieval accuracy while also introducing a checkpoint to confirm whether sufficient information is available before the AI acts. If the information is insufficient, the system can search again or escalate to a human.