Cognitive bias in large language models (LLMs) undermines their reliability due to the practical limitations of existing mitigation methods. This study introduces a divergence-reflection-convergence (DRC) framework, which leverages a computationally efficient and inference-time strategy designed to identify and mitigate cognitive biases without costly retraining.
This study proposes a novel three-stage DRC framework utilizing a competitive reasoning process where the model first generates divergent viewpoints, subsequently reflects on them to extract a decision rule, and finally converges on a robust output. Furthermore, we validate the DRC framework on five popular LLMs across four common dimensions of cognitive bias, quantifying performance with evaluation metrics including asymmetric kappa (AK) and total variation distance (TVD).
The experimental results demonstrate that LLMs refined by the DRC framework provide more consistent and reliable outputs across all scenarios. Notably, this effect is more pronounced for mitigating internal biases than external biases, suggesting the DRC framework is highly effective at correcting LLMs' flawed internal heuristics. In terms of specific evaluation metrics, the DRC framework successfully transforms all negative or near-zero AK scores into positive values and markedly reduces TVD scores, both of which signify a substantial bias reduction.
This study develops a DRC framework as a novel inference-time mitigation scheme for LLM bias that addresses the practical limitations of retraining-based approaches. Its competitive reasoning process offers a scalable solution for deploying responsible AI. Concurrently, we propose a more nuanced evaluation metric, the AK, tailored for quantifying LLM bias in subjective contexts.
