ISSN :2582-9793

A Comparative Study in Large Language Models Usage for Fake News Detection

Original Research (Published On: 10-Nov-2024 )
A Comparative Study in Large Language Models Usage for Fake News Detection

REPEDE STEFAN EMIL

Adv. Artif. Intell. Mach. Learn., 4 (4):2810-2823

REPEDE STEFAN EMIL : "Lucian Blaga" University of Sibiu

Download PDF Here

Article History: Received on: 06-Sep-24, Accepted on: 03-Nov-24, Published on: 10-Nov-24

Corresponding Author: REPEDE STEFAN EMIL

Email: stefan.repede@gmail.com

Citation: REPEDE STEFAN EMIL, Remus BRAD. (ROMANIA) (2024). A Comparative Study in Large Language Models Usage for Fake News Detection. Adv. Artif. Intell. Mach. Learn., 4 (4 ):2810-2823.


Abstract

    

This study presents a comparative analysis of the capabilities of Large Language Models (LLMs) in the automatic detection of fake news. The research focuses on evaluating the accuracy, precision, recall and F1-score of three state-of-the-art LLMs—OpenAI's ChatGPT 4.0 beta, Meta's Llama 3.1, and Google's Gemini—using an extensive dataset of verified true and fake news. By employing a black-box testing method, the study categorizes the LLMs' outputs and assesses their performance based on accuracy metrics. The results indicate moderate success in distinguishing between true and false news, with differences noted between the models and smaller, text classification specialized Natural Language Models (NLP), like Google`s Bidirectional Encoder Representations from Transformers (BERT) model variants, trained on the same fake news dataset. The findings demonstrate the potential of LLMs as tools for combating misinformation, while also emphasizing the current limitations and the need for improvements in their accuracy and reliability. This paper provides insights into the challenges of utilizing LLMs for misinformation detection and highlights the importance of combining technological advancements with our distinct human cognition.

Statistics

   Article View: 179
   PDF Downloaded: 12