Abstract

This study evaluates the performance of extracting data from tables using three large language models (LLMs), namely ChatGPT 4, Custom GPT based on ChatGPT 4, and ChatPDF, in extracting and interpreting quantitative data from tables in financial reports. The models were tested on six questions regarding financial data with varying levels of difficulty using three financial reports from different industries and provided in different formats. The results are compared in terms of accuracy, precision, error rates, and qualitative analysis of the output quality. The results indicate that LLMs have a very limited ability to correctly read and interpret data from tables using annual reports. The study also showed that the same reports including the text yielded better results than the tables alone. The results also indicated that a more specific query can lead to slightly better results. Furthermore, the study suggests that future research should focus on improving the capabilities of LLMs in financial data analysis, including the development of more advanced techniques for data extraction and interpretation.

Statistics