LLM+X: A Survey on LLMs for Finance, Healthcare, and Law

A collection of research papers on Large Language Models for Finance, Healthcare, and Law.

Paper: https://arxiv.org/abs/2405.01769

• Finance
• Healthcare
• Law
• Ethics

Finance

Tasks and Datasets in Financial NLP

Sentiment Analysis (SA)

• (Financial Phrase Bank) Good debt or bad debt: Detecting semantic orientations in economic texts JASIST 2014 [Paper]

• (FiQA Task 1) Aspect-based financial sentiment analysis Open Challenge - WWW 2018 [Homepage]

• (TweetFinSent) TweetFinSent: A Dataset of Stock Sentiments on Twitter FinNLP 2022 [Paper][Github]

• (FinSent) Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing EMNLP 2023 [Paper]

• (BloombergGPT sentiment tasks) BloombergGPT: A Large Language Model for Finance Arxiv 2023 [Paper]

Information Extraction (IE)

Named Entity Recognition (NER)

• (FIN3) Domain Adaption of Named Entity Recognition to Support Credit Risk Assessment Proceedings of the Australasian Language Technology Association Workshop 2015 [Paper]

• (BloombergGPT NER tasks) BloombergGPT: A Large Language Model for Finance Arxiv 2023 [Paper]

Relation Extraction (RE)

• (REFinD) REFinD: Relation Extraction Financial Dataset SIGIR 2023 [Paper]

Event Detection

• (EDT) Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading IJCNLP 2021 [Paper]

Question Answering (QA)

• (FiQA Task 2) Opinion-based QA over financial data Open Challenge - WWW 2018 [Homepage]

• (FinQA) FinQA: A Dataset of Numerical Reasoning over Financial Data EMNLP 2021 [Paper][Github]

• (TAT-QA) TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance ACL 2021 [Paper][Github]

• (DocFinQA) DocFinQA: A Long-Context Financial Reasoning Dataset Arxiv 2024 [Paper]

• (ConvFinQA) ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering EMNLP 2022 [Paper][Github]

• (PACIFIC) PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance EMNLP 2022 [Paper][Github]

• (FinanceBench) FinanceBench: A New Benchmark for Financial Question Answering Arxiv 2023 [Paper][Github]

• (Tab-CQA) Tab-CQA: A Tabular Conversational Question Answering Dataset on Financial Reports ACL 2023 [Paper][Github]

Text-Enhanced Stock Movement Prediction (SMP)

• (ACL-14) Using Structured Events to Predict Stock Price Movement: An Empirical Investigation EMNLP 2014 [Paper]

• (ACl-18) Stock Movement Prediction from Tweets and Historical Prices ACL 2018 [Paper][Github]

• (CIKM-18) Hybrid Deep Sequential Modeling for Social Text-Driven Stock Prediction CIKM 2018 [Paper][Github]

• (BigData-22) Accurate stock movement prediction with self-supervised learning from sparse noisy tweets Big Data 2022 [Paper] [Github]

• (Astock) Astock: A New Dataset and Automated Stock Trading based on Stock-specific News Analyzing Model Arxiv 2022 [Paper][Github]

• (EDT) Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading IJCNLP 2021 [Paper]

Other Financial NLP Tasks

Other Classification tasks

News headline classification

• Impact of News on the Commodity Market: Dataset and Results Arxiv 2020 [Paper]

hawkish-dovish classification

• Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis ACL 2023 [Paper][Github]

Multiple classification tasks

• Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing EMNLP 2023 [Paper]

Sentence boundary detection

• Sentence Boundary Detection in PDF Noisy Text in the Financial Domain FinNLP 2019 [Homepage]

NER+NED

• (BloombergGPT NER+NED tasks) BloombergGPT: A Large Language Model for Finance Arxiv 2023 [Paper]

Reasoning

• (BizBench) BizBench: A Quantitative Reasoning Benchmark for Business and Finance Arxiv 2024 [Paper]

Investment opinion generation

• Beyond Classification: Financial Reasoning in State-of-the-Art Language Models Arxiv 2023 [Paper][Github]

Summarization

• (ECTSum) ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts EMNLP 2022 [Paper][Github]

Financial NLP Tasks Under-Explored for LLMs

Financial fraud detection

• Intelligent financial fraud detection: A comprehensive review Computers & Security 2016 [Paper]

Risk assessment and management

• Machine learning and AI for risk management Disrupting Finance, 2018 [Paper]

• A survey on deep learning for financial risk prediction Quantitative Finance and Economics 2021 [Paper]

• Machine learning for financial risk management: a survey IEEE access 2020 [Paper]

Robo advisor

• Artificial Intelligence for Conversational Robo-Advisor ASONAM 2018 [Paper]

• Artificial intelligence in financial services: a qualitative research to discover robo-advisory services Qualitative Research in Financial Markets 2021 [Paper]

• Research on Generative Artificial Intelligence for Virtual Financial Robo-Advisor Academic Journal of Science and Technology 2024 [Paper]

Compliance and regulations

• The Application of Artificial Intelligence in Financial Compliance Management AIAM 2019 [Paper]

• Classifying sentential modality in legal language: a use case in financial regulations, acts and directives ICAIL 2017 [Paper]

Chatbot services

• Ai-based chatbot service for financial industry Fujitsu Scientific and Technical Journal 2018 [Paper]

• Toward a chatbot for financial sustainability Sustainability 2021 [Paper]

• Text-based chatbot in financial sector: A systematic literature review. Data Science in Finance and Economics 2022 [Paper]

Financial LLMs

Pre-Training and Downstream Task Fine-Tuning PLMs.

• (FinBERT-19) FinBERT: Financial Sentiment Analysis with Pre-trained Language Models Arxiv 2019 [Paper][Github][Model]

• (FinBERT-20) Finbert: A pretrained language model for financial communications Arxiv 2020 [Paper][Github][Model]

• (FinBERT-21) Finbert: A pre-trained financial language representation model for financial text mining IJCAI 2020 [Paper]

• (Mengzi-BERTbase-fin) Mengzi: Towards lightweight yet ingenious pre-trained models for chinese Arxiv 2021 [Paper][Github][Model]

• (FLANG) When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain EMNLP 2022 [Paper][Github][Model]

• (BBT-Fin) BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark Arxiv 2023 [Paper][Github][Model]

Pre-training LLMs

• (BloombergGPT) BloombergGPT: A Large Language Model for Finance Arxiv 2023 [Paper]

Instruction Fine-Tuning LLMs.

• (FinMA) PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance Arxiv 2023 [Paper][Github][Model]

• (Instruct-FinGPT) Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models IJCAI 2023 [Paper][Github]

• (CFGPT) CFGPT: Chinese Financial Assistant with Large Language Model Arxiv 2023 [Paper][Github][Model]

• (InvestLM) InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning Arxiv 2023 [Paper][Github][Model]

• (DISC-FinLLM) DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning Arxiv 2023 [Paper][Github][Model]

• (FinGPT) FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets Workshop on Instruction Tuning and Instruction Following at NeurIPS 2023 [Paper][Github]

• (FinGPT) FinGPT: Democratizing Internet-scale Data for Financial Large Language Models Arxiv 2023 [Paper][Github]

• (FinMA-ES) Dólares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English Arxiv 2024 [Paper][Github]

• (FinTral) FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models Arxiv 2024 [Paper]

Evaluation and Analysis

Performance Evaluation and Analysis for Popular Financial Tasks

• Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks EMNLP 2023 [Paper]

• (PIXIU) PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance Arxiv 2023 [Paper][Github][Model]

New Evaluation Frameworks and Tasks

• (FinLMEval) Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing EMNLP 2023 [Paper]

• (Finben) FinBen: A Holistic Financial Benchmark for Large Language Models Arxiv 2024 [Paper]

• (FBI) Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs Arxiv 2024 [Paper]

• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models Arxiv 2023 [Paper]

• Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams Arxiv 2023 [Paper][Github]

LLM-based Methodologies for Financial Tasks and Challenges

Confidentiality and Scarcity of High-Quality Data

• Generating synthetic data in finance: opportunities, challenges and pitfalls ICAIF 2020 [Paper]

• Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models ICAIF 2023 [Paper][Github]

• Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency LREC-COLING 2024 [Paper]

• No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks Arxiv 2024 [Paper][Github]

Quantitative Reasoning

• (ENCORE) Enhancing Numerical Reasoning with the Guidance of Reliable Reasoning Processes Arxiv 2024 [Paper]

• (PoT) Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks TLMR 2023 [Paper][Github]

• (BRIDGE) Exploring Equation as a Better Intermediate Meaning Representation for Numerical Reasoning of Large Language Models AAAI 2024 [Paper][Github]

• (TAT-LLM) TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data Arxiv 2024 [Paper][Github][Model]

• Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering Arxiv 2024 [Paper]

Multimodal Understanding

• (UReader) UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model EMNLP 2023 [Paper][Github]

• (DocLLM) DocLLM: A layout-aware generative language model for multimodal document understanding Arxiv 2023 [Paper][Github]

• (AFIE) Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs Arxiv 2024 [Paper]

• (MANAGER) Modal-adaptive Knowledge-enhanced Graph-based Financial Prediction from Monetary Policy Conference Calls with LLM FinNLP 2024 [Paper][Github]

Healthcare

LLMs for Healthcare

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining [Paper]

• Clinical BERT: Publicly Available Clinical BERT Embeddings [Paper]

• RadBERT: Adapting Transformer-based Language Models to Radiology [Paper]

• BioMed RoBERTa: Don't Stop Pretraining: Adapt Language Models to Domains and Tasks [Paper]

• MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data [Paper]

• HuatuoGPT, towards Taming Language Model to Be a Doctor [Paper]

• Me LLaMA: Foundation Large Language Models for Medical Applications [Paper]

• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day [Paper]

• BioInstruct: instruction tuning of large language models for biomedical natural language processing [Paper]

• Biomedgpt: Open multimodal generative pre-trained transformer for biomedicine [Paper]

• MEDITRON-70B: Scaling Medical Pretraining for Large Language Models [Paper]

• PMC-LLaMA: toward building open-source language models for medicine [Paper]

• BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains [Paper]

• AlpaCare:Instruction-tuned Large Language Models for Medical Application [Paper]

• Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding [Paper]

Miscellaneous

• MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation [Paper]

• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [Paper]

• The imperative for regulatory oversight of large language models (or generative AI) in healthcare [Paper]

Law

LLMs for Law

• LEGAL-BERT: The muppets straight out of law school [Paper]

• Lawformer: A pre-trained language model for chinese legal long documents [Paper]

• Juru: Legal Brazilian Large Language Model from Reputable Sources [Paper]

• Sabiá-2: A New Generation of Portuguese Large Language Models [Paper]

• Baichuan 2: Open large-scale language models [Paper]

• LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model [Paper]

• Saullm-7b: A pioneering large language model for law [Paper]

• Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language Model [Paper]

• Lawyer LLaMA: Enhancing LLMs with Legal Knowledge [Paper]

Evaluation

• LawBench: Benchmarking Legal Knowledge of Large Language Models [Paper]

• LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [Paper]

• LAiW: A Chinese Legal Large Language Models Benchmark [Paper]

Ethics

Ethics in LLM + Finance

• Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination ICBINB 2023 [Paper]

• Hallucination-minimized Data-to-answer Framework for Financial Decision-makers BigData 2023 [Paper]

• Beavertails: Towards Improved Safety Alignment of LLM via a Human-preference Dataset NeurIPS 2024 [Paper]

Ethics in LLM + Healthcare

• Large Language Models in Medicine Nature medicine 2023 [Paper]

• Ethics of Large Language Models in Medicine and Medical Research The Lancet Digital Health 2023 [Paper]

• AI and Ethics: A Systematic Review of the Ethical Considerations of Large Language Model Use in Surgery Research Healthcare 2024 [Paper]

• The Ethics of ChatGPT in Medicine and Healthcare: A Systematic Review on Large Language Models (LLMs) npj Digital Medicine [Paper]

• Embracing Large Language Models for Medical Applications: Opportunities and Challenges Cureus 2023 [Paper]

• The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models JAMA 2023 [Paper]

• Leveraging Generative AI and Large Language Models: a Comprehensive Roadmap for Healthcare Integration Healthcare 2023 [Paper]

• Challenges and Barriers of Using Large Language Models (LLM) such as ChatGPT for Diagnostic Medicine with a Focus on Digital Pathology–a Recent Scoping Review Diagnostic pathology 2024 [Paper]

• Ethical Dilemmas, Mental Health, Artificial Intelligence, and LLM-based Chatbots IWBBIO 2023 [Paper]

Ethics in LLM + Law

• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models Journal of Legal Analysis 2024 [Paper]

• I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice ACM FAccT 2024 [Paper]

• Evaluation Ethics of LLMs in Legal Domain ArXiv preprint 2024 [Paper]