LLM_Chat_Bot_App

This repository is the final project for the Udemy Course: The Local LLM Crash Course - Build an AI Chatbot in 2 hours!

On this course we understood a minimal python stack for creating a chatbot with the use of hugging face's text models and chainlit a Python framework for the web application front-end for conversation interactions with an LLM similar to OpenAI's UI for ChatGPT locally.

Screenshot of the application's UI running locally.

Basic Actions.

You can write on the message input on of the following commands.

Model Selection

By default on first run the program will attempt the download of the model Orca Mini 3B GGUF if it is not already on cache and use the cached version on sub sequent runs.

Executing any of the commands above on the chat will change the LLM used for the current session.

Use llama3 will attempt to use the SanctumAI Llama 3.1 8B model but in all the testing I did I was unable to run this project with the use of this las model. I tried my best to use Llama 3.1 but was unable to make it run with ctransformers on my local compputer :c

---

Extra Actions

Installation:

Install by running the ./install.sh script or running the following commands:

python3 -m venv .venv
if [ $? -ne 0 ]; then
    echo 'Error while creating virtual environment with python3 -m venv .venv'
    exit
fi
source .venv/bin/activate
pip3 install -r requirements.txt

Executing project:

You can now run the project locally by running the ./run.sh script or running the command:

chainlit run ___/app.py -w

Environment of the implementation

• OS: Ubuntu 24.04 LTS
• Python 3.12.3
• pip 24.0
• ipdb 0.13.13
• chainlit 1.1.404
• ctransformers 0.2.27

Addendum

Many improvements can be added to the whole project but is intended to be a base for new projects that need to have chatbot like interactions with the user.