Requirements

• CUDA-enabled GPU
• Python 3.8+

Setup Instructions

Running Locally with Docker

1. Clone the repository:

   git clone https://github.com/WoSea/bloomz-1b1-api.git
   cd bloomz-1b1-api
   

2. Build the Docker image:

   docker build -t bloomz-1b1-api .
   

3. Run the Docker container:

   docker run --gpus all -p 8000:8000 bloomz-1b1-api
   

4. Interact with the API:

   • Generate text:

   curl -X POST "http://localhost:8000/llm" -H "Content-Type: application/json" -d '{"prompt": "Translate to English: Je t’aime."}'
   

   • Check API health:

   curl http://localhost:8000/alive
   

API Documentation

api_url/docs