HCI-SLM - Exemplary comparing energy consumption of selected small language models

This project provides supplementary information for a publication in preparation.

This work serves in two ways:

First, we encourage to question the use of generative language models in the context of sustainability. We scrutinize algorithmic energy consumption to considerate sustainable usage, and shall raise awareness for emission-based model and hardware selection and optimization. Secondly, we present a limited comparison of two small language models that serves as a guide for selecting a small language model that efficiently performs on local hardware at home.

Selected Models for comparison:

TinyLlama (Apache License 2.0), NanoGPT (MIT)

Implementation

We use Python and the library CodeCarbon for all tests, measurements and plots. Code is organized in subfolders respectively. See instructions.txt.

Citation

Publication in preparation

Further information

We'd like to encourage using and also contributing to (S)LM Benchmarks with a focus on sustainability such as SLM-Bench.