The code in this repository is under constant revision/refactoring. You could say that I am learning Swift as I develop LLMFarm. So don't expect too much from the code you'll find here. If you have any useful comments on the code, its style or architecture, I will be glad to hear them.

LLMFarm_core.swift

LLMFarm_core swift library to work with large language models (LLM). It allows you to load different LLMs with certain parameters.
Based on ggml and llama.cpp by Georgi Gerganov.

Features

• MacOS (13+)
• iOS (16+)
• Various inferences
• Various sampling methods
• Metal (dont work on intel Mac)
• Model setting templates
• LoRA adapters support (read more)
• LoRA train support
• LoRA export as model support
• Restore context state (now only chat history)

Inferences

See full list here.

Sampling methods

• Temperature (temp, tok-k, top-p)
• Tail Free Sampling (TFS)
• Locally Typical Sampling
• Mirostat
• Greedy
• Grammar
• Classifier-Free Guidance

Installation

git clone https://github.com/guinmoon/llmfarm_core.swift

Swift Package Manager

Add llmfarm_core to your project using Xcode (File > Add Packages...) or by adding it to your project's Package.swift file:

dependencies: [
  .package(url: "https://github.com/guinmoon/llmfarm_core.swift")
]

Build and Debug

To Debug llmfarm_core package, do not forget to comment .unsafeFlags(["-Ofast"]) in Package.swift. Don't forget that the debug version is slower than the release version.

Usage

See examples in the Demo Project

Also used sources from:

• rwkv.cpp by saharNooby.
• Mia by byroneverson.
• swift-markdown-ui by gonzalezreal

Projects based on this library

• LLM Farm

App to run LLaMA and other large language models locally on iOS and MacOS.