🧠 Why Jarvis?

Every AI assistant wants your data in the cloud. Jarvis doesn't.

• 🔒 Totally private — runs on local models via Ollama. No accounts, no API keys, no telemetry. Your conversations, screen, camera, files, and memories never leave your machine.
• 🎙️ Voice-first — say "Jarvis…" and it wakes up (with a chime), listens with offline Whisper, and replies in a human neural voice. Have a real back-and-forth conversation, hands-free.
• 👁️ It can actually see — reads your screen with Apple's OCR, watches through your camera, and answers questions about whatever's in front of you.
• 🌐 It knows what's going on — researches the web, reads the best pages and answers with numbered sources; live weather without an API key.
• 🦾 It does things — opens apps, controls your browser, plays anything on Apple Music, switches your smart lights, makes FaceTime calls, takes photos, drafts emails, manages files, runs code — with a permission prompt before anything touches your system.
• 📱 It's in your pocket — message it on Telegram, and "send that to me" puts any answer, chat or document on your phone.
• 🧩 It's an OS, not a chatbot — a living command-center dashboard with agents, agents and teams of agents you build on a canvas and talk to by name, projects, long-term memory, and a plugin system it can extend itself.
• 🆓 Free & source-available — free for any noncommercial use. Fork it, hack it, make it yours.

📸 Screenshots

A team with a whole IT team and a Writer linked in	A single agent's finished job, with the file it saved

Routines — things it does on its own	Setup check — every permission, one click to fix

Long-term Memory	Custom Skills

Smart lights in Settings	⌘K — jump anywhere, or just ask

✨ What it can do

🎨 Make it yours

Themes (hacker green, dark blue, cyberpunk, light), four Core designs, your own accent colour or picture — and a glyph field background: a quiet field of characters with waves of light drifting through it, rippling out from wherever you click.

🖥️ Menu-bar Quick Command

Hit ⌥Space from any app to summon a floating command bar — ask about what's on your screen, get an answer, and get back to work without switching windows. Inside Jarvis, ⌘K jumps to any page, chat or action — or just asks.

☀️ Daily Briefing

"Jarvis, give me my daily briefing" → live weather, today's calendar, and your reminders, narrated.

📱 Text Jarvis from your phone

Message Jarvis on Telegram from anywhere. It does the work on your Mac and replies in the chat:

• "Send me the lease PDF from my Downloads" → finds the file and sends it to you
• "Research the best budget e-bikes and send me a Word doc" → searches, writes a .docx with sources, sends it
• 🎙 Voice notes in, voice notes back · send it files or photos to ask about · approve risky actions with one tap
• "Send that to me" at the Mac → the answer you just got arrives on your phone · "send me the whole chat" · "put that in a document and send it to me" → a Word file in Telegram
• A message cut off by a restart is offered back, so nothing you asked for gets lost

Two-minute setup — see Phone setup.

🚀 Quick Start

# 1. Python 3.11+  —  macOS's built-in python3 is 3.9, which can't run Jarvis
brew install [email protected]          # skip if `python3 --version` already says 3.11+

# 2. Install Ollama and pull a model (https://ollama.com/download)
ollama pull qwen2.5:14b

# 3. Clone and run everything with one command
git clone https://github.com/Salahasfan02/Jarvis.git
cd Jarvis
chmod +x start.sh && ./start.sh

start.sh finds a Python 3.11+ on its own, installs everything on the first run (and again whenever a git pull changes the dependencies), and retries an install that was interrupted.

Your browser opens at http://localhost:5173 and Jarvis is live. Press Ctrl+C (or ./stop.sh) to turn it off. Nothing auto-starts; it only runs when you start it.

🔧 Your model is picked for you. If the configured model (llama3.1 by default) isn't installed, Jarvis switches to one you have — qwen2.5:14b above. Change it anytime in Settings → Model.

🔐 macOS will ask for permissions. The first time Jarvis reads your screen, uses the camera, reads your calendar, or drives another app, macOS prompts for Screen Recording, Camera, Microphone, Calendars, Automation (Mail, Safari, Music) and Accessibility. Grant them in System Settings → Privacy & Security. Email features use the Mail app, so add your accounts (iCloud, Gmail, Outlook…) there.

./start.sh          # terminal 1 — leave this running

cd desktop          # terminal 2
npm install
npm start           # run it from the repo, or:
npm run install-app # build Jarvis.app, put it in Applications and open it

# Terminal 1 — Ollama
ollama serve

# Terminal 2 — backend   (use an explicit 3.11+ interpreter, not bare `python3`)
cd backend && /opt/homebrew/bin/python3.12 -m venv .venv && .venv/bin/pip install -r requirements.txt
.venv/bin/python run.py            # → http://127.0.0.1:8765

# Terminal 3 — UI
cd frontend && npm install && npm run dev   # → http://localhost:5173

Level up (optional, all local)

ollama pull nomic-embed-text   # semantic memory + smarter agent routing
ollama pull llama3.2:3b        # a small, fast helper for titles and background work
ollama pull qwen3.5            # camera & screen vision

Then open Settings to enable them, download the offline Whisper and human Kokoro voice models, pick your theme, and set your wake word.

🖥️ Install as a Mac app

cd desktop && npm run install-app builds Jarvis.app (with its own icon), signs it locally and installs it in Applications. It starts Ollama if needed, only ever runs once, and can open at login (Settings → Desktop app). It's a new app to macOS, so allow the microphone, camera, Accessibility, Screen Recording, Calendars and Reminders once — Setup check lists anything still missing, with a fix button for each (and a restart button when macOS needs one).

The installer signs with a certificate it makes once on your Mac (Jarvis Local Signing), so the permissions you grant survive every rebuild — no re-allowing after an update.

📱 Phone setup (Telegram)

1. In Telegram, message @BotFather, send /newbot, pick a name, and copy the token it gives you.
2. In Jarvis, open Settings → Phone (Telegram), paste the token, and press Connect.
3. Send the /pair 123456 code shown there to your new bot from your phone — or press Open in Telegram if Telegram is on your Mac. That's it.

Things to know:

• Nothing is exposed to the internet. Jarvis polls Telegram with outbound requests only — no port forwarding, tunnel, or public URL. Your Mac must be awake with Jarvis running (on power, enable Prevent automatic sleeping in System Settings → Battery/Energy).
• Only paired accounts get answers; everyone else is ignored. Anything that needs permission (sending a file, reading your messages…) asks with Allow / Deny buttons, or reply yes / no. Dangerous tools (shell, AppleScript, deleting files) are blocked from the phone unless you switch them on.
• Privacy trade-off: Telegram bot chats aren't end-to-end encrypted, so what you send and receive passes through Telegram's servers. The feature is off until you set it up.
• Telegram bots can't take phone calls — voice notes are the hands-free way in (needs the Whisper model; voice replies need Kokoro). Files: up to 50 MB to your phone, 20 MB from it.
• Word documents are saved in ~/Documents/Jarvis; files you send Jarvis land in ~/Downloads/Jarvis. /new starts a fresh conversation, /stop cancels.
• Once paired, "send that to me" at the Mac goes to this chat too.

💡 Smart lights setup (Tapo / Kasa)

1. Make sure your Mac is on the same Wi-Fi as the lights.
2. In Jarvis, open Settings → Smart lights, type your Tapo account email and password, and press Connect. Jarvis finds every light on the network and lists it with a switch, brightness and colours.
3. Say "lights off" — or put the lights in a routine.

Things to know:

• Your password isn't saved. It's used once to connect; Jarvis keeps only a key for each light (~/.jarvis/lights.json, readable only by you), which can't sign in to your Tapo account.
• Jarvis talks to the lights directly on your Wi-Fi, so Alexa and the Tapo app keep working. Cameras and other Tapo devices are left alone.
• If a light moves to a new address on your network, Jarvis finds it again by itself.
• Newer Tapo firmware only lets other apps in when Third-Party Compatibility is on in the Tapo app (Me → Tapo Lab, or Me → Third-Party Services). If Jarvis says your light didn't accept a correct password, turn that on.

🔧 Troubleshooting

brew install [email protected]
rm -rf backend/.venv && ./start.sh

📋 Requirements

💡 Model tip: qwen2.5:14b is the sweet spot for tool-calling on a 16–24 GB Mac. The app is fully model-agnostic — switch models anytime in Settings, and even benchmark them on your hardware.

🔐 Security & Privacy

Jarvis is built to earn your trust:

• Runs 100% locally — the AI never leaves your Mac. The only things that go online are what you ask for: web research and weather (DuckDuckGo/Bing, the pages they find, Open-Meteo), song lookups for Apple Music (Apple's search API), and Telegram if you turn on the phone channel. Smart lights are controlled on your own Wi-Fi, not through the cloud.
• No passwords kept — your Tapo password is used once and forgotten; Jarvis keeps only a per-light key. The Telegram token is stored readable only by you.
• Permission system — every action has a risk level. Safe actions run instantly; anything that touches your system (shell, AppleScript, deleting files) asks first, every time.
• Audit log — every tool call, confirmation, and denial is recorded.
• Nothing is sent behind your back — iMessages are only drafted; an email goes out only when you say send and approve that exact message (this approval can't be switched off).
• Websites can't drive it — the local API only accepts changes and chat connections from Jarvis's own UI, so a page you visit can't talk to Jarvis behind your back.
• Sandboxed code — generated code runs with no network, can't read your keys, mail or messages, can't script other apps, and can only write to its own folder.
• Agents and teams stay in their lane — they can only research the web and write files into their run's own workspace; the pages they publish are sandboxed too, so nothing they make can reach Jarvis or the rest of your Mac.
• Calls show you who first — before Jarvis rings anyone you see the person and the exact number it found; calls never start from the phone or a routine, when you're not at the Mac to talk.

🏗️ Tech Stack

Backend: FastAPI · Ollama · SQLite · faster-whisper (STT) · Kokoro (TTS) · Apple Vision (OCR) · pyobjc · trafilatura (reading web pages) · python-kasa (Tapo / Kasa lights) · Open-Meteo (weather) Frontend: React · TypeScript · Vite · Vitest · Lucide icons (ISC) Desktop: Electron (tray, global hotkeys, floating window), signed locally so permissions persist

Modular by design — new tools, agents, models, and integrations drop in without touching the core. See docs/architecture.md.

🗺️ Roadmap

• Encrypted memory & conversation storage
• First-run setup wizard (auto-pulls the right models)
• Self-contained .app (bundled backend)
• Meeting transcription with speaker separation
• Git integration in the coding workspace
• Community skill & plugin packs

🤝 Contributing

PRs, ideas, and bug reports are welcome. Jarvis is designed to be extended — writing a new tool or plugin is a few lines of Python (see docs/architecture.md).

Run the tests (no Ollama needed):

cd backend && .venv/bin/pip install pytest && .venv/bin/python -m pytest   # 240+ backend tests, any OS
cd frontend && npm test                                                    # interface helpers (Vitest)

Check how well a model handles real requests (needs Ollama; tools are faked, nothing on your Mac is touched): cd backend && .venv/bin/python -m evals.run --model qwen2.5:14b — prints pass/fail and timing for nearly 100 requests like "email my CV to…", "send the weather you just gave me to my phone", "make the bedroom light blue" and "facetime mum".

⭐ Star it

If Jarvis is useful or just fun to poke at, drop a star — it helps other people find it and keeps the project going.

☕ Support

Jarvis is free and built in my spare time. If it saved you time or made you smile, you can buy me a coffee — it fuels the next late-night feature. Thank you! 🙏

📄 License

PolyForm Noncommercial 1.0.0 — free for any noncommercial use: personal projects, hobby, research, education, and nonprofits. You can use, modify, and share it freely — just not sell it or use it commercially. Commercial use requires a separate license from the author.