Built for field workers, logistics, gig economy, and mobile-first industries
Watch it automate a logistics workflow in 60 seconds
Driver texts a photo → Agent handles WhatsApp → Scanner app → Banking app → Invoice submitted
⭐ Star this repo (1100+ → 1,500 goal!) • Quick Start • Book Partnership Meeting
5.3M+ views. 1100+ stars in days. Help us reach 1,500! • Priority partnerships: Mobile QA testing • Consumer Productivity • Request meeting →
Browser agents only work on websites. Computer Use requires a desktop.
But the real economy runs on mobile devices, in places where laptops don't fit:
3 billion Android devices. $40 trillion in GDP from mobile-first workflows. Zero AI agent solutions that actually work on these devices.
Priority partnership area. Android Use automating an entire logistics workflow:
1. Driver takes photo of Bill of Lading
2. Opens WhatsApp, sends to back office
3. Back office downloads image
4. Opens banking app, fills invoice form
5. Uploads documents
6. Submits for payment
# Driver just texts the photo. Agent does the rest.
run_agent("""
1. Get latest image from WhatsApp
2. Open native scanner app and process it
3. Switch to RTS Pro factoring app
4. Fill invoice form with extracted data
5. Upload PDF and submit for payment
""")
Result: Driver gets paid same day instead of waiting weeks. Back-office work eliminated. No laptop needed.
Computer Use (Anthropic)
|
Android Use (This Library)
|
The breakthrough: Android's accessibility API provides structured UI data (buttons, text, coordinates) without expensive vision models.
Real impact: 95% cost savings + 5x faster + works where laptops can't.
Launched with the logistics demo:
Star growth shows real demand. Help us reach 1,500 stars → Star this repo now
Current priority partnerships:
Due to overwhelming demand, we created a meeting scheduler. Request a partnership meeting →
| Industry | Why They Need This | Market Size | Current State |
|---|---|---|---|
| Logistics | Drivers use factoring apps (RTS Pro, OTR Capital) in truck cabs | $10.5T | Manual, no laptop access |
| Gig Economy | Uber/Lyft/DoorDash drivers optimize between apps on phones | $455B | Tap manually, lose 20% earnings |
| Last-Mile Delivery | Amazon Flex, UPS drivers scan packages on handhelds | $500B+ | Proprietary apps, no APIs |
| Field Services | Techs log work orders on tablets on-site | $200B+ | Mobile-only workflows |
| Mobile Banking | Treasury ops, reconciliation on native banking apps | $28T | 2FA + biometric locks |
Total: $40+ trillion in GDP from mobile-first workflows
Browser agents can't reach these. Desktop agents don't fit. Android Use is the only solution.
# 1. Clone the repo
git clone https://github.com/actionstatelabs/android-action-kernel.git
cd android-action-kernel
# 2. Install dependencies
pip install -r requirements.txt
# 3. Setup ADB
brew install android-platform-tools # macOS
# sudo apt-get install adb # Linux
# 4. Connect device & verify
adb devices
# 5. Set API key
export OPENAI_API_KEY="sk-..."
# 6. Run your first agent
python kernel.py
from kernel import run_agent
# Automate the workflow from the viral demo
run_agent("""
Open WhatsApp, get the latest image,
then open the invoice app and fill out the form
""")
Other examples:
"Accept the next DoorDash delivery and navigate to restaurant""Scan all packages and mark them delivered in the driver app""Check Chase mobile for today's transactions"Problem: Drivers lose 20%+ earnings manually switching between DoorDash, Uber Eats, Instacart.
run_agent("Monitor all delivery apps, accept the highest paying order")
Impact: Instant acceptance of best orders. Drivers report 20-30% earnings increase by optimizing across platforms.
Problem: Drivers manually scan 200+ packages/day in proprietary apps.
run_agent("Scan all packages in photo and mark as loaded in Amazon Flex")
Impact: Scan 200+ packages in seconds vs. 20+ minutes manually. Eliminates data entry errors.
Problem: Treasury teams reconcile transactions across multiple mobile banking apps.
run_agent("Log into Chase mobile and export today's wire transfers")
Impact: Automate daily reconciliation. Process 1000+ transactions in minutes vs. hours of manual work.
Problem: Staff extract patient data from HIPAA-locked mobile portals.
run_agent("Open Epic MyChart and download lab results for patient 12345")
Impact: Extract patient data from HIPAA-locked portals. Automate appointment booking and records management.
Problem: Manual testing of Android apps is slow and expensive.
run_agent("Create account, complete onboarding, make test purchase")
Impact: 10x faster than manual QA. Full E2E regression tests in CI/CD pipeline.
Priority partnership area. If you're a QA team looking to automate mobile testing, request a meeting.
┌─────────────────────────────────────────────────────┐
│ Goal: "Get image from WhatsApp, submit invoice" │
└─────────────────────────────────────────────────────┘
↓
┌────────────────────────────────────┐
│ 1. PERCEPTION │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ $ adb shell uiautomator dump │
│ │
│ Accessibility Tree (XML): │
│ <Button text="Download Image" │
│ bounds="[100,500][300,600]"│
│ clickable="true" /> │
│ │
│ Parsed to JSON: │
│ {"text": "Download Image", │
│ "center": [200, 550], │
│ "clickable": true} │
└────────────────────────────────────┘
↓
┌────────────────────────────────────┐
│ 2. REASONING (GPT-4) │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ Prompt: "Goal: Get WhatsApp image"│
│ "Screen: [Download Image button]" │
│ │
│ GPT-4 Response: │
│ { │
│ "action": "tap", │
│ "coordinates": [200, 550], │
│ "reason": "Download the image" │
│ } │
└────────────────────────────────────┘
↓
┌────────────────────────────────────┐
│ 3. ACTION (ADB) │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ $ adb shell input tap 200 550 │
│ │
│ → Image downloaded! │
└────────────────────────────────────┘
↓
Repeat until done
| Approach | Cost | Speed | Accuracy | Works on Device |
|---|---|---|---|---|
| Screenshots (Computer Use) | $0.15/action | 3-5s | 70-80% | Desktop only |
| Accessibility Tree (Android Use) | $0.01/action | <1s | 99%+ | Handheld devices |
Technical advantage: Accessibility tree provides structured data (text, coordinates, hierarchy) without image encoding/OCR.
kernel.py (131 lines)
├── get_screen_state() # Dump & parse accessibility tree
│ └── sanitizer.py # XML → JSON (54 lines)
├── get_llm_decision() # GPT-4 reasoning
└── execute_action() # ADB commands
├── tap (x, y)
├── type "text"
├── home / back
└── done
Total core logic: <200 lines. Simple, hackable, extensible.
from kernel import run_agent
run_agent(
goal="Open WhatsApp and download the latest image",
max_steps=10 # Max actions before timeout
)
# Tap coordinates
{"action": "tap", "coordinates": [540, 1200]}
# Type text
{"action": "type", "text": "Invoice #12345"}
# Navigate
{"action": "home"} # Home screen
{"action": "back"} # Previous screen
# Wait/Complete
{"action": "wait"} # Wait for loading
{"action": "done"} # Goal achieved
from kernel import get_screen_state
screen_json = get_screen_state()
# Returns: [{"text": "Submit", "center": [540, 1200], ...}]
pip install android-useDon't want to host it yourself? Join the waitlist for our managed Cloud API.
What you get:
Priority access for: Trucking/logistics companies and QA testing teams. Request a partnership meeting or join the general waitlist (Coming Q1 2026)
Want to help build the future of mobile AI agents?
git checkout -b feature/factoring-app-supportgit commit -m 'Add RTS Pro integration'git push origin feature/factoring-app-supportSpecial focus: If you work in logistics, gig economy, or field services, your domain expertise is invaluable!
5.3 million views. 12 → 1100+ stars in days. This growth validates the market need.
Help us hit 1,500 stars to show investors/partners there's real demand for mobile AI agents!
⭐ Click here to star this repo now →
Star Progress: ████████████████░░░░░░░░░░░░ 1100+ / 1,500 stars (63% there!)
Every star helps logistics companies and QA teams discover this solution + validates demand for funding
|
⭐ Star on GitHub Support the project |
Book Partnership Trucking/logistics or QA testing |
Share on X Help others find this |
Join Waitlist Cloud API access |
Built by Ethan Lim - AI engineer
I spent a week riding along with truck drivers for a logistics automation project. Sitting in the cab, I watched one driver spend 10 minutes after every delivery manually typing data from a Bill of Lading photo into the RTS Pro factoring app on his phone.
"I do this 15 times a day. Can't use a laptop—there's no room in the cab, no internet half the time, and I'm already holding the phone to take the photo anyway. This app doesn't have an API. It's just me and this phone."
That's when it clicked: The world has AI agents for web browsers and desktop computers, but 3 billion people work on mobile devices that existing agents can't touch.
I looked at existing solutions:
Neither solved the truck cab problem. So I built Android Use using Android's accessibility API instead of expensive vision processing.
The technical bet:
Posted the demo showing a full logistics workflow. 5.3 million views. 150+ inbound messages from companies. 1100+ GitHub stars. The market validated the need immediately and continues to grow.
Based on inbound demand, the path forward is clear:
Business model: Freemium. Open-source core is free forever. Revenue from managed hosting ($0.02-0.05/action) and enterprise features ($5-10k/month for 100+ seat deployments). Target customers: logistics companies (50+ drivers), delivery platforms (1000+ workers), field service orgs.
Defensibility: Network effects from app-specific agent templates. First-mover advantage in mobile-first verticals. Direct partnerships with factoring companies and gig platforms create distribution moats.
Vision: Make AI agents work where 3 billion people actually work—on mobile devices.
→ Request a meeting for partnerships - We created this due to overwhelming demand
Actively seeking:
Already running pilots with 5 trucking/delivery companies.
Since launch:
Market data:
The star growth from 12 → 1100+ validates real demand. Help us reach 1,500 → Star now
MIT License - see LICENSE
Free for personal and commercial use. Build on it, sell services with it, integrate it into your logistics platform.
Why MIT? We want this to become the standard for mobile AI agents. Open source = faster adoption.
Built on:
Special thanks to:
Whether you're in a truck cab, on a delivery route, or in the field...
AI agents should work where you do.
⭐ STAR THIS REPO (Help us reach 1,500!) • WATCH DEMO (5.3M+ views) • SHARE IT
git clone https://github.com/actionstatelabs/android-action-kernel.git
cd android-action-kernel
pip install -r requirements.txt
python kernel.py
Made for field workers • Star • Follow @ethanjlim • Share
53,286 followers · starred Jan 2026
108 followers · starred Dec 2025
760 followers · starred Dec 2025
111 followers · starred Dec 2025
Built for field workers, logistics, gig economy, and mobile-first industries
Watch it automate a logistics workflow in 60 seconds
Driver texts a photo → Agent handles WhatsApp → Scanner app → Banking app → Invoice submitted
⭐ Star this repo (1100+ → 1,500 goal!) • Quick Start • Book Partnership Meeting
5.3M+ views. 1100+ stars in days. Help us reach 1,500! • Priority partnerships: Mobile QA testing • Consumer Productivity • Request meeting →
Browser agents only work on websites. Computer Use requires a desktop.
But the real economy runs on mobile devices, in places where laptops don't fit:
3 billion Android devices. $40 trillion in GDP from mobile-first workflows. Zero AI agent solutions that actually work on these devices.
Priority partnership area. Android Use automating an entire logistics workflow:
1. Driver takes photo of Bill of Lading
2. Opens WhatsApp, sends to back office
3. Back office downloads image
4. Opens banking app, fills invoice form
5. Uploads documents
6. Submits for payment
# Driver just texts the photo. Agent does the rest.
run_agent("""
1. Get latest image from WhatsApp
2. Open native scanner app and process it
3. Switch to RTS Pro factoring app
4. Fill invoice form with extracted data
5. Upload PDF and submit for payment
""")
Result: Driver gets paid same day instead of waiting weeks. Back-office work eliminated. No laptop needed.
Computer Use (Anthropic)
|
Android Use (This Library)
|
The breakthrough: Android's accessibility API provides structured UI data (buttons, text, coordinates) without expensive vision models.
Real impact: 95% cost savings + 5x faster + works where laptops can't.
Launched with the logistics demo:
Star growth shows real demand. Help us reach 1,500 stars → Star this repo now
Current priority partnerships:
Due to overwhelming demand, we created a meeting scheduler. Request a partnership meeting →
| Industry | Why They Need This | Market Size | Current State |
|---|---|---|---|
| Logistics | Drivers use factoring apps (RTS Pro, OTR Capital) in truck cabs | $10.5T | Manual, no laptop access |
| Gig Economy | Uber/Lyft/DoorDash drivers optimize between apps on phones | $455B | Tap manually, lose 20% earnings |
| Last-Mile Delivery | Amazon Flex, UPS drivers scan packages on handhelds | $500B+ | Proprietary apps, no APIs |
| Field Services | Techs log work orders on tablets on-site | $200B+ | Mobile-only workflows |
| Mobile Banking | Treasury ops, reconciliation on native banking apps | $28T | 2FA + biometric locks |
Total: $40+ trillion in GDP from mobile-first workflows
Browser agents can't reach these. Desktop agents don't fit. Android Use is the only solution.
# 1. Clone the repo
git clone https://github.com/actionstatelabs/android-action-kernel.git
cd android-action-kernel
# 2. Install dependencies
pip install -r requirements.txt
# 3. Setup ADB
brew install android-platform-tools # macOS
# sudo apt-get install adb # Linux
# 4. Connect device & verify
adb devices
# 5. Set API key
export OPENAI_API_KEY="sk-..."
# 6. Run your first agent
python kernel.py
from kernel import run_agent
# Automate the workflow from the viral demo
run_agent("""
Open WhatsApp, get the latest image,
then open the invoice app and fill out the form
""")
Other examples:
"Accept the next DoorDash delivery and navigate to restaurant""Scan all packages and mark them delivered in the driver app""Check Chase mobile for today's transactions"Problem: Drivers lose 20%+ earnings manually switching between DoorDash, Uber Eats, Instacart.
run_agent("Monitor all delivery apps, accept the highest paying order")
Impact: Instant acceptance of best orders. Drivers report 20-30% earnings increase by optimizing across platforms.
Problem: Drivers manually scan 200+ packages/day in proprietary apps.
run_agent("Scan all packages in photo and mark as loaded in Amazon Flex")
Impact: Scan 200+ packages in seconds vs. 20+ minutes manually. Eliminates data entry errors.
Problem: Treasury teams reconcile transactions across multiple mobile banking apps.
run_agent("Log into Chase mobile and export today's wire transfers")
Impact: Automate daily reconciliation. Process 1000+ transactions in minutes vs. hours of manual work.
Problem: Staff extract patient data from HIPAA-locked mobile portals.
run_agent("Open Epic MyChart and download lab results for patient 12345")
Impact: Extract patient data from HIPAA-locked portals. Automate appointment booking and records management.
Problem: Manual testing of Android apps is slow and expensive.
run_agent("Create account, complete onboarding, make test purchase")
Impact: 10x faster than manual QA. Full E2E regression tests in CI/CD pipeline.
Priority partnership area. If you're a QA team looking to automate mobile testing, request a meeting.
┌─────────────────────────────────────────────────────┐
│ Goal: "Get image from WhatsApp, submit invoice" │
└─────────────────────────────────────────────────────┘
↓
┌────────────────────────────────────┐
│ 1. PERCEPTION │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ $ adb shell uiautomator dump │
│ │
│ Accessibility Tree (XML): │
│ <Button text="Download Image" │
│ bounds="[100,500][300,600]"│
│ clickable="true" /> │
│ │
│ Parsed to JSON: │
│ {"text": "Download Image", │
│ "center": [200, 550], │
│ "clickable": true} │
└────────────────────────────────────┘
↓
┌────────────────────────────────────┐
│ 2. REASONING (GPT-4) │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ Prompt: "Goal: Get WhatsApp image"│
│ "Screen: [Download Image button]" │
│ │
│ GPT-4 Response: │
│ { │
│ "action": "tap", │
│ "coordinates": [200, 550], │
│ "reason": "Download the image" │
│ } │
└────────────────────────────────────┘
↓
┌────────────────────────────────────┐
│ 3. ACTION (ADB) │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ $ adb shell input tap 200 550 │
│ │
│ → Image downloaded! │
└────────────────────────────────────┘
↓
Repeat until done
| Approach | Cost | Speed | Accuracy | Works on Device |
|---|---|---|---|---|
| Screenshots (Computer Use) | $0.15/action | 3-5s | 70-80% | Desktop only |
| Accessibility Tree (Android Use) | $0.01/action | <1s | 99%+ | Handheld devices |
Technical advantage: Accessibility tree provides structured data (text, coordinates, hierarchy) without image encoding/OCR.
kernel.py (131 lines)
├── get_screen_state() # Dump & parse accessibility tree
│ └── sanitizer.py # XML → JSON (54 lines)
├── get_llm_decision() # GPT-4 reasoning
└── execute_action() # ADB commands
├── tap (x, y)
├── type "text"
├── home / back
└── done
Total core logic: <200 lines. Simple, hackable, extensible.
from kernel import run_agent
run_agent(
goal="Open WhatsApp and download the latest image",
max_steps=10 # Max actions before timeout
)
# Tap coordinates
{"action": "tap", "coordinates": [540, 1200]}
# Type text
{"action": "type", "text": "Invoice #12345"}
# Navigate
{"action": "home"} # Home screen
{"action": "back"} # Previous screen
# Wait/Complete
{"action": "wait"} # Wait for loading
{"action": "done"} # Goal achieved
from kernel import get_screen_state
screen_json = get_screen_state()
# Returns: [{"text": "Submit", "center": [540, 1200], ...}]
pip install android-useDon't want to host it yourself? Join the waitlist for our managed Cloud API.
What you get:
Priority access for: Trucking/logistics companies and QA testing teams. Request a partnership meeting or join the general waitlist (Coming Q1 2026)
Want to help build the future of mobile AI agents?
git checkout -b feature/factoring-app-supportgit commit -m 'Add RTS Pro integration'git push origin feature/factoring-app-supportSpecial focus: If you work in logistics, gig economy, or field services, your domain expertise is invaluable!
5.3 million views. 12 → 1100+ stars in days. This growth validates the market need.
Help us hit 1,500 stars to show investors/partners there's real demand for mobile AI agents!
⭐ Click here to star this repo now →
Star Progress: ████████████████░░░░░░░░░░░░ 1100+ / 1,500 stars (63% there!)
Every star helps logistics companies and QA teams discover this solution + validates demand for funding
|
⭐ Star on GitHub Support the project |
Book Partnership Trucking/logistics or QA testing |
Share on X Help others find this |
Join Waitlist Cloud API access |
Built by Ethan Lim - AI engineer
I spent a week riding along with truck drivers for a logistics automation project. Sitting in the cab, I watched one driver spend 10 minutes after every delivery manually typing data from a Bill of Lading photo into the RTS Pro factoring app on his phone.
"I do this 15 times a day. Can't use a laptop—there's no room in the cab, no internet half the time, and I'm already holding the phone to take the photo anyway. This app doesn't have an API. It's just me and this phone."
That's when it clicked: The world has AI agents for web browsers and desktop computers, but 3 billion people work on mobile devices that existing agents can't touch.
I looked at existing solutions:
Neither solved the truck cab problem. So I built Android Use using Android's accessibility API instead of expensive vision processing.
The technical bet:
Posted the demo showing a full logistics workflow. 5.3 million views. 150+ inbound messages from companies. 1100+ GitHub stars. The market validated the need immediately and continues to grow.
Based on inbound demand, the path forward is clear:
Business model: Freemium. Open-source core is free forever. Revenue from managed hosting ($0.02-0.05/action) and enterprise features ($5-10k/month for 100+ seat deployments). Target customers: logistics companies (50+ drivers), delivery platforms (1000+ workers), field service orgs.
Defensibility: Network effects from app-specific agent templates. First-mover advantage in mobile-first verticals. Direct partnerships with factoring companies and gig platforms create distribution moats.
Vision: Make AI agents work where 3 billion people actually work—on mobile devices.
→ Request a meeting for partnerships - We created this due to overwhelming demand
Actively seeking:
Already running pilots with 5 trucking/delivery companies.
Since launch:
Market data:
The star growth from 12 → 1100+ validates real demand. Help us reach 1,500 → Star now
MIT License - see LICENSE
Free for personal and commercial use. Build on it, sell services with it, integrate it into your logistics platform.
Why MIT? We want this to become the standard for mobile AI agents. Open source = faster adoption.
Built on:
Special thanks to:
Whether you're in a truck cab, on a delivery route, or in the field...
AI agents should work where you do.
⭐ STAR THIS REPO (Help us reach 1,500!) • WATCH DEMO (5.3M+ views) • SHARE IT
git clone https://github.com/actionstatelabs/android-action-kernel.git
cd android-action-kernel
pip install -r requirements.txt
python kernel.py
Made for field workers • Star • Follow @ethanjlim • Share
53,286 followers · starred Jan 2026
108 followers · starred Dec 2025
760 followers · starred Dec 2025
111 followers · starred Dec 2025