NVIDIA APIs

Experiments with NVIDIA Jetson VLM (Vision-Language Model) APIs — running VILA, LLaVA, and Neva models locally on Jetson devices, with OpenAI-compatible API endpoints and Gradio interfaces for video frame analysis.

Project Structure

.
├── nvidia_apis.py          # Flask API — VILA & Neva via NVIDIA cloud endpoints
├── llava_api.py             # Flask API — LLaVA via local Jetson container
├── llava_openaicomp.py      # Flask API — LLaVA with OpenAI-compatible format
├── vila_url.py              # VILA — video frame extraction + API call
├── vila_stream_call.py      # VILA — streaming response processing
├── interface.py             # Gradio UI — video → frame → VLM analysis
├── start.py                 # Launcher — starts Flask + Gradio together
├── llava_container_terminal.py  # LLaVA container terminal commands
├── notlar.txt               # Development notes (Turkish)
├── cat.jpg                  # Sample image
├── ornek_video.mp4          # Sample video
└── versions/                # Iterative development versions (v1–v9)
    ├── app_v1.py            # v1: Simple LLM
    ├── app_v2.py            # v2: VLM
    ├── app_v3_gradio.py     # v3: Debugging + Gradio
    ├── app_v4_gradio.py     # v4: Working but prompt not dynamic
    ├── app_v5_gradio.py     # v5: Dynamic prompt
    ├── app_v5.3_gradio.py   # v5.3: Refinements
    ├── app_v6_gradio.py     # v6: OpenAI-compatible API format
    ├── app_v7_gradio.py     # v7: Video support (frame display issue)
    ├── app_v9_gradio.py     # v9: Final — frame display + video working
    ├── app_v*_model.py      # Model (Flask server) for each version
    ├── app_v*_apiCall.py    # API caller for each version
    └── frame_alma.py        # Frame extraction utility

Version History

Models Used

• VILA — NVIDIA's vision-language model (cloud API + local)
• LLaVA — Running locally via Jetson container (nano_llm)
• Neva — NVIDIA cloud API
• Meta-Llama-3-8B — Text generation via MLC

Tech Stack

• VLM Framework: nano_llm (Jetson)
• API: Flask (OpenAI-compatible /v1/chat/completions endpoint)
• UI: Gradio
• Video: OpenCV (frame extraction)
• Hardware: NVIDIA Jetson (with Docker containers)

Setup

# On Jetson device
jetson-containers run $(autotag nano_llm)

# Start API + Gradio
python3 start.py

Notes

See notlar.txt for development notes including Docker mount commands, container setup, and NVIDIA API lab references.