Eye tags photos on the device so apps can find and filter them: what a photo is (screenshot, receipt, selfie), what is in it (558 scenes), how it came out (blurry, eyes closed), who is in it (group shot, portrait) and whether it is sensitive, each with a calibrated confidence where 0.5 is the recommended cutoff and 0.9 a strict one. It also scores photos, finds people, pets and faces, and picks crops.
| File | Format | Size | Runs on |
|---|---|---|---|
eye.mlmodelc | Compiled Core ML, one function per task | 48MB | Apple platforms; fastest on the Neural Engine, also runs on CPU and GPU |
eye.tflite | LiteRT, one signature per task | 66MB | Android, Linux, Windows, Node, the browser |
eye_meta.json | JSON | 880KB | Inputs, outputs, decoding constants and the tags (calibration curves and rules) |
Both files hold the same functions under the same names. Every input is RGB (or gray for sharpness) in 0...1, shape (1, C, H, W): float16 for Core ML, float32 for LiteRT. eye_meta.json has the exact preparation and how to decode each output.
| Function | Input | Outputs |
|---|---|---|
photo | 384x384 centre crop | embedding, type, reject reasons, sensitive content, scenes, rotation, aesthetic, keeper, cover, focus and bokeh, pet attributes |
photo_peak | as photo | photo's outputs and the peak moment |
spatial | whole frame squashed to 384x384 | where people look, pet and person boxes, shot type, scores for 1,296 crop boxes |
faces_detect | 768x768, the photo letterboxed | face heatmap, size, offset, landmarks |
faces_analyze | 128x128 crop, 1.4x a face box | eyes open, blink, smile, gaze, expression, pose, quality and more |
sharp_4x3, sharp_3x4, sharp_16x9, sharp_9x16 | gray, the shape closest to the photo's aspect | sharpness, blur map |
Every tag has a confidence calibrated so 0.5 is the recommended cutoff and 0.9 a strict one. At 0.5: photo types 0.8 to 1.0 precise (screenshot 0.98, receipt 1.00), scenes 0.88 median, people 0.95, eyes closed 0.92. The keep-or-delete reasons (forgettable, bad framing, ...) are less reliable; use them for suggestions. For anything automatic, such as hiding sensitive photos, use the strict cutoff.
On the same test sets, through the same scoring code:
| Test set | Metric | eye.mlmodelc | eye.tflite (CPU) |
|---|---|---|---|
| KonIQ-10k | aesthetic SRCC | 0.6288 | 0.6291 |
| COCO val2017 | person AP50 | 0.3814 | 0.3837 |
| MIT1003 | where people look, CC | 0.7153 | 0.7195 |
| CUHK cropping | crop IoU | 0.8244 | 0.8241 |
| WIDER Face val | AP, hard | 0.6806 | 0.6812 |
| CEW | eyes open AUC | 0.9943 | 0.9945 |
One photo, photo_peak, on a Mac with an Apple M6:
| Runtime | ms |
|---|---|
| Core ML, Neural Engine | 5.5 |
| LiteRT, GPU (Metal) | 30 |
| LiteRT.js, WebGPU, in Chrome | 41 |
| LiteRT, CPU, 4 threads | 49 |
| LiteRT.js, wasm with threads, in Chrome | 176 |
eye.tflite is not yet measured on Android devices.Attribution and license texts for third-party components are in THIRD_PARTY_NOTICES.md.
Released under the Desert Ant Labs Source-Available License v1.0 (see LICENSE.md). Free for commercial use up to 100,000 MAU per Model. Contact [email protected] for commercial licensing at larger scale.
Eye tags photos on the device so apps can find and filter them: what a photo is (screenshot, receipt, selfie), what is in it (558 scenes), how it came out (blurry, eyes closed), who is in it (group shot, portrait) and whether it is sensitive, each with a calibrated confidence where 0.5 is the recommended cutoff and 0.9 a strict one. It also scores photos, finds people, pets and faces, and picks crops.
| File | Format | Size | Runs on |
|---|---|---|---|
eye.mlmodelc | Compiled Core ML, one function per task | 48MB | Apple platforms; fastest on the Neural Engine, also runs on CPU and GPU |
eye.tflite | LiteRT, one signature per task | 66MB | Android, Linux, Windows, Node, the browser |
eye_meta.json | JSON | 880KB | Inputs, outputs, decoding constants and the tags (calibration curves and rules) |
Both files hold the same functions under the same names. Every input is RGB (or gray for sharpness) in 0...1, shape (1, C, H, W): float16 for Core ML, float32 for LiteRT. eye_meta.json has the exact preparation and how to decode each output.
| Function | Input | Outputs |
|---|---|---|
photo | 384x384 centre crop | embedding, type, reject reasons, sensitive content, scenes, rotation, aesthetic, keeper, cover, focus and bokeh, pet attributes |
photo_peak | as photo | photo's outputs and the peak moment |
spatial | whole frame squashed to 384x384 | where people look, pet and person boxes, shot type, scores for 1,296 crop boxes |
faces_detect | 768x768, the photo letterboxed | face heatmap, size, offset, landmarks |
faces_analyze | 128x128 crop, 1.4x a face box | eyes open, blink, smile, gaze, expression, pose, quality and more |
sharp_4x3, sharp_3x4, sharp_16x9, sharp_9x16 | gray, the shape closest to the photo's aspect | sharpness, blur map |
Every tag has a confidence calibrated so 0.5 is the recommended cutoff and 0.9 a strict one. At 0.5: photo types 0.8 to 1.0 precise (screenshot 0.98, receipt 1.00), scenes 0.88 median, people 0.95, eyes closed 0.92. The keep-or-delete reasons (forgettable, bad framing, ...) are less reliable; use them for suggestions. For anything automatic, such as hiding sensitive photos, use the strict cutoff.
On the same test sets, through the same scoring code:
| Test set | Metric | eye.mlmodelc | eye.tflite (CPU) |
|---|---|---|---|
| KonIQ-10k | aesthetic SRCC | 0.6288 | 0.6291 |
| COCO val2017 | person AP50 | 0.3814 | 0.3837 |
| MIT1003 | where people look, CC | 0.7153 | 0.7195 |
| CUHK cropping | crop IoU | 0.8244 | 0.8241 |
| WIDER Face val | AP, hard | 0.6806 | 0.6812 |
| CEW | eyes open AUC | 0.9943 | 0.9945 |
One photo, photo_peak, on a Mac with an Apple M6:
| Runtime | ms |
|---|---|
| Core ML, Neural Engine | 5.5 |
| LiteRT, GPU (Metal) | 30 |
| LiteRT.js, WebGPU, in Chrome | 41 |
| LiteRT, CPU, 4 threads | 49 |
| LiteRT.js, wasm with threads, in Chrome | 176 |
eye.tflite is not yet measured on Android devices.Attribution and license texts for third-party components are in THIRD_PARTY_NOTICES.md.
Released under the Desert Ant Labs Source-Available License v1.0 (see LICENSE.md). Free for commercial use up to 100,000 MAU per Model. Contact [email protected] for commercial licensing at larger scale.