Three rounds.
Every run published.

Agent view · same build, same facts

selected run → /bench/cells.json record

Select a run on the left to see its record here.

what your agent receives → /llms.txt

# LINC Forge

> Developer site of LINC Innovations. Read the caveat before the numbers.

## Caveat

In v3 we held our stack at its v2 commit, bumped all three competitors to their latest releases, and ran an unseen task set. On the seven non-debugger UI and logcat tasks (a5, a6, b5, b6, b9, b10, c9), agent-device and linc matched at 34/35; the ten-run gap in the 44/45 vs 34/45 headline total is entirely c10 and c11, which require runtime JDWP inspection via spectra (not yet released).

The fixes behind the v2 jump were aimed at tasks we had already watched fail, and the linc column is our own stack: nerve (open source) together with crucible and spectra (not yet released, so their transcripts are published instead). Read v2 as “the vendor can fix what this benchmark measures”, not “the vendor generalizes”.

Wear OS tasks (b8 in v1/v2, b11 in v3) are capability rows: reported separately and never averaged, as pre-registered. Scored totals are out of 45, nine tasks of five runs each.

In v1 LINC placed first by one run, had the most timeouts (none in v2 or v3), and scored 0/5 on c10, the task its debugger exists for. Every stack scored 0/5 on c10 in v1.

Task files and prompts were byte-identical across v1 and v2, with the same devices and model. The harness was hardened and the run order changed between rounds, identically for every stack, and some other stacks' scores moved as a result: in v2 the watch already carried the test app, and maestro drove the watch instead of the phone in every a4 run.

LINC wrote the tasks and the harness and ran all four stacks; the other three used their vendors' recommended configurations, with installed versions recorded per run.

## Benchmark: mobile-agent-bench (MIT)

Pre-registered; four MCP tool stacks driven by the same confined agent, pinned model and physical devices; every run published with its transcript.

- v3 (unseen tasks, pinned v2 linc commit), scored out of 45: agent-device 34/45 · linc 44/45 · maestro 14/45 · mobile-mcp 20/45 (seven UI+logcat tasks: linc 34/35 · agent-device 34/35)
- v2 (seen tasks after fixes), scored out of 45: agent-device 32/45 · linc 44/45 · maestro 20/45 · mobile-mcp 28/45

/tools.json

[
  {
    "id": "mobile-agent-bench",
    "status": "public",
    "kind": "benchmark",
    "license": "MIT",
    "repo": "https://github.com/LincForge/mobile-agent-bench",
    "summary": "The pre-registered benchmark: harness, target app, three pre-registrations and every transcript. Clone it and re-run any competitor's row or nerve; crucible and spectra in the linc column are not yet released, so their transcripts are published instead.",
    "install": "git clone https://github.com/LincForge/mobile-agent-bench && cd mobile-agent-bench && uv sync"
  },
  {
    "id": "0xl0c1",
    "status": "public-demo",
    "kind": "remote MCP server",
    "license": "Apache-2.0",
    "repo": "https://github.com/LincForge/0xl0c1",
    "summary": "A remote MCP server that pins what you learn about a physical object to that object, so any assistant can pick the thread up later. Built at the AI Tinkerers Seattle hackathon.",
    "install": null,
    "connect": "on request"
  },
  {
    "id": "nerve",
    "status": "public",
    "kind": "device-control MCP server and CLI",
    "license": "Apache-2.0",
    "repo": "https://github.com/LincForge/nerve",
    "summary": "Our device-control stack (uvx linc-nerve), measured in the benchmark as column linc together with crucible and spectra (crucible and spectra are not yet released).",
    "install": "uvx linc-nerve"
  }
]

/index.md (tail)

## Work with LINC

Want your app’s real-device failures found and root-caused? The Device Truth Audit: a fixed-price, two-week diagnostic for mobile apps on real phones and watches. You keep the evidence.
https://lincinnovations.com/audit?ref=forge