Lasso MCP · Benchmarks

Benchmarks · measured live

Fewer tokens, as many right answers

A real model, gpt-5.4-mini, did each task three ways: with Lasso, with ordinary file tools (read, search, list), and with the files uploaded. Token counts are OpenAI's own usage records for every call of the task.

Excel workbook, 5,000 rows

98%

fewer tokens than file tools

4 of 4 right with Lasso, 2 of 4 with file tools

PDF report, 40 pages

84%

fewer tokens than file tools

3 of 3 right, the same as with file tools

3D assemblies (STEP)

83%

fewer tokens than file tools

7 of 7 right with Lasso, 6 of 7 with file tools

Code and document lookups

69%

fewer tokens than file tools

32 of 32 right with Lasso, 29 of 32 with file tools

Tokens per task with Lasso, by kind of file

Each bar is what the model spent with Lasso, as a share of what it spent on the same tasks with ordinary file tools. Shorter is better; past the line, Lasso cost more.

gpt-5.4-mini, live, 30–31 October 2026; lookups over two runs in October. Hover a bar for tokens and right answers. The full numbers are in the table below.

What the model was asked

Four questions from the live runs, word for word, with what Lasso returned and what each way of reaching the files cost. None can be answered from one line of one file: each needs the structure around the part, the row or the sentence.

3D model · STEP assemblyRight with all three

“In Engineering/Flange assembly.step, what is the angle in degrees between the z axis of Sensor:1 and the z axis of Bolt:3?”

Why it needs the whole assembly. Each part's axes are stored in its own coordinates. The sensor is tilted and the bolts are turned around a bolt circle, so the angle only exists once both placements are composed through the assembly.

Lasso returned, in one callAnswer: 30°

"Bolt:3" from "Sensor:1": 48.2183 mm apart; … axes x 90°, y 90°, z 30° apart (assembly frame: "Sensor:1" at [90, 30, 20], "Bolt:3" at [50, 55, 10])

Tokens for the task

3D model · STEP assemblyRight with all three

“In Engineering/fixture assembly.step, how far apart are the origins of Pin:1 and Pin:2, in mm?”

Why it needs the whole assembly. Each pin's origin is 0, 0, 0 in its own part. Where the two pins sit, and so how far apart they are, is only defined by the assembly that places them.

Lasso returned, in one callAnswer: 58.31 mm

"Pin:2" from "Pin:1": 58.3095 mm apart; in "Pin:1"'s frame at [50, 30, 0], turned 90° about z (assembly frame: "Pin:1" at [20, 15, 10], "Pin:2" at [70, 45, 10])

Tokens for the task

Excel · 5,000 rowsOnly Lasso answered

“What is the total Amount for the West region in Finance/Transactions 2025.xlsx?”

Why it needs the whole table. 1,014 of the 5,000 rows are in the West. The total needs every one of them, picked out by another column. With file tools and with the file attached, the model said it could not work it out.

Lasso returned, in one callAnswer: 4,867,231.76

Transactions "Amount" total where Region is "West": 4867231.76, over 1014 of 5000 rows

Tokens for the task

PDF · 40 pagesRight with all three

“According to Reports/Annual review 2025.pdf, what was the warranty reserve at the end of 2025?”

Why it needs the whole report. The report gives the reserve twice, half a year apart, in the same words. Lasso returns both with their page and section, so the model can tell which one is year end.

Lasso returned, in one callAnswer: $4.2 million

p.9 §9. Products: "At the end of June, the warranty reserve stood at $3.6 million, …"
p.23 §23. Products: "The warranty reserve at year end was $4.2 million, up from $3.1 million a year earlier, …"

Tokens for the task

gpt-5.4-mini, live, 30 October 2026. Lasso's answers are shown as the model received them, shortened where marked “…”.

Mid-size reports, before and after the fix 31 Oct

On 8–9-page reports, Lasso first did no better than file tools: the model often asked Lasso, then read the report anyway. The same 18 tasks, a day apart:

Model read the report itself after asking Lassotasks of 18

10→1

Tokens for the 18 tasks54% fewer; file tools used 90,547 the same day

147.8K→68.5K

Right answersof 18; file tools got 8

7→9

A fresh set of 8–9-page reports 31 Oct

Eighteen questions written after the fix by a separate author, run live for the first time. Tokens for all 18 tasks:

Uploading the reports was cheapest here and got the most right. On short and mid-size documents, attaching them is hard to beat; Lasso's lead grows with the size of the file.

When to use Lasso, and when to attach the file

Lasso against uploading the files with the question.

Large files and 3D modelsUploads send the whole file on every call. Lasso used 99% fewer tokens on the 5,000-row workbook, 74% fewer on the 40-page PDF and 59% fewer on the STEP assemblies, and was never less accurate.

A page or two, a small sheetAttaching costs less than one tool call: two 1–2-page PDF tasks took 755 tokens uploaded against 6,875 with Lasso. On small sheets Lasso still got more right (7 of 7 against 4 of 7).

Mid-size reportsOn the fresh 8–9-page set, uploading got 14 of 18 right in 60.5K tokens; Lasso 12 of 18 in 69.4K.

Every live result, by kind of file

Right answers and total tokens for each way of reaching the files. Savings are Lasso's, over the tasks both ran.

Kind of file	Tasks	Lasso	File tools	Upload	Lasso vs file tools	Lasso vs upload
3D modelSTEP parts and assemblies	7	7/729,229	6/7167,941	7/771,166	−82.6%	−58.9%
3D modelIFC building, 4 MB; both arms wrong	1	0/13,040	0/170,261	too largeto attach	−95.7%	–
Excel5,000 rows	4	4/45,974	2/4369,647	2/4566,456	−98.4%	−98.9%
Excel6–12 rows	7	7/741,632	4/743,554	4/74,141	−4.4%	+905%
PDF40 pages	3	3/310,882	3/369,085	3/341,354	−84.2%	−73.7%
PDF8–9 pages, after the fix (31 Oct)	36	21/36137,921	17/36185,796	14/1860,520 · fresh set only	−25.8%	+14.6%fresh set
PDF8–9 pages, before the fix (30 Oct)	19	10/19121,491	9/19114,841	13/1974,065	+5.8%	+64.0%
PDF1–2 pages	2	1/26,875	1/26,049	2/2755	+13.7%	+811%
Code and document lookupstwo runs, October	32	32/32	29/32	–	−69.2%	–

Connect an agent

One tool, on your machine

Lasso's MCP server reads the library the Lasso app keeps. Add your folders in Lasso first, then point your agent at it and name the folders it may read.

Claude Desktop and other MCP hosts

{
  "mcpServers": {
    "lasso": {
      "command": "/Applications/Lasso.app/Contents/MacOS/node",
      "args": [
        "/Applications/Lasso.app/Contents/Resources/app/src/agent/cli.js",
        "mcp", "--scope", "Finance"
      ]
    }
  }
}

Claude Code

claude mcp add lasso -- \
  /Applications/Lasso.app/Contents/MacOS/node \
  /Applications/Lasso.app/Contents/Resources/app/src/agent/cli.js \
  mcp --scope Finance

An agent sees only the folders you name with --scope: a folder's name as Lasso shows it, its path, or "*" for every enabled folder.
Credential files and anything you've excluded from Lasso's suggestions stay out.
It runs locally over stdio: no port, no account, no API key. The agent gets only the evidence it asks for.

Method

How these were measured

Live runs. OpenAI's gpt-5.4-mini at low reasoning effort, through the Responses API, choosing its own steps. In each task it had either ordinary file tools (read, search and list, with PDF, Word and Excel files converted to text), those tools plus Lasso's one tool, or no tools and the files attached to the question. Tokens are the provider's usage records: every input and output token of every call in the task.

Tasks. Generated libraries whose answers are known: code and documents for lookups, 3D models, workbooks and reports for the rest. Answers are graded against the facts the files were written with, including whether conflicting sources and caveats were reported.

Runs. Lookups: two runs per arm, October 2026. Kinds of file: one run on 30 October ($1.01). Mid-size reports: one run on 31 October ($0.47). The same task can vary a lot between runs, so small single-task differences are noise; the differences on large files and 3D models are not. The lookups were measured with an earlier version of the tool; the changes since kept every lookup answer right offline.

The benchmark code, every task and the full reports are in the repository: docs/agent/README.md.