---
title: "CAD agents need work traces more than perfect demos"
locale: "en"
url: "https://irz.fr/en/articles/cadbench-work-traces-agent-cad-en"
markdown_url: "https://irz.fr/en/articles/cadbench-work-traces-agent-cad-en.md"
category: "ai"
tags: ["AI", "CAD", "agents", "benchmark", "workflows"]
published_at: "2026-08-23T17:52:00.000Z"
author: "Léa Perrin"
translation: "https://irz.fr/fr/articles/cadbench-work-traces-agent-cad-fr.md"
---

# CAD agents need work traces more than perfect demos

CADBench, Task Model Induction and Skill Recorder point to the same constraint from different angles: automating expert software requires verifiable traces of real work, not just a polished demo.

The question “can an agent use software?” gets interesting after the polished demo disappears, when there is a trace of the work left on the table.

In CAD, a task is not just a chain of clicks. The system has to recognize a geometric intention, choose operations, handle constraints, produce an executable solid, and leave behind a model somebody can edit later. CADBench attacks that problem from the evaluation side: 18,000 samples, six benchmark families, five input modalities and six metrics for measuring editable CAD program generation.[1](https://anniedoris.github.io/CADBench/)[2](https://arxiv.org/abs/2605.10873) A separate paper on **Task Model Induction** starts from computer-use traces — screenshots, mouse events, keyboard events — and tries to recover what tasks were being pursued, in a form that can be audited.[4](https://arxiv.org/abs/2608.20319) Microsoft Skill Recorder records a work session, reconstructs an intent and ordered steps, then turns the result into a reusable Skill or Automation for an agent.[5](https://github.com/microsoft/skill-recorder)

These are not the same announcement. Read together, though, they point to the same constraint: before agents can be trusted inside professional software, observed work has to become something people can test.

## CAD exposes the shortcut

CADBench focuses on reconstructing editable CAD programs from images or 3D observations, a distinction that matters because producing a plausible mesh is not the same as producing CAD code that executes, creates a valid solid and remains compact enough to be useful.[2](https://arxiv.org/abs/2605.10873)

The benchmark combines families derived from DeepCAD, Fusion 360, ABC, MCB and Objaverse, with clean meshes, noisy meshes, single-view renders, multi-view renders and photorealistic renders. Its metrics cover volumetric IoU, Surface IoU, Chamfer distance, valid shape rate, token count and operation count.[1](https://anniedoris.github.io/CADBench/)[2](https://arxiv.org/abs/2605.10873) The public Hugging Face dataset exposes the same 18,000-row structure across six splits, with CAD files and image modalities attached to the samples.[3](https://huggingface.co/datasets/DeCoDELab/CADBench)

A visual score would hide too much: an agent can draw something that resembles the part while producing unusable CAD code, or get a good global overlap while missing surface details and wasting operations. CADBench makes those failures separable.

> Evaluation
> **What CADBench measures**
> - IoU, SIoU, Chamfer: Geometry
> - A program that creates a valid solid: Execution
> - Tokens and operations: Economy
> - Clean, noisy, single-view, multi-view and PBR inputs: Robustness
> Sources: CADBench project page and arXiv paper.

The paper’s results explain why this matters. Under idealized inputs, specialized mesh-to-CAD methods substantially outperform general vision-language models, but some of those specialized systems become fragile when the input modality changes. The authors identify three recurring failure modes: reconstruction degrades with geometric complexity, CAD-specialized models can be brittle under modality shift, and model rankings change depending on the metric used.[2](https://arxiv.org/abs/2605.10873)

So the useful question is not only “which model wins?” but also “which kind of mistake would make this result unusable in a workshop, an engineering office or a manufacturing pipeline?”

## Traces replace the hand-written recipe

The other half of the signal comes from work traces. **Inducing Task Models from Computer-Use Traces** starts from a very ordinary situation: someone uses a computer, while screenshots and mouse or keyboard actions are passively recorded. The proposed TMI system tries to discover latent tasks, disentangle interleaved goals, and reconstruct a task model made of hierarchical objectives plus a procedure model for the observed execution.[4](https://arxiv.org/abs/2608.20319)

The abstract’s numbers are careful but worth noting. On controlled human and agent trajectories, TMI reaches 0.974 agreement with ground-truth task groupings, reconstructs 74.9% of the observed execution steps, and skills derived from its task models improve held-out task accuracy by 30.0% over the strongest baseline cited by the authors.[4](https://arxiv.org/abs/2608.20319)

Raw traces are not clean, and that is part of their value. A real session contains open windows, interruptions, dead ends, repeated checks and useless gestures; it shows work as it happens, not as a manual imagines it.

Skill Recorder takes a more product-shaped route: Microsoft’s repository describes a desktop app that records a screen session — clicks, app and window switches, visited pages and optional narration — then uses GitHub Copilot CLI to reconstruct an intent and ordered steps, from which the user can generate a Skill or an Automation.[5](https://github.com/microsoft/skill-recorder) The README also makes the safety boundary explicit: recording and frame extraction happen locally, but analysis sends selected data to GitHub’s cloud, and users are warned not to capture secrets.[5](https://github.com/microsoft/skill-recorder)

## What changes

AILANTA groups these publications under **Computer-Use Workflow Learning**: turning work sessions, traces and benchmarks into learning infrastructure for agents.[6](https://ailanta.tech/signals/computer-use-workflow-learning/) The market framing is still early, but the grouping is useful.

For IRZ, the important shift is material. A lot of software automation has lived in two poor formats: the fragile macro that replays gestures, or the abstract documentation that explains what should happen. The work cited here moves toward a third format: a trace rich enough to audit, transform, test and reuse.

In CAD, that shift is concrete: an agent should not merely “see” a part, because it has to reconstruct editable intent, choose geometry that executes, and accept judgment from several metrics at once. In office or business software, the agent should not merely imitate a video; it has to extract the task, separate useful steps from noise, and produce a procedure a human can read.

The limits are just as clear. CADBench measures CAD program reconstruction from standardized inputs; it is not yet an autonomous agent spending a full day inside Fusion 360. TMI reconstructs task models in a research setting; that does not guarantee robustness in every professional application. Skill Recorder can turn one example into a reusable procedure, but the recording still has to be reviewed, especially when sensitive data might appear on screen.[2](https://arxiv.org/abs/2605.10873)[4](https://arxiv.org/abs/2608.20319)[5](https://github.com/microsoft/skill-recorder)

Still, the direction is visible. Software agents improve when work stops being a spectacular thirty-second demo and becomes something recorded, measured and criticized; for design tools, the stronger threshold may be an agent whose work can be inspected afterwards.

## References

1. [CADBench project page, CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation](https://anniedoris.github.io/CADBench/)
2. [Anna C. Doris et al., CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation, arXiv:2605.10873v2](https://arxiv.org/abs/2605.10873)
3. [DeCoDELab/CADBench dataset on Hugging Face](https://huggingface.co/datasets/DeCoDELab/CADBench)
4. [Yucheng Jiang et al., Inducing Task Models from Computer-Use Traces, arXiv:2608.20319](https://arxiv.org/abs/2608.20319)
5. [Microsoft Skill Recorder repository](https://github.com/microsoft/skill-recorder)
6. [AILANTA, Computer-Use Workflow Learning, August 23 2026](https://ailanta.tech/signals/computer-use-workflow-learning/)