A report claims 94% accuracy on a test set. Did those numbers come from the attached code — or did someone just type them into the manuscript and hope nobody looks?
The usual answer: rerun the thing. Days of compute, GPUs the reviewer doesn't have, and a lab environment you'd need to rebuild byte for byte. Nobody does that twice.
K-Veritas tries to cut that loop at the root. A Go CLI built with Cobra, static binaries a few megabytes in size, one ambition: stitch commands, metrics, environment and final report into a cryptographically signed trace, verifiable server-side or offline, without ever re-executing the computation.1 Prebuilt binaries — CLI and attestation server alike — ship for Linux, macOS and Windows.3
The project slammed into Hacker News in late August, and the reception was openly hostile.5 "It doesn't really prove anything other than the author signed the report," exe34 summed it up. Almost right — which is precisely why this is worth looking under the hood.
Four commands
The K-Veritas flow boils down to four commands. kveritas init opens a session that records an experiment identifier, a machine fingerprint and the project location. kveritas run -- python train.py executes any command under watch: duration, exit code, pre- and post-run file hashes, and the stdout and stderr streams are all captured during execution.1
The capture goes beyond file contents. The runner samples the machine about ten times per second, and records the metrics, phases and seeds the script itself prints to standard output.
This is not a black box deciding what counts. It is a stdout protocol: your script writes a line like KVERITAS_METRIC name=accuracy value=0.94, and the runner attaches it to the running session, hashed like everything else.1 Note the implication: your script has to cooperate. If the metric never appears on stdout, there is nothing to sign.
Metrics tied to a trace
The tricky part of any execution attestation is proving that the displayed metric came from this specific run, and not from another one.
K-Veritas handles this on two levels. First, direct association: every metric and every claim is recorded with the output line number where it appeared, inside the session that was running at that moment. The trace and the metrics are not two files pasted together; they are born together from the same process monitor.1
Second, an execution-coherence checker called HMCA, deliberately blind to the metrics themselves. Instead of judging whether a score is plausible, it compares how the telemetry channels — CPU, memory, GPU, activity — co-fluctuate during the run.1
The idea fits in one line: a real execution moves its channels as one thing. A fabricated, replayed or stitched-together trace produces telemetry coherences that do not hold together. When the channels drift too far apart, the checker returns an explicit warning: execution channels not coherent, possible fabrication or replay.1
Caution is not a slogan here: below too small a sample, HMCA abstains rather than accuses.1 A statistical signal, not a forensic proof.
A report anyone can verify
When the session is sealed, kveritas seal produces a PDF that stays readable by humans. But the part that matters for verification is not in the visible text: the signature is inserted after the end of the file, between two dedicated markers, together with the public key needed to check it.1
The cryptographic machinery is nothing exotic: session data is hashed into a canonical JSON, that hash goes into a data_hash:nonce:signed_at payload, and the whole thing is signed with RSA-PSS-SHA256 using a 4096-bit key.1 The attestation server holds the private key; the CLI embeds the matching public key as a trust anchor.1
Someone receiving the report can therefore check two distinct things. First: the report is intact, nobody touched the signed data. Second: the signature comes from the K-Veritas key — or it does not.
That is the nuance the Hacker News comment was missing. K-Veritas is not telling reviewers to trust the author. It offers a path where origin verification is possible without rerunning the experiment — provided you lean on the attestation server as a trusted third party.
Hence the difference between two CLI outputs. If the report is signed with the K-Veritas key, verification prints VERIFIED. If the author signed with their own key (offline mode), the output prints SELF-ATTESTED, followed by an unambiguous note: the signature is valid, but the origin of the report cannot be confirmed, and the results should be treated as unverified.1
Proof of execution, not proof of truth
Now comes the trap the project's website does not help you avoid. The homepage promises results that are "authentic, reproducible and honestly reported".2 The protocol spec is more restrained: it says K-Veritas records what an experiment or an AI agent did, as signed evidence, so any reviewer can independently check what ran and who did it.2
That is not the same promise.
What the signature actually guarantees is a chain: a session started on a given machine, a precise command executed, files that produced certain hashes, declared metrics that appeared in the output of that execution, and a seal at a signed moment. That is a proof of execution.
It is not a proof that the result is correct. A signature does not stop a leaked dataset, a misleading metric parser or an over-trained model. On that point the project documentation is honest: it states explicitly that the protocol does not detect some forms of train/test contamination, and that it cannot on its own prove that a private evaluation set is fair — the former limitation being presented as future work.6
Back to the opening example: the report says 94%, the trace says 94% appeared in a real output of a real command, on a real machine, inside a sealed session. The two do not overlap. Writing one in place of the other means confusing the chain that produces the proof with the scientific validity of the result.
Back to the opening example: the report says 94%, the trace says 94% appeared in a real output of a real command, on a real machine, inside a sealed session. The two do not overlap. Writing one in place of the other means confusing the chain that produces the proof with the scientific validity of the result.
What still needs proving
The elegance of the solution lies in what it does not claim to cover — once you read the code rather than the website.
Three checkers complete HMCA. A compute-cost certificate verifies that the declared FLOPs could physically fit within the observed time, energy and memory.1 Its bounds are deliberately generous: the goal is to never accuse an honest run, not to catch clever fraudsters. A declaration announcing a monstrous load with no usage trace at all, on the other hand, is flagged for review.1
A provenance registry builds a hash chain linking each file state to the previous one — a tamper-evident watermark of everything touching the project — with the whole thing sealed inside the session.1
And a claims-check feature crosses the figures cited in a manuscript against the values recorded in the sealed report, to surface paper claims that have no signed counterpart.1
Same philosophy throughout. Nobody checks the elephant; they check that an elephant passed through.
A demanding crowd
The August 31 Hacker News thread zeroed in on what matters. Several exacting comments reminded everyone that "real" execution proof would be a verifiable proof of computation — running inside a hardware enclave, or a zero-knowledge proof generated by a dedicated machine.5 K-Veritas, by contrast, just collects traces and signs them.
The point stands, but it compares a GT coupe with a work truck. Zero-knowledge proofs and enclaves exist and are powerful; they are also expensive to produce, often specialised per type of computation, and hard to push onto researchers who write their script in three days.
What K-Veritas builds is a working tool for reproduction: it lowers the cost of verifying execution without asking reviewers to redo everything, and it raises the cost of the banal lie ("I ran my script — just look at the paper"). Sophisticated fraud still gets through — the proof remains an attestation of process, not a mathematical proof of content.
A trace is not a rerun
The editorial question underneath is more interesting than the tool itself: should reproducibility artifacts become cryptographic proofs?
Computer science venues already run an artifact badging system, steered by the ACM, rewarding papers whose artifacts are evaluated, available or validated.4 But the evaluation stays human, relative and voluntary; it depends on reviewer time and goodwill.
K-Veritas proposes a more radical alternative: turning attested artifact presence into a property that anyone can check automatically, instead of depending on the time and goodwill of reviewers.2 That is not the same as independent reproduction — the most common badges (artifacts available, artifacts evaluated) do not prove another lab reran the code successfully; the "results validated" level does require a real re-execution. The K-Veritas approach remains a useful middle rung between "download my repo" and "start over from scratch."
The project is young, and adoption remains to be proven: as of writing, its public visibility is mostly the launch thread and the repository's own exchanges. That is an on-the-ground impression, not a measured statistic. It is not the first to try, and it will not be the last. Its current contribution is elsewhere: showing, in code, what an execution proof can reasonably contain — and at what price.
For the reviewer, pocket this next time a report prints VERIFIED: check first what exactly got signed. Very often it will be the promise "this ran" — not "this is true." Telling the two apart is the entire difference between useful verification and blind trust.