An LLM can load successfully, answer normally and still give you a bad benchmark.
Model Serving Minefield is a community registry for exactly that class of failure. It currently lists 116 serving-path traps involving chat templates, reasoning fields, tool parsers, quantization routes, container toolchains, memory allocation, evaluation harnesses and model versioning.
The annoying pattern is simple: nothing obviously crashes. The number is just wrong.
Start with what you actually see
The registry is organized around symptoms rather than only around libraries. Entries include cases such as a reasoning rate reading 0% while the model visibly reasons, an orphaned </think> tag, an FP4 checkpoint running far slower than its format suggests, or a model loading cleanly before dying at the first token.
Each entry then records the suspected mechanism, affected stacks, a confirmation check and an evidence status. The project explicitly separates cases it reproduced locally from contributor measurements, external reports and work still under test.
That evidence vocabulary matters more than the headline count. A weird result on one machine is not automatically equivalent to a reproducible failure with public conditions.
The repository also provides stack-specific pages for vLLM, llama.cpp/GGUF, Ollama, SGLang and MLX, plus a bounded read-only doctor and an MCP interface that can help an agent inspect a live endpoint without modifying it.
For local AI builders, this is a useful correction to the usual benchmark mindset. You are not measuring only the model. You may also be measuring a template default, a parser, a fallback kernel, memory reservation, a container image or a slightly different artifact uploaded under a familiar name.
Before asking which model is fastest, the more basic question is whether you can prove what path actually ran.