Methodology
How the Benchtale desk turns agent output into a published review.
Drafted by AI agents. Verified by humans. Auditable in three steps. Three input streams feed the desk, two human gates stand between an agent draft and a live review.
Quick summary
Where the data comes from
Three named input streams.
Every score and every sentiment weight on the desk traces back to one of these three streams. The corpus hash for each stream is pinned on every audit log.
- Retailer scrapesDaily; price-history window of 90 days is kept per product.
Spec sheets, MSRPs in local currency, in-stock flags and price-drop pings pulled from retailer product pages on a daily cadence. Each row carries the retailer name, the captured price, and a hash of the page body at scrape time so a later reader can re-derive what the desk saw.
Why it matters · A review is not useful without ground-truth on what the product costs today. The scrape feed is how the desk keeps MSRP and current street price honest — and is the input the price-to-performance weighting sits on top of.
- Reddit + specialist-forum corporaRolling ingest on every review kick-off; megathreads are tracked across their full lifecycle.
Threaded sentiment from r/hardware, r/buildapc and r/AMDHelp, plus long-form discussion on the L1 Tech forum, the Hardware Unboxed community, and CreativeCOW. Each excerpt is hash-pinned to the source thread and carries a polarity weight from the agent and a sample-size count from the corpus.
Why it matters · Specs are not the whole picture. Owner-reported pain (driver regressions, PSU headaches, CUDA fallbacks) is what tells a reader whether the headline number survives contact with a real desk — that signal lives in the forums, not on the product page.
- Synthetic benchmark suiteRe-run on every firmware/driver shift and every quarterly review refresh.
Controlled test rigs run 3DMark Time Spy Extreme, Cyberpunk 2077 at native 4K (with and without vendor upscaling), and Blender BMW cycles under declared conditions: same driver, same scene file, same power-profile. The agent re-runs each test on a schedule so the corpus is repeatable, not a one-shot screenshot.
Why it matters · Synthetic does not mean canned — it means the inputs are pinned. The same three workloads re-run on the same hardware let the desk catch generation-over-generation deltas, driver regressions, and firmware wins without attributing ambient variance to a vendor.
Who checks it
Two human gates before publication.
Both gates run on every review. Anything that does not survive both is rewritten or pulled — never quietly published.
- Fact-check pass
Who · A human fact-checker on the editorial desk — not the agent, not the prompt author.
Pass criteria · Per-claim check against the source corpus for every numeric value, every vendor name, and every release date. The fact-checker marks each claim kept / rewritten / pulled, and signs the verdict on the audit log before the second gate runs.
What it blocks · A claim the fact-checker cannot tie to a hash-pinned source row does not move forward. The agent is queried for a tighter grounding; the claim is rewritten or removed before the second gate.
- Token-level audit
Who · A second reviewer reading the agent’s output diff against the materials used in this run.
Pass criteria · Token-level diff of the agent draft against the source corpus and the prior run’s verdict, plus a hash-pinned copy of the corpus referenced for the run. Reads for hallucinated numbers, vendor names that do not appear in the corpus, and any claim the agent inferred rather than sourced.
What it blocks · A drift between the agent draft and the materials — hallucinated specs, invented benchmarks, regressions on a prior verdict — blocks publication. The agent re-runs with the diff as feedback; nothing ships until the diff is empty.
How to inspect any past audit log
The audit log is on a public route.
Every published review links to its own audit log at /audit/[review-id]. The log carries the fact-checker’s verdict, the token-level diff against the source corpus, and a hash-pinned copy of the materials used for that run. The example below is the live audit stub for our RTX 4090 vs. RX 7900 XTX round-up — the desk wires the live feed there on the next deploy.
View the example audit log →