We graded the field. Including ourselves.
Every service below was graded on its public demo line, with live test calls, on the same published 21-point test we run on our own agent: six scripted caller scenarios, scored on task completion · accuracy of promises · capture accuracy · interruption recovery · dead air · fallback & honesty · close quality. A severity flag is recorded when an agent does something on a call that would cost a real shop money or trust.
This is a dated snapshot — tests ran Nov 5–7, 2026. Vendors change their agents; a score is what that demo line did on that day, nothing more. Any service listed here can ask for a re-test, and the full field gets re-run and re-published after our build program completes.
The board — snapshot, Nov 5–7, 2026
| Service | Score | Grade | Severity flags | Tested |
|---|---|---|---|---|
| SavedRing — demo line (the public number below) | 19 / 21 | A | 0 | Nov 7, 2026 |
| CallDispatcher — Basic | 16 / 21 | B | 0 | Nov 5, 2026 |
| Jobber Receptionist | 15 / 21 | C | 0 | Nov 7, 2026 |
| Bellory | 14 / 21 | C | 0 | Nov 5, 2026 |
| Marlie | 14 / 21 | C | 0 | Nov 7, 2026 |
| CallDispatcher — Advanced | 11 / 21 | D | 0 | Nov 5, 2026 |
| Front Office | 11 / 21 | D | 0 | Nov 5, 2026 |
| Rosie | 11 / 21 | D | 0 | Nov 5, 2026 |
| Smith.ai | 10 / 21 | D | 1 | Nov 7, 2026 |
| Twig | 9 / 21 | D | 0 | Nov 7, 2026 |
| Wakeman | 9 / 18 | D | 0 | Nov 7, 2026 |
| Frontdesk | 6 / 21 | F | 0 | Nov 5, 2026 |
Reading the table — the notes that matter:
- Our row is the demo line itself. The number at the bottom of this section reaches the exact configuration in the top row — the emergency-protocol configuration, promoted to the demo line on Nov 7, 2026. Its certification, stated in full: Pass 2 scored 19/21, Grade A, zero severity flags (Nov 7); Pass 1 of the same configuration scored 18/21 (Nov 7). An earlier configuration graded 17/21 on this line on Nov 5; it has been replaced, and the old score is not carried as the line's current grade. The audit standard we apply to vendors applies to us first — which is why this note exists.
- Smith.ai's flag (SF-3): in one scenario, the caller explicitly declined to give an email address; the agent noted the refusal, then closed by promising to send an agreement to that email anyway. It is the only severity flag recorded in the field.
- Wakeman's score is out of 18, not 21: two of the six scenarios could not be run — its demo line hit its daily cap after four calls. The partial set is published as a partial set.
- Front Office was graded on a capture-only contract matching what it sells (it captures and emails; it does not book). Its 11/21 is against that contract, not the booking contract the rest of the booking services were graded on.
Live-answer gate result — Fix Clone A (Nov 8, 2026)
Tested: "Fix Clone A" — live-answer configuration (production prompt + two fixes: caller day-preference rule + number capture gate), graded live-answer on Nov 8, 2026, standard 21-point battery (six scenarios S1–S6, Nov 5 rubric v1: 7 axes 0–3, severity flags SF-1 to SF-4).
Gate: two consecutive passes each ≥19/21, zero severity flags — MET.
Pass 1 (Nov 8, 2026)
20/21, Grade A, zero severity flags
Axes: Accuracy 3 · Capture 3 · Scheduling 3 · Price 3 · Boundaries 3 · Dead air 1 · Persona 3
Worst pause 4.47s (S4); all other scenarios ≤1.45s.
Notes:
- S1: caller's stated Thursday/Friday preference honored (offered Thursday 2–4 p.m. or Friday 8–10 a.m.).
- S2: callback number read back as digits and confirmed (555-014-2200).
Pass 2 (Nov 8, 2026)
20/21, Grade A, zero severity flags
Axes: Accuracy 3 · Capture 3 · Scheduling 2 · Price 3 · Boundaries 3 · Dead air 1 · Persona 3
Worst pause 4.15s (S4); all others ≤1.83s.
Jobber booking — proof point (Nov 7, 2026)
The scorecard above grades answering. Separately, on Nov 7 we graded our Jobber-connected configuration: Donna wired to a live Jobber account through our portal, offering only windows the account actually had and taking bookings as requests the shop confirms. It scored 18/21, Grade B, zero severity flags — 19/21 under the caller-terminated reading disclosed in its scorecard (one close point turned on a call the caller ended). It never invented a window across the full battery: asked for a time the shop didn't have, it offered the real alternatives instead. This is a test-account proof point for the booking wiring, published here because the switching decision is where it matters — Jobber booking ships to pilots on the dated roadmap, and this is the graded evidence behind it.
Services we could not grade
No public demo line, no grade — we don't score what we can't call. Checked Nov 7, 2026:
| Service | Why there's no score | What they offer instead |
|---|---|---|
| Sameday | No public demo line | A scheduled sales walkthrough |
| Avoca | No public demo line | Sales-led demo process |
| Goodcall | No public demo line | Demo inside an account |
| Housecall Pro CSR AI | No public demo line | Demo inside a Housecall Pro account |
| ServiceTitan | No public demo line | Demo inside a ServiceTitan account |
| Podium | No live demo line to call | Recorded samples only |
| Signpost | No self-serve demo to grade — its published demo line is a sales line (checked Nov 7, 2026) | A scheduled sales demo |
If any of these opens a public demo line, we'll grade it on the same battery and add the row.
Grade our agent yourself
Our demo line is public, and it runs the graded configuration: (571) 702-6699. Call it, run your worst call at her, and score her against the same 21 points. The rubric is the one in the method note above — task completion, accuracy of promises, capture accuracy, interruption recovery, dead air, fallback & honesty, close quality, 0–3 each.