Building woob's voice receptionist was the easy part, compared to trusting it enough to actually answer calls unsupervised. Vapi's own Test Suite tooling didn't fit our setup: we use an assistant request webhook pattern that builds a different assistant configuration per building on the fly, and that dynamic behavior isn't something a standard test harness is built to exercise.

So we ended up with a three layer approach instead. Unit level checks on the webhook logic itself. Scripted conversation simulations that walk through the call flows a real tenant or prospect would actually use. And a smaller set of live test calls before anything goes live for a new building.

Uploaded Image

We also spent time evaluating dedicated voice agent testing platforms, Coval, Hamming, and Cekura, to see whether any of them would replace part of that stack. Useful tools, but none replaced the need to test against our own dynamic, per-building configuration.

The unglamorous parts mattered most in practice: making sure a caller's actual phone number passes through cleanly across the whole chain, from a carrier's forwarding service through Twilio to Vapi, and making sure the assistant doesn't give a slow, sleepy first response after sitting idle. Small things, but they're the difference between "this sounds like a real receptionist" and "this sounds like a bot."