How should a clinic test an AI receptionist?

On a real phone, with awkward scripts, checking what it did and not just what it said. The tests we run before anything reaches a patient.

Test it on a real phone, with the awkward conversations your patients actually have, and check what it did afterwards, not just what it said. A demo where the AI answers three polite questions tells you almost nothing.

Use a real phone, not a test panel

Most AI tools come with a test window. It’s useful for wording, and it can mislead you about everything else. In our own testing, the test panel misled us about how a booking request played out across several messages; the live text test was the one that told the truth. The test that counts is a text from your own phone to the clinic number, or a real call, with the hand-offs and notifications switched on.

Write the awkward scripts

Polite single questions are easy. These are the conversations that find the problems:

  • three texts in a row, sent seconds apart, before the AI has replied to the first;
  • someone asking for their partner: “Can I get my wife in this week?”;
  • a late reply to yesterday’s message;
  • “Are you a robot?”;
  • a complaint, or something clinical the AI shouldn’t answer;
  • a message at 11 p.m. on a Saturday;
  • someone who wants a person and says so.

Check what it did, not what it said

After each script, look at what happened, not just the reply. Did the right person get notified? Did the conversation land in the callback list? Did anything change that shouldn’t have? While testing a feature that isn’t live yet, an agent told a test patient a cancellation was taken care of when the system behind it had refused it. The reply read perfectly. Only the check afterwards caught it, and it was fixed before any patient saw it.

Test on a practice account first

We test changes on a demo clinic with test patients before it goes near a real one, and the release test is a fixed script run on a real phone. When it passes, the change goes out. When it doesn’t, it doesn’t.

Re-test after every change

A small change to the instructions can break something three steps away. Keep the scripts, and run them again every time.

Go deeper

Read more

See it working against your own Jane setup

Book a short demo and we'll walk through how Emma handles your patient conversations.

A demo is a live call with a founder — not a sales rep.

  • Live in 5–10 business days
  • Jane stays exactly as it is