In recent years, medical chatbots have evolved from clunky symptom checkers to pretty convincing digital confidants. Studies are piling up showing that AI assistants can now help with everything from triaging patients in emergency departments to offering mental health support in chat windows at 2 a.m.—no couch required.

Take MayaMD, for instance. It’s been used in hospital settings to assist clinicians with differential diagnoses and to reduce diagnostic errors. In a 2023 study, it was shown to improve triage accuracy compared to traditional methods. Meanwhile, platforms like Woebot Health have delivered cognitive-behavioral therapy in chatbot form to tens of thousands of users, many of whom report reductions in anxiety and depression symptoms. Yes, a chatbot with better bedside manner than your last therapist.

And here’s the part where I roll up my sleeves: nearly two years ago, we built vitiligo.ai—a specialized AI assistant designed for people living with vitiligo. It’s more than just a Q&A engine. It’s a digital support system that helps users understand treatment options, connect and prepare for better conversations with their dermatologists. In supporting this tool day in and day out, we’ve seen firsthand both the power and the pitfalls of AI in a highly sensitive medical context. It’s been educational. It’s been humbling. And yes—it’s still a work in progress.

So, when news dropped last week that a large language model has “passed” the Turing Test in the wild (here’s the paper if you’re curious), it naturally raised eyebrows—and a few alarms.

In the study, participants were asked to chat for five minutes with either a human or a language model, then guess which they had spoken to. The AI fooled people more than half the time. That’s a milestone, no doubt. Alan Turing predicted this moment in 1950. But he didn’t say what we should do when it actually happened.

Cue Gary Marcus. In his typically razor-sharp blog post titled “AI has (sort of) passed the Turing Test. Now what?”, he argues that we’re celebrating the wrong finish line. Today’s large language models don’t understand the world—they simulate conversation based on patterns. They’re charismatic, confident, and often compelling… but they don’t know when they’re wrong. Worse, they often don’t care.

And in healthcare, that’s not just a bug—it’s a liability.

A medical chatbot that sounds fluent can build trust fast. That’s good—until it gives dangerously wrong advice with the same cheery tone. If it misreads a patient’s concern or fabricates a reference (we’ve seen this too many times), it’s not just an oopsie. It’s a missed diagnosis, a delayed treatment, a lawsuit waiting to happen.

This is why Turing’s test isn’t enough for medical AI. What we need is something closer to a Hippocratic Test: Can this AI avoid harm? Can it recognize its own uncertainty? Will it defer to a human clinician when the stakes are high?

That’s the real frontier—and it’s not about mimicking conversation. It’s about earning trust through transparency, accuracy, and humility.

At vitiligo.ai, we’ve taken this to heart. We’ve baked in clear disclaimers, designed guardrails, and made sure the AI knows when to say, “You really should talk to your doctor.” But even with all that, we’re constantly monitoring for edge cases, hallucinations, and the subtle creep of overconfidence. It’s like raising a very smart teenager—full of potential, but absolutely in need of supervision.

So yes, an AI passed the Turing Test. But in medicine, that’s just table stakes.

The real test? Helping people, safely and consistently, without pretending to be something they’re not.

And if we can teach our chatbots to do that? Then maybe—just maybe—they’ll be ready to sit at the grown-ups’ table in healthcare.

 

Yan Valle