I Don't Know, and It Depends

Case of the Week, Min Wu, PhD · ai-public-health.com

Every semester, before my students present, I give them the same tip for the Q&A that follows. There is a best answer and a second-best answer, and neither is the one they walk in hoping to give.

The best answer is "I don't know." It is honest, and it saves the room a lot of time — nobody has to sit through five minutes of confident invention while everyone quietly wishes we could move on to the next question.

The second-best answer is "it depends." For years I treated it as the consolation prize — what you say when you don't have the real answer. Lately I've started telling students it might be the more useful of the two, because "it depends" is turning out to be the honest answer to almost everything involving AI.

Key Takeaways

• Min Wu argues trusting an AI's answer isn't a single yes/no call — it depends on the question type, your expertise, and the tool.
• Expertise matters more with AI, not less — a polished answer gives no signal about whether it's actually right.
• The riskiest setup: a judgment call, answered by a novice, using the most polished tool — nothing there can catch a mistake.

I share AI chatbot scenarios with my students the same way I'd share a tricky case study, and the question they always ask is some version of: should I trust what it just told me?

My answer is always the same. It depends. Not as a dodge — it depends on three specific things, and once you can name them, you stop asking "can I trust AI" as if it had one answer.

1.What Kind of Question You're Asking

Some tasks have a right answer that exists whether or not you or the AI agrees with it. Does the code compile. Is the math correct. Did the sentence translate accurately. If your task is one of these, you're in good shape almost no matter what else is true — check the output against reality, and you don't need to trust anything.

Other tasks don't work that way. What's the strongest argument for my thesis. What's the right call in this case study. These are judgment calls, and there's no answer key to check against. This is where the other two questions start to matter.

2.How Much You Already Know

I've watched two students run the identical prompt through the identical chatbot for a case-study assignment. One had taken three related courses. The other hadn't taken any. Both got back a fluent, well-organized answer. Only one of them could tell that the AI had quietly skipped the hardest part of the question.

That's the part that surprised me most, watching this happen semester after semester: expertise matters more with AI, not less. A polished answer gives you no information at all about whether it's actually right — only your own knowledge of the subject can catch what the polish is hiding.

3.Which Tool You're Using

Some AI tools are ones someone can open up and inspect — trace exactly why they answered the way they did, or retrain the parts that keep getting it wrong. Others you can only access through an app or a website, take the answer, and that's the whole relationship. Neither is automatically the safer choice. But this one matters less than students expect, because being able to "look inside" a tool only helps if you have the skill to use that access. For most students, most of the time, it's the first two questions that decide whether an answer deserves trust.

Where It Actually Gets Risky

Here's the combination I'd flag if a student only remembers one thing: a judgment-call question, answered by someone new to the topic, using the tool that feels the most trustworthy because it's the most polished and the most familiar. That's the riskiest pairing there is — not because the tool is bad, but because nothing in that setup is positioned to catch a mistake. Not the student, who doesn't know the subject well enough yet. Not the task, which has no answer key. The confidence in the writing is doing all the convincing, and confidence was never the thing that made an answer true.

The mirror image is the strongest pairing there is: someone who already knows the subject, using more or less any tool at all. Expertise catches the mistake in the moment, full stop. Which is also, I think, the real answer for a student who doesn't have that expertise yet — not to give up on the assignment, but to go find someone who does. A professor, a TA, a source you can check the AI against. That's not a lesser move than trusting the chatbot. It's the same move as saying "I don't know" out loud instead of guessing: knowing exactly where your own knowledge runs out, and doing something about it instead of covering for it.

Fluent is not the same as right.

Observer's Insight

I keep coming back to why "I don't know" and "it depends" are the two answers I actually want in that Q&A, out of everything a student could say. They're not the impressive answers. They're not even, on the surface, the confident ones. But they're the two answers that are honest about the edge of what someone actually knows — and that edge is exactly what a fluent AI paragraph will never show you on its own.

What I'm less sure of is whether "go find someone who does" is actually available to every student at the moment they need it. Office hours have limits, and not every subject has an obvious person standing by to ask. I don't have a clean answer yet for the student who is genuinely on their own with a judgment call and no expert in reach.

Maybe that's the real thing I'm trying to teach in that Q&A, long before AI ever entered the room: the willingness to say where your knowledge runs out is worth more than the ability to sound like it doesn't.

As a member of the Springer Nature Author Affiliate Program, I may earn a small commission from purchases made through this affiliated link to my book. Support the author by checking out my textbook, Artificial Intelligence in Public Health: https://tidd.ly/4mH9389