The oral Certifying Exam is a performance, not a written test. You do not study for it so much as rehearse for it. And the single most important variable in that rehearsal is not the case you are given — it is the person, or the system, asking the questions. A mediocre case examined well teaches more than a perfect case examined poorly.
Most residents understand this implicitly. They seek out mock orals because they know reading is not enough. But far fewer stop to evaluate the quality of the mock oral itself. They assume any practice is good practice. It is not. The value of a session is almost entirely determined by the proctor, and a poor proctor can do more harm than good — reinforcing habits the real examiners will punish.
Why the Proctor Matters More Than the Case
A clinical case is, in the abstract, just content. The material — acute cholecystitis, a perforated ulcer, a trauma activation — exists in any textbook. Reading about it and answering a multiple-choice question tests recall. But the oral board does not test recall. It tests whether you can think through a case out loud, in sequence, while someone interrupts you, withholds information, and challenges your decisions.
That dynamic cannot come from a case alone. It can only come from a proctor. The proctor decides what information to reveal, when to push back, how hard to push, and whether your answer is actually defensible or merely fluent. A good proctor turns a static scenario into a live examination that approximates the real thing. A poor proctor turns the same scenario into a pleasant conversation that feels productive but teaches you little about how you will actually perform.
This is why the proctor matters more than the case. Two residents can practice the identical scenario and get completely different value from it depending on who runs it. The case provides the stage. The proctor provides the pressure, the probing, and the judgment — and those are the things the exam measures.
Consider the two failure modes directly. One kind of proctor asks leading questions that telegraph the answer — "Would you like to give antibiotics?" — and then agrees with whatever you say. Another talks through the case for you, filling every silence and supplying the plan you failed to produce. Neither is preparing you. Both leave you feeling competent, and neither would survive two minutes with a real examiner, who will sit quietly and wait for you to commit.
The 7 Criteria of a Good Proctor
So what actually makes a good mock oral proctor? Seven criteria separate a useful proctor from a waste of time. They apply whether the proctor is a faculty surgeon, a co-resident, or an AI system.
1. Realistic Case Presentation
A good proctor presents the case the way an examiner would — a short opening stem, limited initial information, and a prompt for you to respond. They do not read the entire history up front, and they do not hand you the diagnosis. The case should force you to generate a differential and then narrow it, exactly as you will have to on exam day.
2. Probing Follow-Up Questions
An examiner never accepts your first answer and moves on. A good proctor presses: why that antibiotic, what would change your plan, what if the patient is on anticoagulation. The follow-up is where the oral board lives. It tests whether your reasoning holds up beyond the rehearsed opening line.
3. Structured, Competency-Based Grading
A useful proctor evaluates you against a defined set of competencies — diagnosis, workup, operative plan, complications, communication — rather than a vague overall impression. Structure makes the grade reproducible. Without it, you get a gut reaction instead of a result you can act on.
4. Specific and Actionable Feedback
General praise and general criticism are equally useless. "Good job" and "you need to read more" tell you nothing. A good proctor points to specific moments: where your differential was too narrow, where you committed too slowly, where you failed to name the life-threatening diagnosis first. Specific feedback is the only kind that changes next week's performance.
5. Consistency Across Sessions
A single session is a snapshot. Its value comes from comparison — is this session better than the last one? A good proctor grades the same competencies the same way every time, so your scores mean something over weeks of practice. An inconsistent proctor gives you numbers you cannot trust.
6. The Right Level of Challenge
The proctor should calibrate difficulty to where you actually are. A proctor who never pushes lets you coast; one who relentlessly hammers you destroys confidence without building skill. The right proctor keeps you at the edge of your ability — challenging enough to expose weakness, fair enough to let you succeed.
7. Honest Pass/Fail Verdicts
This is the criterion most proctors — especially well-meaning human ones — fail. It is uncomfortable to tell a colleague they would not have passed. But a mock oral that cannot tell you "you failed" is a mock oral that cannot tell you anything. Honest verdicts, given with the reasoning behind them, are what make practice consequential.
The Difference Between Feedback and Grading
Residents often use "feedback" and "grading" interchangeably, but they are different things, and a good proctor does both.
Grading is a verdict. It answers a yes-or-no question: would you have passed this session? It is a score or a pass/fail call against a standard. Grading is what makes a mock oral honest. Without it, you leave a session feeling vaguely better or vaguely worse, with no objective sense of where you stand.
Feedback is an explanation. It answers a different question: why did you get that grade, and what specifically should you change? Feedback is the reasoning behind the verdict — the specific moments, decisions, and omissions that produced the outcome.
The two are complementary, and each is incomplete without the other. Grading without feedback is a number you cannot act on. Feedback without grading is reassurance without a standard. A useful proctor gives you both: a clear verdict on where you stand, and a specific account of how to move.
This distinction matters in practice because residents tend to overvalue the parts of a session that feel good and undervalue the parts that are actually useful. A grade is useful precisely because it can sting. Feedback is useful precisely because it is specific. If you leave every session feeling entirely comfortable, you are not being graded, and you are not improving as fast as you could.
How to Evaluate Any Proctor (Human or AI)
The seven criteria are not just a description of a good proctor; they are a checklist you can use to evaluate any proctor before you invest hours in them. Whether you are booking time with a faculty surgeon, trading cases with a co-resident, or choosing an AI practice tool, ask the same questions.
Start with realism. Does the proctor present cases the way the ABS examiners do — a short stem, a demand for a differential, and a running series of follow-ups — or do they lecture? A proctor who talks more than you do is teaching, not examining. Teaching has its place, but it is not a mock oral.
Then look at the output. After a session, do you receive a structured assessment across defined competencies, with specific examples pulled from what you actually said? Or do you get a warm but vague summary? The answer tells you whether the proctor is grading you or just encouraging you.
Finally, look at honesty. The single clearest signal is this: does the proctor ever say no? A proctor that never fails you, never pushes back hard, and never tells you a plan is wrong is not preparing you for an exam whose examiners will do all three. Comfort is the enemy of a useful mock oral.
You should also assess calibration over more than one session. A single easy case or a single generous grade proves little. Run the proctor through a few sessions and look for a pattern: does the difficulty track your ability, does the feedback stay specific, does the verdict stay honest. One strong session might be luck. A consistent pattern is a proctor worth keeping.
Human proctors bring judgment and lived experience that no system can fully replicate. But they are also scarce, expensive, and often reluctant to be harsh with a colleague. AI proctors are always available and consistent — but they are only as good as their underlying model and prompts. The right answer for most residents is not one or the other. It is to use whatever proctor actually meets the seven criteria, and to hold all of them to the same standard.
The Bottom Line
The mock oral is only as good as the proctor running it. A good proctor presents realistic cases, probes your reasoning, grades against a structured standard, gives specific feedback, stays consistent, calibrates the challenge, and tells you the truth — including when the truth is that you would not have passed.
Before you spend hours practicing, spend a few minutes evaluating the proctor. Ask whether the session will push you, whether the feedback will be specific, and whether you will get an honest verdict. If the answer to any of those is no, find a different proctor. The hours you invest in oral board preparation are too valuable to waste on a pleasant conversation that does not prepare you for the real thing.