When the AI Interviewer Meets the AI Candidate: What Are We Actually Hiring For?

I recently came across a post about an AI-powered interview where an AI avatar was interviewing a candidate. The candidate tried to joke with the AI, the avatar reacted to it, and the whole interaction felt strangely human—and at the same time, strangely unnatural.

The post raised several questions: Is it depressing to be interviewed by an AI? What happens when a candidate has a speech disability? Can an AI distinguish between someone genuinely struggling to communicate and someone deliberately trying to manipulate the system? And if a company relies heavily on AI even for its first interaction with candidates, does that tell us something about the company itself?

I don’t think the answer is simply that “AI interviews are bad.”

The bigger question is: What exactly are we measuring when AI becomes the gatekeeper between a candidate and a human being?

When the candidate learns to interview the interviewer

Hiring has already started becoming an interesting AI-versus-AI problem.

Companies use AI to screen resumes, rank candidates, conduct interviews, analyze responses and sometimes identify suspicious behavior. Candidates, meanwhile, use AI to improve resumes, prepare interview answers, practice responses and, increasingly, assist them during interviews.

Now imagine taking that one step further.

An AI interviewer asks, “Tell me about a difficult production issue you solved.”

Another AI, sitting on the candidate’s side, generates the answer.

The interviewer asks a follow-up question. The candidate-side AI generates another answer.

At that point, the human candidate may barely be participating in the assessment.

We could eventually reach a strange situation where AI is interviewing AI on behalf of humans.

If that happens, what exactly have we measured? Technical knowledge? Problem-solving ability? Or simply the ability to produce the signals that another AI expects to see?

That is a very important distinction.

The problem with optimizing for the evaluator

We already know candidates try to optimize their resumes for applicant-tracking systems. They learn which keywords, formats and phrases are more likely to get noticed.

The same thing can happen with AI interviews.

If candidates discover that certain expressions, speaking styles, facial behaviors or answer structures produce better scores, they will naturally optimize for those signals.

And this creates an uncomfortable possibility: the best candidate at doing the job may not be the best candidate at passing the AI evaluation.

Those are not necessarily the same skill.

A candidate may be an excellent QA engineer who thinks deeply about risk, asks difficult questions and takes time to reason through a problem. Another candidate may be extremely polished, quick with answers and very good at producing the kind of responses an automated interviewer expects.

Which one does the system prefer?

That depends on what the system is actually measuring.

What happens when context disappears?

This is where I have a bigger concern about AI avatars.

Imagine a candidate is answering an important question during an interview. Their internet connection suddenly drops. They reconnect after 20 seconds and continue.

A human interviewer would probably understand what happened.

Now imagine the candidate’s microphone suddenly stops working, there is unexpected noise at home, someone interrupts them, or they need a few seconds longer to think about a difficult technical question.

A human might simply say, “No problem. Take your time.”

But an automated system may not always understand the difference between candidate behavior and circumstances surrounding the candidate’s behavior.

The candidate could potentially end up being evaluated for something they had no control over.

This becomes even more important when we talk about accessibility. What happens when the candidate has a speech impairment, hearing impairment, a neurological condition, an accent that the system doesn’t handle well, or simply communicates differently?

A person’s communication style is not necessarily a reliable measurement of their ability to perform the job.

A real interview can contain information that isn’t in the answer

Consider two candidates interviewing for a QA position.

Candidate A is nervous and takes a few seconds before answering. Their communication isn’t particularly polished, but when asked about a production incident, they explain how they isolated the problem, checked logs, reproduced the issue, challenged an assumption made by the development team and verified the fix.

Candidate B is extremely confident and gives fluent, immediate answers. Everything sounds impressive, but when asked, “What did you personally investigate when the issue occurred?” the answer remains vague.

An experienced interviewer may recognize the difference between presentation quality and actual experience.

That is one reason I believe human interaction still matters.

The goal should not be to find the person who performs best in an interview. The goal should be to gather enough evidence to determine whether the person can actually do the work.

But humans are not automatically unbiased either

This doesn’t mean humans are perfect.

A human interviewer can be biased too. They can be influenced by appearance, accent, confidence, educational background, previous companies, personality, similarity to themselves or simply the mood they are in that day.

But there is an important difference.

Human bias is not inevitable. An experienced interviewer can consciously recognize their own assumptions, ask follow-up questions and change their opinion when new information appears.

AI can have a different problem.

AI systems operate according to the data, criteria, models and rules used to build and evaluate them. If historical hiring data contains patterns or preferences that shouldn’t have been there, those patterns can potentially become embedded into the system and then applied consistently at scale.

So I don’t think the argument should be “humans are biased and AI is unbiased” or “AI is always more biased than humans.”

The better question is:

What assumptions have we encoded into the system, and who is checking whether those assumptions are still valid?

Who defines what a “good candidate” looks like?

This question becomes particularly important when AI is asked to score things like confidence, communication or leadership potential.

What does “confidence” actually mean to the system?

• Speaking quickly?
• Maintaining eye contact?
• Smiling?
• Using certain words?
• Answering without pauses?
• Speaking in a particular style?

Those definitions don’t appear by magic. People design the evaluation criteria. Engineers build the system. Data scientists select and validate data. Organizations decide what outcomes matter.

So when an AI evaluates a candidate, we shouldn’t only ask, “Is the AI biased?”

We should also ask:

“Whose definition of a good candidate has been encoded into this system?”

Historical data can also become a trap

There is another issue that concerns me even more as technology changes rapidly.

AI systems are very good at finding patterns in historical data. But the future doesn’t necessarily look like the past.

Imagine a company trains its hiring process using the characteristics of employees who succeeded during the last five or ten years.

The system learns what successful employees looked like historically.

But technology has changed dramatically during those years. AI-assisted development, AI evaluation, new automation approaches and entirely new roles are changing what technical teams need.

The person who will be extremely valuable tomorrow may not resemble yesterday’s successful employee.

This is where human judgment becomes particularly valuable.

An experienced interviewer can encounter something that doesn’t fit the existing pattern and say, “This is unusual. Let me understand it better.”

A rigid system may simply classify the same candidate as an anomaly.

Innovation often comes from people who don’t perfectly match the existing pattern.

If we train our hiring systems only to recognize yesterday’s successful employee, how will they recognize tomorrow’s?

The first impression problem works in both directions

This is probably the biggest reason I question AI being the first interviewer.

Consider the company’s side.

AI interviews the candidate and generates a report:

“Strong technical responses. Moderate communication. Several long pauses. Confidence concern.”

Then a human interviewer meets the candidate.

The human may believe they are starting with an open mind, but the AI has already created a narrative.

Now the interviewer may unconsciously look for evidence that confirms what the AI reported.

Instead of:

“Let me meet this person and understand them.”

the process can become:

“Let me see whether the AI was right.”

That is a very different starting point.

Now consider the candidate’s side.

Their first interaction with the company is an AI avatar.

They may immediately think:

“This company relies heavily on AI.”

Then:

“I wonder how much human judgment exists here.”

And perhaps:

“If this is how they operate, maybe I should use AI for everything too.”

So the candidate is also forming a first impression of the organization.

The interview is not only the company’s opportunity to evaluate the candidate. It is the candidate’s opportunity to evaluate the company.

Perhaps the human should come first

This is why I currently lean toward a different model.

Not:

AI interviewer → human interviewer → decision

but:

Human conversation → AI assistance → human evaluation

The first interaction could be with a recruiter or interviewer. The human gets an opportunity to understand the candidate without first being anchored by an AI-generated score.

After that conversation, AI can still be extremely useful.

It can summarize notes, identify areas that weren’t explored, suggest follow-up questions, compare evidence against defined competencies and help the interviewer avoid forgetting important points.

For example, after a QA interview, AI could say:

“The candidate discussed UI automation extensively, but API testing and debugging methodology were not explored. Consider asking about those areas.”

That is a useful role for AI.

The AI isn’t deciding whether the candidate is good.

It is helping the human ask better questions.

Who interviews the interviewer?

There is another question we rarely ask.

Companies are increasingly asking AI to evaluate candidates.

• But who evaluates the AI?
• Who tests whether the AI interviewer is producing false positives and false negatives?
• Who checks whether it performs differently with different accents?
• Who validates accessibility?
• Who checks what happens when the candidate’s internet fails?
• Who verifies that a technically strong but unconventional candidate isn’t being rejected simply because they don’t resemble historical candidates?
• Who tests whether an AI candidate can fool the AI interviewer?

As someone from a software testing background, this question immediately comes to mind:

What are the test cases for the AI interviewer?

If we would never release a production system simply because an AI said, “It passed,” why should we reject a human candidate simply because an AI said, “They failed”?

AI should have an appeal mechanism too

There is one principle I would like to see in AI-assisted hiring:

A candidate should have a path to human review.

If a technical failure, accessibility issue, unexpected interruption or incorrect AI interpretation affected an assessment, the candidate should have a way to request human consideration.

Automated decisions shouldn’t become irreversible simply because the system produced a score.

In software, we have retries, logs, error handling, escalation paths and human intervention.

Why shouldn’t high-impact hiring systems have similar safeguards?

This isn’t an argument against AI

I don’t think the answer is to remove AI from recruitment.

AI can help recruiters handle enormous application volumes. It can reduce repetitive work. It can summarize information, identify missing areas, structure interviews and help teams spend more time on meaningful conversations.

The concern begins when efficiency becomes more important than accuracy.

There is a huge difference between:

“AI helps us process 50,000 applications.”

and:

“AI decides which 500 people deserve human attention.”

The first is automation of workload.

The second is automation of opportunity.

That distinction matters.

The future shouldn’t be Human vs AI

Perhaps the better question isn’t whether humans or AI should conduct interviews.

It is:

What should AI be allowed to decide, and what should always remain a human responsibility?

AI can process patterns.

Humans can challenge patterns.

AI can summarize evidence.

Humans can investigate context.

AI can suggest questions.

Humans can have the conversation.

AI can identify inconsistencies.

Humans can decide whether those inconsistencies actually matter.

And most importantly, humans can recognize when the existing model itself may be wrong.

That last capability may become increasingly important as technology changes faster than historical data can keep up.

So, what are we actually hiring for?

If an AI interviewer can be fooled by another AI, if candidates learn to optimize themselves for the scoring system, if historical data determines what a “good candidate” looks like, and if unexpected human circumstances can be interpreted as negative signals, then we need to step back and ask a fundamental question:

Are we hiring people who can do the job, or people who are good at passing the hiring system?

I don’t have a problem with AI being part of the hiring process.

I have a problem with the possibility that the hiring process itself becomes the thing candidates have to learn to beat.

Maybe the best model is not:

Human OR AI.

Maybe it is:

Human first. AI as a copilot. Human makes the final judgment—with evidence, context and the ability to challenge the machine.

Because when someone’s career is on the other side of that decision, efficiency should never become a substitute for understanding.

Bookmark