The Rise of ChatGPT Interviews

Remote interviews is a stalwart of the software engineering industry for years. It provides a low-cost way to vet candidates without making a heavy commitment. Once remote work took over, it even became common to interview and hire someone without ever seeing them in person. Unfortunately, as new ways of doing work rise, there will always be those who look to game the system.

In recent years, with the rise of ChatGPT and Generative AI, I noticed a suspicious trend of interviews that felt inauthentic and robotic. It’s quite obvious that interviewers are using AI during the interview process and it unfortunately is a drain on interviewers’ time. I suspect there are even organizations taking advantage of this and creating pools of workers that are taking advantage of companies by not only using ChatGPT during the hiring process, but using AI and “assisted” work on the day-to-day job.

Detecting AI During Interviews

Unfortunately, I’ve seen my fair share of interviews where I was confident the candidate was using AI. There are a few tell-tale signs of this. Watch for these clues to know if you’re candidate is authentic or not.

Pause, Stutter, Roll

This is possibly the most obvious sign of AI during interviews. When asked a technical question, the candidate will pause and stutter. It’s obvious they don’t know how to answer. They will notably pause for many seconds and then out of nowhere, start rattling off an answer that is far too polished. It is subtle and you may not catch onto it right away. But this long pause, followed by an improbably refined response on many questions is a dead giveaway.

If a candidate is authentic, they would start answering right away with some details, possibly pause to think, then finish their answer with additional details. They won’t necessarily use perfect terminology, but will provide a conceptually correct answer. ChatGPT candidates will initially sound like they have no clue, but then roll a solution using precise terminology and excessive detail.

Scanning Eyes

Growing up, when I saw celebrities doing commercial endorsements, it was obvious when they were reading off cue cards because you could see their eyes scanning left and right. I see the same with many interviewees that I suspect are using ChatGPT. I can see their eyes scanning text and while it may be subtle, the signs are there. Your usual candidate will do their best to look at the interviewer when answering questions. When pausing to think, they may look up. These are all normal responses.

Candidates using ChatGPT can’t hide their eyes. Look out for the subtle scanning of the eyes as if they’re reading text. Combined with other signs described here, you can get more confidence that your candidate isn’t as real as they look.

Polished, Unnatural Responses

If your candidate sounds like they are reading from a book, chances are they are. The authentic candidates will provide detailed answers, but as an interview is an impromptu set of questions, they won’t be prepared to provide dictionary-like responses to technical questions. This is normal. You want to see if they are conceptually sound and have a good foundation of concepts and technologies.

But when the responses are nearly pitch perfect and often go into excessive detail, you can use it as a sign of AI support. They will reference perfect terminology and detail concepts that are unrealistic for every candidate to be able to recite perfectly.

Code Perfect

I always ask coding questions during interviews. Even if it’s a first-round screening interview, I’ll ask a basic question to vet out their comfort writing code. I’ve done enough to know how candidates will approach these questions and what is a practical response.

When I see candidates write responses that are way too polished and adding precondition and edge case checks from the beginning that I wouldn’t even consider until the core algorithm is written, I know they are using AI. I will feed my questions into AI tools to see what they spit out and can tell when a candidate is using these tools if they create nuanced answers that go beyond what most candidates may consider in an interview.

If a candidate is obviously strong and shows a lot of initiative, these nuanced responses may be expected. But at the point I ask the coding questions, it will be clear if the candidate has that kind of potential or is exceeding their abilities.

Other Unusual Interview Signs

There are a few other signs that could signal an inauthentic interviewer. These signs normally don’t mean anything on their own, so I only rely on these if one of the above, more direct signs of AI use in interviews is present. These are factors that build more confidence of a suspect interviewer.

Suspicious Environments

I like for an interviewer to have their background showing and not using a custom background, which hides their environment. If I’m interviewing for a company, I want to give perception that I’m real and authentic. I will not use a fake background or blur my background. When I see a candidate using a custom background, I immediately start to question if they are authentic.

I suspect they are in some kind of call center or environment where they are being coached. While I don’t have concrete evidence of this, I frankly wouldn’t be surprised if there are organized efforts to take advantage of companies with inauthentic candidates. Sometimes there are unusual noises or the sense that other people are in the environment. All are signs of a candidate looking to cheat the system.

The John Smith Candidate

Now this might come off as racist but when a candidate has a very common, western name (“John Smith”) but is a non-native English speaker and has an Asian, African, or other Non-European background, I start to get suspicious. The candidate has a name that is common in the country he is interviewing for, but is uncommon in his country of origin. Now I totally know there are many countries around the world where Christianity is prevalent and western names are common, but I’ve seen way too many candidates doing this that I can’t feel confident of their authenticity.

I will not make this judgement on its own. Only if I see other signs of potential AI use or inauthenticity, do I factor in this as a possibility. I just happen to see this way more often than I think is reflected in the general population to believe these candidates are 100% authentic.

Post-Hiring Weirdness

Finally, there’s a chance that you or someone else at your organization may miss the signs of a suspicious candidate. They can be deceiving if they answer all your questions exceptionally well and appear to be a strong candidate. Be sure to keep a close eye on all your candidates in the early days and perform frequent check-ins. You want to get the sense that they are in learning mode and doing their best to assimilate into your organization’s culture.

Watch for pecularities like the candidate relying on external parties to get guidance on tasks via screen sharing or other mechanisms. There are some clever folks out there and they will do their best to take advantage of companies.

Conclusion

To summarize, here are signs you should look for to know you’re interviewing ChatGPT:

1. Unnatural pauses: The candidate pauses for many seconds before rolling out a crystal clear answer.
2. Scanning eyes: The candidate scans their eyes as if they’re reading a document.
3. Polished responses: The candidate provides polished responses that are unnatural for an interview.
4. Perfect code: The candidate answers coding questions with perfect code that is unnatural for an interview.
5. Suspicious environment: The candidate obscures his background to hide his location.
6. Suspicious name: The candidate’s name is common in your country, but is not common in his country of origin.

It’s unfortunate that there are fraudsters out there who will take advantage of companies in unsuspecting ways. With the rise of remote work, it’s not as easy to police your employees as it was before. Yet, this is no reason to toss in the towel on remote work. Adapt and be on the lookout for suspicious signs. In my experience, the signs of candidates using Generative AI and other sneaky tactics are there if you keep a close eye out.

Stay vigilent!