Guide Chapters
- Why interviewer training is so important
- 5 fundamental blocks of the interviewer training process
- Creating your interviewer training framework
- The four-step framework
- Evaluating your existing processes
- Enable upfront knowledge sharing
- Onboarding and training interviewers
- Calibrate, track, adjust, repeat
- The future of interviewing is human
Share This Guide
At this point, we’ve told you all the reasons you need to implement an interviewer training framework in your organization. We’ve given you the key building blocks that define a good process. But we’ve not told you how to build that process, and the steps you need to take to start creating or optimizing your training.
Enter our four-step framework to help you do just that. Wherever you are on your interviewer training journey, this framework is designed to do three things:
- Equip you with actionable next steps you can start implementing in your own organization.
- Provide you with the expert examples and insight to understand how top-performing organizations approach their own training.
- Help you develop the buy-in and shared ownership you need to create a sustainable, scalable interviewer training process that becomes embedded in your organization’s hiring ecosystem.
The four-step framework
At the evaluation stage, you’ll take a deeper look at your existing processes, and evaluate what’s working — and what’s not. You’ll define what ‘good’ looks like at your organization, and perform job analyses to help define your question and answer rubrics in step two.
In the knowledge-sharing stage, it’s all about front-loading the resources and information your trainees need to get started. You’ll need to create question and answer rubrics, define your training tracks, and get buy-in across your organization.
Once you’re at the onboarding stage, it’s time to kick off your process. Identify your best interviewers, create peer-driven shadow paths, and track the impact of training.
By the time you reach calibration, you’ll need to think about what defines interviewer training success for your organization. We’ll outline the metrics you can track to keep your process on track. We’ll also outline how to maintain quality and consistency, recalibrate your interviewers, and define cross-functional ownership of your process.
Let’s get started.
Evaluating your existing processes
Building a great interviewer training process relies on knowing exactly what your organization is optimizing for. This means evaluating your blind spots, understanding what ‘good’ looks like for you, and outlining the structure of your interview process. To do all of that, you’ll need to take a deep dive into your hiring data.
Step one: Key goals
Identify blind spots. Use data to identify strengths and weaknesses in your existing process.
Define what ‘good’ looks like. Understand what you’re optimizing for in your training process to help you define your criteria for success.
Perform a job analysis. Define the qualities, skills and knowledge needed for each role, and what a great candidate looks like in each.
Identify blind spots in your hiring funnel
The best way to get a feel for the success of your interview process is by taking a deep dive into your hiring data. Use a mix of qualitative and quantitative data pulled from your ATS, candidate experience surveys, and social media, to inform your understanding of any snags in your existing process.
- Hiring: Examine how successfully you’re converting candidates in detail. Look at time-to-hire, time-to-fill and offer acceptance rates.
- Candidate experience: Look at post-interview surveys and candidate Net Promoter Score (NPS). Take a look at your Glassdoor score for qualitative data on candidate experience.
- Operational: Review the average amount of time it takes to schedule an interview with internal stakeholders, and the average time delay in organizing an interview due to scheduling conflicts. Identify your current population of certified interviewers compared to the amount of interviews.
- Interview: Review scorecards to identify if any existing interviewers tend to skew high or low when evaluating candidates — try segmenting this by interview stage. If you have interview analytics tooling in place, look at conversational metrics, including amount of questions asked, candidate share of voice, and consistency across interviewers.
If you’re already running interviewer training, you can analyze the amount of time it takes to train one interviewer or schedule training in. You can also ask existing employees for qualitative data on how they feel the training went, and if they felt prepared going into the interview room.
For larger organizations, there may be additional levels of complexity across office locations and geographies or departments, resulting in blind spots across your interview process.
“One of the more challenging aspects of managing a consistent interview process for larger organizations centers around visibility and localization of hiring. For example — how do you make sure your APAC team isn’t hiring according to a completely different criteria to your US headquarters?
This complexity can make it challenging for organizations to maintain their core culture as they make new hires in different geographies, and keep interviewers calibrated across office locations and departments.”
Siadhal Magos
CEO, Metaview
Define what good looks like for interviewer training
Good interviewer training looks different for every organization — it’s the reason why a well-trained interviewer at The Walt Disney Company is unlikely to be evaluating candidates the same way as one at Tesla.
When interviewers are coming into your training process with varying levels of experience from different organizations, there’s a very strong chance they won’t be calibrated for what good looks like at yours.
Think of this as defining the common language that everyone entering your training process needs to learn to stay aligned on giving a good interview. So how can you define what you’re optimizing for with your interviewer training process?
- Identify your interview process success metrics: Consider your current process, and your markers for success in an interview context. What makes a successful interview process at your organization? What should your candidate experience look like? What training do your interviewers need to uphold this process?
- Know what you’re optimizing for: Consider your highest performing interviewers and the bar you’re trying to set with your training. Well-trained interviewers are consistent, rigorous, relevant, and calibrated. They go beyond asking behavioral questions, and identify high quality hires. They sell your organization and its values, and create a great candidate experience.
- Define criteria for success: Define what good looks like for each of your criteria. For example, the definition of success for a consistent interviewer could be that your interviewers evaluate candidates using the same framework and criteria each time. They demonstrate a strong understanding of rating scales and follow a structured process that eliminates bias.
Perform a job analysis
Multiple studies show that the job analysis is an essential component of a more consistent and fair interview process.[15] This is because having a clear understanding of a role and its purpose gives hiring panels a clear overview of what they need to look for in an interview, reducing the likelihood of discrimination and bias.
Performing a job analysis of each role you plan to hire means you can develop a set of questions that drives consistency in your interview process, as well as whole-team alignment.
- Identify key qualities, skills, and knowledge: Define the qualities, skills, competencies, soft skills, and knowledge you’re looking for alongside hiring committees and managers. If it’s a new role, understand any gaps in the existing team and identify the business objective for the role.
- Define assessment criteria for candidates: Once you have a clear idea of the traits you’re looking for in a candidate, you’ll need to build a framework of what poor, good, and exceptional candidates might look like in each of the key traits.
[15] Campion, 1997.
Enable upfront knowledge sharing
Creating a successful interviewer training process relies on making sure you have a defined structure, roadmap and set of resources before you kick off your training. This will make your process more efficient and provide a clear direction of travel for your trainees.
Step two: Key goals
Define training roadmap and resources. Use the data you collected in step one, plus the structure of your interview process to build out a training roadmap and set of accompanying resources
Build question rubrics. Build role- and department-specific question rubrics that help your interviewers align on what ‘good’ looks like when evaluating candidate answers.
Communicate your ‘why’. Get your interviewers invested in the importance of interviewing well and its positive impact on your organization.
Define your training roadmap and resources
Being strategic about your training structure from the beginning will mean you have a clear roadmap for trainees to follow, and that there will be no room for mistakenly training interviewers on potentially redundant topics. It also means you can set clear expectations for your trainees on how long the process will take, and any milestones that they’re required to hit along the way.
Segment your training process into different training tracks that build specific contextual knowledge, like in our example structure below:
- Basic: Consider this your 101 training that sets out the basic principles every interviewer needs to know, no matter their experience or department. Make sure this outlines your training roadmap and structure, includes practical instructions such as how to use your ATS, and communicates your ‘why’ for training.
- Compliance: Introduce key concepts around how to avoid discriminatory interview behaviors and questions. Many organizations also take this opportunity to provide some unconscious bias training — remember that this should be supplemented by an organization-wide DEI effort.
- Behavioral: Interviewers must learn the core skills to ask behavioral questions, evaluate candidate answers consistently, and understand rating scales.
- Technical: Training should focus on helping interviewers direct conversation around how candidates approach a question or problem, evaluate coding interviews, and provide high quality feedback.
You’ll also need to consider different resource formats and ways of distributing your resources to trainees. We recommend a combination of collaborative and individual learning opportunities as well as a range of formats, including live training, tech-enabled micro-learning or access to learning platforms, and peer-driven training in the form of shadow paths.
Build question and answer rubrics
Once you’ve built your roadmap, you’ll need to define your question banks and answer rubrics for each role. This will give your interviewers a clear set of guidelines to follow on what questions to ask, what attributes to look for, and what level of performance is required for each question. Check out our Interview Questions Hub to help you get started.
Defining this framework will help interviewers stay calibrated at an individual and a group level. It will also provide structure and objectivity to decision-making processes, reducing bias, prejudice and interpretation.
This will depend on the level of structure in your hiring process — but whatever you choose, remember that consistency is key. Trainees and interviewers must apply the same evaluation criteria across different candidates.
- Leverage the expertise of the hiring team: Good rubrics are based on collective experience and subject matter expertise. Align with key members of the hiring team — either hiring managers, or talent magnets that are deeply aligned with your organization’s hiring philosophy — to get a sense of what a high quality candidate looks like, and the attributes you’re looking for.
- Categorize key criteria into hiring pillars: Use these qualities to identify the categories within each role that are important to test for to reveal high quality candidates, and design questions that roll up into these pillars.
- Define passing criteria: You’ll need to help interviewers understand how to assess candidate answers to interview questions with a rubric. Rubrics can range from basic pass/fail approaches to outlining a typical answer. Defining your passing criteria for each question creates structure and consistency, meaning bias is less likely to enter the decision-making process. It also means that in the case of a technical interview in the case of a pass/fail task, interviewers know to ask more questions about a candidate’s approach or problem-solving ability. Scorecards or recordings from previous interviews can be powerful learning tools here to help trainees unpack what existing interviewers perceive as your passing criteria.
“We needed a way to thread the consistency of our passing criteria throughout our hiring process to give our interviewers the knowledge to understand how to reveal an A player candidate.
We did that by being really structured in the interview questions we asked. We worked with hiring team members to categorize the key pillars that were important to test for — so in engineering we were looking at topics like algorithms, data structures and design, for example. We wrote questions that rolled up into those pillars — and worked with the engineering team to make sure that these questions would help us identify those top candidates.
We designed our questions and rubrics to be a dialogue where the interviewer got a true signal of what it would be like to work with that person — how their brain worked, what questions they asked, how they engaged the interviewer across the table. That, to us, felt more authentically representative of how someone would be when they joined our team.”
Angela Miller
Head of Recruiting, Instabase
Communicate your ‘why’
‘Whys’ are important. They help teams connect to a shared purpose, and give them a collective reason to care about its success. In organizations, the ‘why’ connects your team members to your mission — whether that’s to help people belong anywhere (Airbnb), or to enable human life on Mars (SpaceX).
The same goes for your interviewer training process. Your trainees need to be invested in how interviewer training will connect to your organization’s success, and how interviewing well will become their responsibility. That means you need to help them visualize how their training and high-quality interviews will positively impact the product, the people they get to work with, and ultimately, the success of the company.
You’ll need to communicate this message to both interviewers and senior stakeholders to get buy-in across your organization:
- Getting interviewer buy-in: Outline some of the key principles as part of your basic training. Focus your messaging on how interviewing well impacts them personally, such as getting to work with talented new employees and building a better product. Embed this across a variety of platforms, including live training sessions, email, Slack, and training resources.
- Getting senior leader buy-in: Senior leader buy-in is vital to the long-term success of your training. Create a slide deck that focuses on the positive impact of investing in interviewer training at a whole-organization level. Highlight its positive impact on your quality of hires, acceleration of product development, and profitability.
For Angela, getting buy-in to training was paramount — not only for driving understanding in the importance of the process, but also pride at being a part of building a great company.
“It was really important to us to create an infrastructure — a muscle — under our recruiting philosophy, so that it didn’t become a case of recruiting against the business. We needed everyone in the process to understand that maintaining our quality bar for hiring was key to maintaining our culture, quality of product and customer experience.
We communicated this at the start of our training. We focused on explaining how we interview and why we do it this way — but it was also really important for us to connect that back to our trainees’ personal investment in the company.
We wanted our trainees to be really proud to be a part of our interview community, and emphasized the impact they would have on the future of the company by helping select the talent that comes in the door. It was a galvanizing rally to get people excited to do the training — it gave them a clear purpose and mission.”
Angela Miller
Head of Recruiting, Instabase
Onboarding and training interviewers
Once you’ve built a process roadmap and structure, it’s time to kick off your training process. Here, you’ll need to create a clear movement path through your process, and enable collaborative learning processes that drive practical knowledge transfer.
Step three: Key goals
Evaluate your interviewers. Use data to understand your team’s level of experience with interviewing and evaluate trainees new to the process.
Create shadow paths. Enable peer-driven learning by pairing up trainees with mentors and fostering shadow paths across a range of experiences.
Schedule training and track progress. Create a reliable scheduling and tracking framework to project manage training.
Evaluate your interviewers
Before you can start training, you’ll need a better overview of your interviewers’ differing levels of ability. You’ll also need to identify the interviewers that best uphold your quality bar, candidate experience and company values, as they will help model your best practice to your trainee population.
You’ll need to split your trainees into three groups:
- New trainees: This group includes first-time managers, or employees that have never interviewed before.
- New-to-you: This group includes employees who have experience interviewing in the past, but require calibration.
- Best interviewers: This group will set the bar for your interviewer training and reverse shadowing.
Not sure where to start? If you don’t have hard data on your interviewers’ performance to hand, Siadhal recommends taking a closer look at your historical hiring data to identify the best interviewers in your existing population.
“Historically, organizations have evaluated interviewers based on information gathered outside of the interview room — one example might be that if someone communicates well in a meeting, they’re likely to be a good interviewer.
This isn’t a completely invalid judgment, but it is deeply subjective of who an organization feels best represents their culture.
Now, we have much better data to inform that decision. Let’s say, for example, you have interviewers who tend towards extremes — they’re more likely to say ‘yes’ or ‘no’ all of the time. Taking a look at how decisions have been made in the past can help you identify patterns of behavior.
Look for interviewers that have a more consistent distribution in how they evaluate candidates. Identify the interviewers that get pulled into the most hiring loops, and dig into your offer acceptance rate. You’re likely to find patterns that help you identify your best interviewers.”
Siadhal Magos
CEO, Metaview
Create shadow and reverse-shadow paths
Research based on social learning theory demonstrates how powerful behavior modeling training can be when it comes to long-term skills development and behavior change.[16] In the interviewer training process, this can be achieved with shadowing and reverse-shadowing techniques.
Shadowing is an observational form of learning, when a trainee interviewer sits on an interview, observing a talented peer in action. In reverse shadowing, the trainee interviewer conducts an interview while an experienced interviewer observes and provides feedback after the interview.
Shadowing is powerful, because it fosters a collaborative, engaged process where trainees are provided with actionable next steps on how to improve their performance, while certified interviewers consistently sharpen their skills.
Creating effective shadow paths relies on five key principles:
- Pair trainees with best practice examples: Pair trainees with your best interviewers to shadow their interview technique and get a frame of reference for what great looks like at your organization. An interview analytics platform can be a particularly powerful tool at this stage, because you can hand-pick your best interview recordings that guarantee high quality self-directed learning without the need for manual scheduling.
- Provide opportunities to see skills in action: Trainees require exposure to a range of core interview techniques. Shadowing is a great opportunity to model key practical and skills-based techniques, including asking open-ended questions, how to steer candidates, time management, probing for more detail, and learning how to give candidates room to answer. For technical shadowing, trainees will also need all the resources to hand, including any code written or designs drawn.
- Create alignment and calibration: Similarly to observing new skills and techniques, trainees need access to a range of interviews of candidates with differing abilities to help calibrate them on why decisions are made, and give them context on different methods of evaluation.
- Create opportunities for reverse-shadowing: After shadowing, trainees need opportunities to practice what they’ve learned, in a mock interview or live interview process. This should give them the opportunity to put specific elements of their training into action — such as asking a specific question, or understanding how to ask follow-up questions.
- Share data and feedback: Trainees must be given specific, relevant, and timely feedback on their performance after they participate in shadowing and reverse-shadowing sessions — this will help them connect the feedback to their actions and behavior. Give interviewers data on how their candidate evaluations stack up against those of their peers, as well as their track record for predicting great hires. You can also use an interview analytics platform to provide specific analysis on development points like low-rigor questions, or candidate share of voice — this can be particularly powerful when coupled with an example transcript or video recording.
[16] Paul J Taylor, Darlene F Russ-Eft and Daniel W L Chan, ‘A meta-analytic review of behavior modeling training’, Journal of Applied Psychology (2005).
For Angela, shadow pairings were a priority for driving collaborative learning and gating the quality of knowledge transfer.
“In every training cohort, we would assign trainees with a one-on-one mentor from a previous interviewer training group. The trainee would shadow their mentor in a couple of interviews with real life candidates to see applicable examples of their specific training track.
Then, the mentee would work through some mock dialogue training with their mentor. Once the trainer felt they’d mastered things at that stage, we scheduled the trainee in for a live interview.
In our training model, the mentor ultimately held the gate code on whether or not this person could be certified to ask that question. That mentor relationship was instrumental in driving a collaborative dialogue, but also it really reinforced our passing criteria to our trainees.”
Angela Miller
Head of Recruiting, Instabase
Schedule and track progress
Scheduling in shadowing interviews and tracking your trainees’ progress can be one of the more challenging aspects of managing their training — especially when you’re managing multiple processes at once.
However, when you’re rigorous with tracking training progress, it means you have a clear overview of where your trainees are in the process. This means you can reliably maintain your quality bar, and identify any trainees that require additional shadowing sessions or training.
There are a few options for tracking this process:
- Spreadsheet or manual processes: For leaner training processes, spreadsheets and scheduling are a reliable — if not fast — manual tracking method that helps move trainees through the training funnel.
- ATS: Angela recommends setting up mock interview shadowing workflows within your ATS. As trainees fill out scorecards and leave feedback, you have a clear view of their progress.
- Interview analytics platform: Some interview analytics platforms — like Metaview — remove operational bottlenecks with automated shadow paths based on your best interview recordings, while providing recruiting teams with full visibility of training progress.
Calibrate, track, adjust, repeat
Once your training process is up and running, it’s going to be pretty gratifying to watch trainees remove the proverbial training wheels and lead interviews with confidence and rigor. It’d be easy to stop here, but we’ve got one more essential step for you: Calibration.
Calibration is the process of developing the skills needed to evaluate and score candidates fairly. It’s essential to making sure your interviewers are evaluating candidates consistently — both at an individual level, and across your organization. It’s an ongoing process, rather than an end point.
By now, you’ll understand why regularly calibrating your interviewers is a key part of building a consistent and efficient interview process. But calibrating your interviewer training process as your organization scales is equally as important.
To do that, you’ll need to continually measure the success of your training, adjust your process, and gather feedback on how it’s working.
Step four: Key goals
Calibrate interviewers. Maintain your quality bar, track interviewer load, and maintain a list of lead interviewers.
Track key success metrics. Review the success of your training by identifying your key hiring metrics, and giving key data back to interviewers.
Define ownership. Establish a cross-functional team and meeting cadence to review data, track progress, and take group ownership of your process.
Calibrate your interviewers and training process
Knowledge is like a muscle — if you’re not using it regularly, then it starts to wither. It’s exactly the same principle with your interviewer training process. When interviewers aren’t calibrated regularly, they could become misaligned with your hiring philosophy, leading to low-rigor interviews and inconsistent decision-making.
Your first step to keep things calibrated is to build a list of all certified interviewers, adding to it as new ones exit your training process. Then, you’ll need to set up a regular cadence for reverse-shadowing.
Reverse-shadowing your certified interviewers is essential for their ongoing learning, and it’s an effective way to keep them in step with the skills and knowledge they need to maintain your hiring bar and candidate experience.
This process can be managed manually using a calendar and spreadsheet. You can also automate this process using an interview analytics platform. Trainees are scheduled automatically based on a set cadence, and can stay calibrated asynchronously by reverse-shadowing existing interview recordings.
Sometimes, your calibration process and interview data may reveal a few blind spots in an individual interviewer’s performance that require more personalized coaching and support. Scott looks for a few red flags when he has a hunch that things aren’t quite on track.
“Interviewing is a skill you’ll continue to develop forever — so we think of calibrating our interviewers as part of their ongoing training. But sometimes, interviewers need a little extra support to keep them aligned with what our organization is looking for. That has to be a data-driven decision — and there are a few data points that give us that signal.
We look for interviewers that consistently operate against the general rating trends, or interviewers that tend to skew towards extreme positives or negatives. We also keep an eye on the overall distribution of yes/no decisions — so if an interviewer suddenly starts to skew down to making a positive decision 30% of the time, that gives us information that the interviewer needs recalibrating.
When that happens, we use that data to work with the interviewer directly to identify the problem and help them get recalibrated on what we’re looking for as an organization with extra training or coaching.”
Scott Nelson
Head of Software Recruiting, Anduril
Define and track key success metrics
Keeping your interviewers calibrated is one part of ensuring the long-term success of your interviewer training process. The other is keeping your process itself calibrated to your organization’s evolving hiring needs.
This involves defining your metrics for success, and evaluating on a regular basis if your training is still fulfilling your needs for hiring quality, candidate experience, individual skills development, and at an operational level.
Tracking this data long-term means you can adjust training over time, optimize for efficiency and consistency, and ultimately, ensure better outcomes — for both your trainees, your candidate experience, and your quality of hire.
There are five key types of data you can use to inform your decision-making:
- Overall success metrics: Identify the key metrics that indicate successful training at an organizational level, including time-to-hire, offer acceptance rate, scorecard consistency, and the speed candidates progress through your hiring pipeline.
- Candidate experience: Keep an eye on data on candidate NPS and survey feedback — this can give you quantitative and qualitative clues as to the consistency of your interview process.
- Operational data: Monitor key operational metrics of your process, such as the amount of certified interviewers you have, time-to-train, and interviewer load. This will help you identify moments when your system is becoming stressed.
- Interviewer KPIs: At an individual level, interviewers need regular, actionable data that helps them pinpoint elements of their own performance that require improvement. Keep track of offer acceptance rates, candidate NPS, scorecard calibration and the quality of written feedback for each interviewer, and feed this back to them on a regular basis to maintain performance. If you’re using an interview analytics tool, you can also establish more granular benchmarks for interview performance, including amount of questions asked, candidate share of voice, and question stacking. Be mindful that this data should be framed as developmental feedback, rather than critical feedback.
- Process feedback: Gather feedback from hiring managers, bar raisers, or hiring committee members on their confidence levels when making decisions based on the feedback from certified interviewers. Check in on interviewers for feedback on the process and their interview load.
Gather feedback and define ownership
Historically, ownership of the interviewer training process has fallen firmly to HR, talent, and recruiting teams. That makes logical sense when we consider interviews as solely a talent acquisition process.
But interviewing well impacts everyone at your organization. This is why establishing mutual, cross-functional ownership of your training process is crucial to its long-term success. This group should include your People, talent, and recruiting teams as standard, as well as senior stakeholders, hiring managers, and any other members of your hiring committee.
Scott recommends creating a cross-functional team or committee that meets on a regular basis to discuss progress of training, feedback from candidates and interviewers, and a review of current interviewing trends at your organization.
“We are constantly looking at our process and feedback from both sides to identify new training needs and blind spots in our process.
We pull our interviewing and hiring data biweekly, gather as many candidate anecdotes as we can, and review interview transcripts and recordings. We take a look at batting averages, how new hires are doing at the company, if there have been any hires that haven’t worked out. We take in the level of feedback we’re getting from both candidates and interviewers.
This gives us a clear overview of emerging trends so we can recalibrate and coach interviewers quickly, and iterate on our process.”