Ten Considerations for Selecting the Right Hiring Assessment Provider

With so many different types of assessments on the market (and so many different assessment providers), the choice of selecting pre-employment assessments for your workforce is not a decision to be taken lightly. Hiring assessments can have a far-reaching impact, including the quality of the people you hire, legal implications, and ultimately the quality of your brand delivery.

When it comes to choosing a pre-employment assessment provider, a company needs to start with the end in mind. Essentially, the company ought to start by answering two key questions:  1) “What activities need to be performed for us to deliver value to our customers?” and 2) “What are the knowledge, skills, abilities and other characteristics (KSAOs) required to effectively and efficiently do those activities?” Once those questions are answered – or at the very least asked – a firm can begin to formulate an idea about what types of assessments and process will give the firm the best chance of success. 

This paper provides HR executives with information on what is most important to consider when choosing an assessment provider and presents questions to ask potential vendors to help you make the best possible decision.

1. Targeted Assessments

Some assessment firms utilize profiles of incumbents as a basis for hiring new people for a particular job. In effect, this lets companies hire “more of the same.” That may not seem like such a bad thing, particularly if a company feels that their workforce is already comprised of the best performers in the industry. But it could be that the profiles are targeting aspects of personality that may not have anything to do with relevant job performance. For example, one common profile assessment measures “acceptance of authority” as a characteristic necessary for success in a management role, but the reality is innovation or entrepreneurial spirit might be just as important – or more so – in certain positions or in certain companies. The point here is that assessments must be relevant to the job for which they will be used and linked to specific competencies needed for that job. Even a valid personality profile will serve no real benefit if it is too broad to offer specific help with specific jobs within an organization. Thus, firms would be wise to determine whether the constructs measured in an assessment are unequivocally related to the specific job in question.

2. Technically Sound Assessments

Using only reliable and valid assessments is required for legal defensibility and making good hiring decisions. Reliability relates to the consistency of measurement and validity indicates that an assessment accurately measures what it intends to measure. All hiring assessments should be backed by a comprehensive and sound technical manual that addresses both of these issues among many others. If the assessment provider does not have a technical manual for an assessment they are recommending, one should view the applicability of the assessment in question with caution. But the existence of a technical manual does not in and of itself guarantee quality. Firms should determine whether the population characteristics, sample size, validity coefficients and other aspects of the study are compelling, particular compared with the studies outlined in the technical manuals for other assessments under consideration.

3. Fair and Unbiased Assessments

Assessments should be culturally sensitive and free from bias (e.g., assessments should not include culturally biased language). One might think that is obvious, but assessments have routinely been used in situations in which they should not be used. One such example is the industry’s reliance on reading comprehension components of cognitive ability instead of using assessments that result in less adverse impact. Fairness is a claim that many assessment providers glibly gloss over by stating that they meet EEOC guidelines. But this should be thoroughly questioned by companies who believe in a diverse workforce and want to have a reputation for fairness among job seekers in their communities. While minimum thresholds of fairness may ostensibly be met, companies should question whether they could benefit from having higher standards of inclusion. In every case, companies should carefully consider the potential consequences of using any assessment in the communities and countries in which they operate.

4. Proof

Assessment providers should readily offer to prove the effectiveness of a hiring assessment through either a validation study or subsequent analysis of the impact of their assessments on your business and measured business results. Be upfront and ask potential assessment providers if they are willing to demonstrate an assessment’s effectiveness. A good provider can accomplish this task a number of ways, but they should leap at the opportunity to do so. Avoid providers who seem reluctant to put their money where their mouth is.

5. Clear, Understandable Assessment Results

Assessment results should be straightforward, easy to understand, and easy to base decisions upon (it shouldn’t take a rocket scientist to decipher the meaning of hiring assessment results). Because assessment results must be properly and consistently interpreted in order for an assessment to work (and to avoid legal challenges), subjectivity in interpretation should be minimized where possible. Results should be offered at both the individual and group levels and should be appropriate for the decisions made at each level of the organization. For example, a hiring manager in the field does not need to have a lengthy personality profile report to make a simple “hire/no hire” decision. The entire selection system and sequence of assessments should provide an efficient AND thorough process through which the best of your qualified candidates become your new and valued employees.

6. Contextually Considerate Assessments

Both job requirements and organizational context should be considered when developing hiring assessments. Specifically, there should be a clear and overarching strategy to the use of the assessment(s) that considers what the organization is trying to achieve and how selecting individuals will help the organization achieve its objectives. “Cultural fit” is often bandied about as a prerequisite for being hired, but often the assessment of that fit is determined subjectively by whoever is conducting the job interview. In truth, an organization’s culture – its values, standards, expectations – can be predetermined and measured through customized assessments that can predict the likelihood of a candidate’s fit with respect to the company’s values and other aspects of organizational culture.   

7. Complementary Tool Availability

Organizations should use conceptually aligned assessments that complement each other. This especially holds true for a well-designed hiring system that has multiple components (e.g., applications, interviews, developmental tools). Some assessment providers will offer a single test that is purported to provide everything necessary to hire stellar performers, but the reality is any one assessment is likely to measure only SOME of what is required for the job. By contrast, suppose you have five imperfect measures that each measure something different. In total, you end up measuring more of the candidates’ potential for performance on the job. The key here is for companies not to put all their eggs in one basket.  

8. Adequate Staffing and Resources

Any reputable firm developing hiring assessments will be staffed with competent people with the appropriate degrees. At a minimum, companies should look for assessment firms that have capable staff members with advanced degrees related to measurement and hiring assessment development. There are any number of excellent technology firms that employ an array of talent management components on their platforms, but the technology-focused firms do not necessarily have the industrial and organizational psychology resources to carry out validation studies or to continually develop new and relevant assessments for changing workforces within industries. Companies should be careful to distinguish between technology solutions and assessment solutions and ensure they are getting all they need from the suite of vendors they choose.  

9. Strong Service and Support

Be sure that any assessment provider you are considering partnering with is willing to learn about your organization’s unique needs and issues. A good assessment provider views your potential relationship as a mutually beneficial, two-way street and provides support as a way of ensuring a long-term benefit to you. As an example, be sure the assessment provider is willing to monitor your results for you, providing information such as pass rates for the population as a whole as well as for groups protected under the EEOC, source reports that tell you where your best candidates come from and ROI studies that prove the extent to which your investment in hiring is paying off.

10. Candidate Friendliness

Hiring assessments should create a positive experience for candidates. They should provide clear directions, be as short as possible, and appear to be work-related to candidates. In addition, candidate information should be kept confidential and corresponding data should be secure. Especially for companies whose candidates may also be your customers or whose reputation is in part derived from the experience that your candidates have in the hiring process, it is critical that your hiring assessments reflect your company’s positive and welcoming brand.

Questions to Ask Potential Assessment Providers

Use the list of questions below to help you evaluate a potential pre-employment assessment provider.

Technical Competence

  • Is there a comprehensive assessment manual available that describes how the assessment was developed?
  • Does the manual include sections covering both assessment reliability and validity?
  • Does the manual describe potential outcomes of using the assessment?
  • Are sample assessment reports/results pages available?
  • Are there definitions of specific competencies measured by the assessment(s)?
  • What specific job performance dimensions were targeted in the development of the assessment?
  • What positions can the assessment be used for?
  • How will you ensure the assessment is job-related?
  • Do you have information regarding the validity of the assessment across relevant demographic groups?
  • Do you conduct ongoing adverse impact analyses? Can this evidence be provided?
  • Have you conducted research to test the performance of the assessment across cultures?
  • Are specific standards around cultural sensitivity and inclusion strictly adhered to? If so, how?
  • What are the credentials of your assessment development professionals?

Proof

  • What kinds of results can you offer as evidence of the effectiveness of the assessment?
  • Are you willing to conduct a research or pilot study within our organization?
  • What other research studies have you conducted with other organizations? What did you learn?

Contextual Performance

  • What conceptual framework was used to develop the assessment?
  • Are there benefits to using the assessment beyond improving task-based job performance?
  • Does the assessment get at both the “what” of the job and “how” the job needs to be performed?
  • How do you ensure the assessments and delivery methods remain cutting edge?
  • What other hiring tools are available and how might these other tools add value?
  • What might be missing by using the assessment alone?

Support

  • Do you offer training regarding the interpretation of assessment results?
  • How quickly do you provide results?
  • What kind of support staff do you have (e.g., IT department, service department)?
  • What proactive support do you offer (including but not limited to compliance reports, adverse impact analyses, source reports)?
  • What do you offer in terms of group-level results and corresponding reports? What support is available in interpreting these reports?
  • Will we have a dedicated point-of-contact?
  • Can you help us implement if necessary? What does that look like? What is your experience in deploying these types of solutions in large/medium/small organizations?
  • What training is available at the corporate and field level? How do you support new managers who will be using the system?
  • What is your percentage of up-time for assessment delivery?

Organizational Credibility

  • How long have you been in business?
  • What industries have you worked with?
  • Can you provide references?

Candidate Experience

  • Can I take the assessment?
  • Are the candidate instructions easy to understand and follow?
  • Is the length of the assessment reasonable for your organization and population?
  • What safeguards are in place to protect candidate privacy and confidentiality?

Intangibles

  • Does the provider make you feel special or do you just feel like they are taking another order?
  • Does the assessment appear job-related?

 

Download PDF

Please complete this form to download the PDF file.