Strategic Staffing, 2e (Phillips/Gully) Chapter Measurement 1) Criterion data is information about measures used to make projections about outcomes Answer: FALSE Diff: Page Ref: 200 Skill: Concept Objective: 2) Class ranking and finish place in a race are examples of ratio measures Answer: FALSE Diff: Page Ref: 200 Skill: Concept Objective: 3) A task knowledge test in which a candidate is expected to score at least 75 out of 100 to advance to the next stage in the selection process is an example of a criterion-referenced measure Answer: TRUE Diff: Page Ref: 200 Skill: Concept Objective: 4) A normal curve typically is typically bimodal Answer: FALSE Diff: Page Ref: 202 Skill: Concept Objective: 5) Scores on norm-referenced measures have meaning only when compared to the scores of others Answer: TRUE Diff: Page Ref: 201 Skill: Concept Objective: 6) The median describes the arithmetic average of the scores in a distribution Answer: FALSE Diff: Page Ref: 202 Skill: Concept Objective: Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 7) A personality test that correlates -0.5 with job performance should not be used to make staffing decisions Answer: TRUE Diff: Page Ref: 205 Skill: Concept Objective: 8) Administering a computer test of programming skills in the same way to every job candidate is an example of reliability Answer: FALSE Diff: Page Ref: 216 Skill: Concept Objective: 9) Assessing whether two interviewers rate the same candidate's interview the same is an example of test-retest reliability Answer: FALSE Diff: Page Ref: 214 Skill: Concept Objective: 10) If all the candidates who score high on a skills test go on to have high job performance, the skills test has high criterion-related validity Answer: TRUE Diff: Page Ref: 217 Skill: Concept Objective: 11) Correlation analysis can only assess the relationship between two variables Answer: TRUE Diff: Page Ref: 205 Skill: Concept Objective: 12) The median is the score that is in the 50th percentile Answer: TRUE Diff: Page Ref: 202 Skill: Concept Objective: 13) Standard scores are raw scores that have been converted into an expression of the percentage of people who score at or below that score Answer: FALSE Diff: Page Ref: 202 Skill: Concept Objective: 2 Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 14) Construct validity reflects the perceived job relatedness of a measure Answer: FALSE Diff: Page Ref: 217 Skill: Concept Objective: 15) False positives occur when a weak applicant is erroneously classified as being a good hire Answer: TRUE Diff: Page Ref: 221 Skill: Concept Objective: 16) Predictive data is information used to A) measure important outcomes of the staffing process B) categorize people into appropriate labels C) make projections about desired outcomes D) assess central tendency Answer: C Diff: Page Ref: 200 Skill: Concept Objective: 17) Information about important outcomes of the staffing process is called A) predictive data B) criterion data C) reliability D) validity Answer: B Diff: Page Ref: 200 Skill: Concept Objective: 18) Years of experience, salary in dollars, and weight are examples of what type of measure? A) nominal B) ordinal C) interval D) ratio Answer: D Diff: Page Ref: 200 Skill: Concept Objective: Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 19) Edwin is the HR manager at a customer care unit with close to 1,000 employees He wants to statistically analyze the service data to make the recruitment process more effective by identifying desirable and undesirable qualities of employees Edwin decides to use ratio measurement to measure the customer handling time for each employee Which of the following, if true, would specifically support the decision to gauge turnaround time using ratio measurement? A) Edwin plans to analyze the data using techniques such as correlation and statistical modeling B) Edwin wants to measure call data accurately by using an absolute scale C) Edwin is trying to obtain a ranking of the employees rather than finding the extent of difference in productivity D) Edwin wants to reduce the complexity of the observations drawn from the analysis Answer: B Diff: Page Ref: 200-201 Skill: Critical thinking Objective: AACSB: Reflective thinking skills 20) If job candidates must be able to lift 40 pounds to be allowed to advance to the next stage of the hiring process, lifting ability is what type of measure? A) criterion-referenced B) norm-referenced C) standardized D) subjective Answer: A Diff: Page Ref: 201 Skill: Application Objective: AACSB: Analytic skills 21) If the top scoring thirty percent of job applicants on a job knowledge test advance to the next stage of the hiring process regardless of their job knowledge test score, the job knowledge test is what type of measure? A) criterion-referenced B) norm-referenced C) standardized D) subjective Answer: A Diff: Page Ref: 201 Skill: Application Objective: AACSB: Analytic skills Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 22) When are the mean, median, and mode like to be at the same point in a distribution of scores? A) when the sampling error is high B) when the distribution is positively skewed C) when the distribution is negatively skewed D) when the distribution is a normal curve Answer: D Diff: Page Ref: 202 Skill: Concept Objective: 23) What would the median of the following distribution be 2, 3, 3, 3, 3, 4, 5, 6, 7, 8, 8? A) B) C) D) Answer: B Diff: Page Ref: 202 Skill: Application Objective: AACSB: Analytic skills 24) What would the mode of the following distribution be 2, 3, 3, 3, 3, 4, 5, 6, 7, 8, 8? A) B) C) D) Answer: A Diff: Page Ref: 202 Skill: Application Objective: AACSB: Analytic skills 25) Which of the following refers to the spread of the data around the midpoint? A) variability B) validity C) coefficient D) regression Answer: A Diff: Page Ref: 202 Skill: Concept Objective: Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 26) What would the range of the following distribution be 2, 3, 3, 3, 3, 4, 5, 6, 7, 8, 8? A) B) C) D) Answer: D Diff: Page Ref: 202 Skill: Application Objective: AACSB: Analytic skills 27) Which of the following values is closest to the standard deviation of the following distribution 2, 3, 3, 3, 3, 4, 5, 6, 7, 8, 8? A) B) C) D) Answer: B Diff: Page Ref: 202 Skill: Application Objective: AACSB: Analytic skills 28) Which of the following is true of a distribution with positive skew? A) The scores will mostly be in the negative end of the distribution B) The mode falls between the mean and the median C) The mode would be shifted to the right of the distribution D) The mean would be to the left of the distribution Answer: D Diff: Page Ref: 202 Skill: Concept Objective: 29) If a test score is at the 90th percentile, then which of the following is true? A) Only 10% of the test takers scored higher than this B) This is in the bottom 10% of the distribution C) About 90% of the test takers scored higher than this D) This is the most commonly observed score in the test Answer: A Diff: Page Ref: 202 Skill: Application Objective: AACSB: Analytic skills Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 30) A indicates how many standard deviations above or below the mean a given score lies A) norm-referenced measure B) raw score C) criterion-referenced measure D) standard score Answer: D Diff: Page Ref: 203 Skill: Concept Objective: 31) Which of the following is true of a z score of -3.2 in a distribution? A) The score lies above the mean of the distribution B) Performance is lower than the group average C) The score is above the 50th percentile D) Several people scored below this score Answer: B Diff: Page Ref: 203 Skill: Application Objective: AACSB: Analytic skills 32) The correlation coefficient between a selection test and job performance is This means the selection test A) perfectly predicts job performance B) and job performance are unrelated C) has adverse impact on job performance D) is positively related to job performance Answer: B Diff: Page Ref: 205 Skill: Concept Objective: Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 33) Edwin is the HR manager at a customer care unit with close to 1,000 employees He wants to statistically analyze the service data to make the recruitment process more effective by identifying desirable and undesirable qualities of employees Edwin observes a high positive correlation between the employees' ability to adapt and the turnaround time However, he decides to avoid using this criterion when recruiting employees Which of the following, if true, would MOST strengthen this decision to avoid the criterion? A) The statistical significance of the correlation was found to be sixty percent B) Another trait, honesty, had a higher correlation coefficient than employees' ability to adapt C) The sample size used by Edwin was significantly larger than what was required D) Multiple regressions were observed among the variables used for the analysis Answer: A Diff: Page Ref: 208 Skill: Critical thinking Objective: AACSB: Reflective thinking skills 34) If the correlation coefficient for the relationship between a job sample test and job performance is -0.7 and significant then the employer should A) not hire people who pass the test B) hire people who score high on the job sample test C) hire people who score low on the job sample test D) rerun the analyses because a coefficient of -0.7 is impossible Answer: C Diff: Page Ref: 208 Skill: Application Objective: AACSB: Analytic skills 35) When predictor and job performance scores have been obtained, the predictor can be considered valid and useful if the correlation coefficient A) has both practical and statistical significance B) has statistical significance alone C) is insignificant D) has practical significance alone Answer: D Diff: Page Ref: 208 Skill: Concept Objective: Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 36) Which of the following is true about multiple regressions? A) It allows the use of one predictor when predicting an outcome B) It works only on variables that are at the interval or ratio level C) The higher the correlation between variables the greater the regression D) Regression is done in the form of an equation Answer: D Diff: Page Ref: 208 Skill: Concept Objective: 37) The following equation is the result of performing a multiple regression analysis: Job performance = 10 + (5 = job knowledge) + (0.7 = conscientiousness), where job knowledge is measured on a scale of 0-5 and conscientiousness is measured on a scale of to 100 Which of the following conclusions is correct? A) Conscientiousness is less important than job knowledge B) If a person scored on job knowledge and 100 on conscientiousness he or she would have the maximum predictive score possible C) Neither job knowledge nor conscientiousness predicts performance D) If a person scored on both job knowledge and conscientiousness, his or her predictive score is Answer: B Diff: Page Ref: 210 Skill: Synthesis Objective: AACSB: Analytic skills 38) How dependably or consistently a measure assesses a particular characteristic is the measure's A) reliability B) validity C) standardized score D) percentile Answer: A Diff: Page Ref: 210 Skill: Concept Objective: 39) Failure to measure important aspects of the attribute to be measured results in error A) deficiency B) random C) contamination D) standard Answer: A Diff: Page Ref: 212 Skill: Concept Objective: Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 40) When multiple versions of the same test are developed to deter cheating, which of the following reliabilities is used to ensure the two forms are comparable? A) test-retest reliability B) inter-rater reliability C) parallel forms reliability D) internal consistency reliability Answer: C Diff: Page Ref: 213 Skill: Concept Objective: 41) The extent to which items on a given measure assess the same construct is assessed by A) test-retest reliability B) inter-rater reliability C) parallel forms reliability D) internal consistency reliability Answer: D Diff: Page Ref: 214 Skill: Concept Objective: 42) Which of the following is true regarding the standard error of measurement (SEM)? A) The SEM is a useful measure of the accuracy of individual scores B) The higher the SEM, the more accurate the measurements C) With longer periods between administrations, SEMs are likely to go down D) If the SEM is 1, then the observed score is the true score Answer: A Diff: Page Ref: 214-215 Skill: Concept Objective: 43) If the items on an assessment method not seem to be job-related, then, which of the following is true? A) The assessment method's criterion-related validity is low B) The assessment method's construct validity is low C) The assessment method's face validity is low D) The assessment method's content validity is low Answer: C Diff: Page Ref: 217 Skill: Concept Objective: 10 Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 44) Asking all job candidates questions from the same interview schedule in the same way is an example of A) validity B) correlation C) reliability D) standardization Answer: D Diff: Page Ref: 222 Skill: Concept Objective: 45) The is the percent of employees who are defined as currently successful performers A) base rate B) selection ratio C) SEM D) coefficient Answer: A Diff: Page Ref: 223 Skill: Concept Objective: 46) What is the difference between predictive and criterion data and how is each used? Answer: Predictive data is information about measures used to make projections about outcomes For example, data collected to predict turnover or job performance is predictive data Similarly, one could measure the intelligence of job candidates to see if it predicts some component of job success This is also predictive data In terms of the general staffing system, predictive data can come from any part of the hiring process and can include information on sourcing quality, the basic qualifications of applicants, and their traits, competencies, and values Criterion data is information about important outcomes of the staffing process Traditionally, this data includes measures of the job success of employees Criterion data can include job performance, training outcomes, promotability ratings, days missed, safe work days, and any other job outcome of interest At a general level, criterion data can also include all outcome information relevant to the evaluation of the effectiveness of the staffing system against its goals This can include measuring a company's return on investment related to its staffing measures, employee job success, time-to-hire, time-to-productivity, promotion rates, turnover rates, and new hire fit with company values Criterion data is used to measure and assess desired outcomes of interest Predictive data is used to make predictions about those outcomes If statistically and practically significant relationships are found between predictive and criterion data, then the predictors can and should be used to make staffing decisions This type of data-based decision making will provide a competitive advantage to organizations over those that not use data to make staffing decisions Diff: Page Ref: 200 Skill: Concept Objective: 11 Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 47) What are norm- and criterion-referenced measures and when is each appropriate? Answer: For criterion-referenced measures, or standards-based assessments, the scores have meaning in and of themselves This approach is most appropriate when the measure has a clearly definable connection to a particular knowledge, skill, or ability and measurable standards for passing have been established For example, firefighter candidates might be expected to reach a certain criterion measure, such as running up four flights of stairs in full gear in a certain amount of time before they can advance to the next stage of the hiring process Another example is passing a written driver's license with a score of 85% correct or better On criterion-referenced measures it is easy to see what a particular score indicates about proficiency or competence Norm-referenced measures have meaning only when compared to the scores of others For example, a score of 80 on a leadership inventory means nothing by itself The relevance of the score is established in reference to the scores of others If the scores of average score was 70 then clearly 80 is above average whereas if the average score was 90 then clearly 80 is below average The variability of the scores of others also is an important consideration Often it is useful to convert raw scores on norm-referenced measures into standard scores (or percentiles) because this provides the kind of comparative information needed to use the scores Diff: Page Ref: 201 Skill: Concept Objective: 48) Why is standardization crucial for effective measurement? What are the properties of a standardized measure? Answer: Standardization is crucial for effective measurement because it increases accuracy, improves comparability of scores, increases reliability, reduces contamination error, and enhances validity Standardization is the consistent administration and use of a measure It reflects the consistency and uniformity of the conditions as well as the procedures for administering an assessment method Computerization helps to ensure that all respondents receive the same instructions and the same amount of time to complete the assessment Standardization increases the fairness of the assessment system Because maintaining standardized conditions is the responsibility of the people administering the assessment, training all administrators in proper procedures and control of conditions is critical This is true for interviewing as well as any other assessment approach In addition to being legally important, standardization is also valuable because recruiters should consistently evaluate candidates on their competencies, styles, and traits Diff: Page Ref: 214 Skill: Concept Objective: 12 Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 49) What are reliability and validity, and why they matter in staffing? Answer: Reliability refers to how dependably, or consistently, a measure assesses a particular characteristic Validity is the most important issue in selecting a measure It refers to how well a measure assesses a given construct and the degree to which specific conclusions can be made or predictions based on observed scores Validity is the cornerstone of strategic staffing Obviously, if the measures are low in reliability (taking into account the type of reliability being considered) then validity is difficult to establish Both are critical to staffing because effective staffing systems use high quality data to infer relationships and make predictions It is difficult to identify determinants of job success if the data is low in reliability and validity As a result, the measures will not be useful Reliability is also important because it ensures that the scores and data used are representative of people's true scores Validity is also important because it ensures that the tests are measuring what is being measured and that the assessments are useful for predicting job related outcomes Thus, validity establishes the legal defensibility of the staffing system Ultimately, high-quality talent can be predicted using various kinds of tests, then they give a competitive edge over firms that not use valid tests for selection Reliability and validity increase the likelihood that employers will be able to predict high quality talent Diff: Page Ref: 210-215 Skill: Concept Objective: 13 Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall 50) Describe how you would create and validate an assessment system Answer: Begin with a job analysis to understand determinants, requirements, and indicators of job success This helps identify the important knowledge, skills, abilities, and other characteristics (KSAOs) and competencies required of a successful employee Clearly understand what it is that you want to measure and the reasons for measurement Next, identify reliable and valid methods of measuring or assessing important KSAOs or competencies Either collect data to conduct a reliability analysis or refer to published materials to determine if acceptable levels of reliability exist, taking into consideration the type of reliability being evaluated Next, develop a system for collecting the data A variety of methods for measuring important characteristics are available and it is important to establish the integrity and usefulness of the data generated by each measure Examine each measure to ensure that it has an appropriate mean and standard deviation and look for measures that appear to have useful variation Then, collect measures of job success and either correlate or regress them with predictive measures Regression analysis can be used to evaluate any redundancies among the measures and to assess how well the group of measures predicts job success After establishing which measures predict job success best, examine the potential adverse impact and cost of the measures Use the validity of information along with cost and adverse impact to determine the final set of measures Next, develop a method for combining the scores For example, one can probably use the regression equation to weight the scores Additionally, develop selection rules to determine which scores are considered passing Implement the assessment system, ensuring that those assessing candidates or using the system are properly trained Attempt to standardize the testing and assessment conditions Ensure that assessment instruments are kept secure and maintain confidentiality of results Finally, periodically reevaluate the usefulness and effectiveness of the system to ensure that it continues to predict job success without adverse impact or exceptional cost Diff: Page Ref: 222 Skill: Synthesis Objective: AACSB: Analytic skills 14 Copyright © 2012 Pearson Education, Inc Publishing as Prentice Hall ... of test- retest reliability Answer: FALSE Diff: Page Ref: 214 Skill: Concept Objective: 10) If all the candidates who score high on a skills test go on to have high job performance, the skills test. .. personality test that correlates -0.5 with job performance should not be used to make staffing decisions Answer: TRUE Diff: Page Ref: 205 Skill: Concept Objective: 8) Administering a computer test of... of job applicants on a job knowledge test advance to the next stage of the hiring process regardless of their job knowledge test score, the job knowledge test is what type of measure? A) criterion-referenced