Criteria and Work Analysis

Criterion

Criteria is standards used to help make evaluative judgements.

• Conceptual criterion = theoretical standards/construct. E.g. intelligence or job performance
• Actual criterion = operational or empirical standards that researchers measure. E.g., respectively, the Wonderlic Test and Supervisor Ratings and Attendence
• These are important to I/O psychologists
• Requires careful consideration about what to measure so that we can capture as much of the conceptual criterion as possible. We want conceptual and actual to overlap as much as possible, but this is difficult.

Criterion Relevance is the degree to which actual criteria and conceptual criteria coincide.

Criterion Deficiency is the degree to which actual criteria fails to overlap conceptual criteria

• can reduce but not eliminate

Criterion Contamination is the bias (extent actual criteria consistently measures something else) and error (extent to which actual criteria is related to nothing at all).

• e.g., low reliability, rater bias, cheating
• unrelated to conceptual criterion

Work (Job) Performance

Organizations want good performers, so what data can be used to determine whether an employee is a "good performer"?

Some Job Performance criteria:

• production (objective)/performance evaluations (subjective)
• sales
• tenure or turnover
• absenteeism
• accidents
• thefts
• counterproductive work behavior
• emotional labor
• adaptive and citizen behavior

Work (Job) Analysis

During a work analysis, a psychologist investigates

• what activities are performed as part of the job
• what characteristics or qualifications does an employee need to be able to do the job well
• in what settings and under what conditions is the job performed

Two approaches: task-oriented or worker-oriented procedures.

Task-oriented procedures:

• examines work activities
• discrete units of work (what is accomplished)
• incumbents then rate tasks on various dimensions (frequency + importance)
• Functional Job Analysis (FJA) rates work on things, data, and people

Worker-oriented procedures:

• examines human characteristics and qualifications like knowledge, skills, abilities, and others (work style, personalities, values)

To collect information for work analysis:

• questionnaires, inventories
• direct observation
• interviews

Questionnaires are commercially available (e.g., Position Analysis Questionnaire, Common Metric Questionnaire). templates, and can be developed in-house. They focus on: knowledge, skills, experience, qualifications, duties performed daily/less frequently, equipment/materials used, time spent on different job duties, physical and emotional input, level of job satisfaction, salary and compensation, work conditions, and additonal comments

• Pro: structured, good starting point for JA, used for many jobs, standardized basis for comparing jobs, and not time intensive for job analysts
• Cons: can be outdated, question misinterpretation, inaccurate information, not useful for new jobs, may not capture details, non-response rate.

Direct Observation of Work A job analyst observes an employee and records what they do and do not do. Observation is the only job analysis method that allows the job analyst to directly obtain data through witnessing work.

Pros: objective, observer can become familiar with setting, tools, relationship, and tasks

Cons: intrusive, could change behavior or work output (hawthrown effect), observes only a sample of work, can't be used for those jobs where mental effort, concentration or planning are frequent activities.

Interview of Subject Matter Experts: job analysts or HR professionals conduct interviews with relevant personnel to describe the task and duties performed and identify the personal competencies necessary for successful performance.

Interviews are conducted in small group formats for more reliable results.

SME (Subject matter experts) can be job incumbent, job supervisor, or trained work analysts.

Pro: can gather information about how the work is done, can be useful for jobs where most tasks are not observable, good for management cons: incumbent may exaggerate, supervisor may not be knowledgable, may omit information, interviewer inexperience or bias can influence reliability, and its time consuming.

Other work analysis techniques:

• log sheets/diary: obtain log sheets from employees with information about their tasks and the time spent on each tasks for at least one full work week
• Desk audits: observe employees doing their jobs at different times of the day and days of the week and track what they do and for how long
• Job comparison = compare different jobs in the department as well as job grade or job family to show where it falls on the pay scale.

O*NET is the Occupational Information Network, and its a free online database that contains hundreds of occupational definitions.

• developed under the sponsorship of the US department of labor/employment and training administration
• "interests" are based on John L. Holland's vocational model

ONET provides description of experience requirement, occupational requirements, occupation specific requirements, worker requirements, worker characteristics, and labor market characteristics.

Uses of work analysis information:

• provides information needed to develop job description or job posting
• offers rationale for selection tests, interview questions, or other selection techniques
• organizes positions into job families to determine compensation levels, grade level
• provides information as to content of training needed
• provides basis for content of performance appraisal, used for vocational counseling, can help ADA criteria

Evaluating this information. Research finds:

• task-oriented methods have less error than worker-oriented because tasks are more discrete and observable
• knowledgeable raters make a big difference in rating accuracy
• greater accuracy when employees do not perceive the job analysis process attached to compensation decisions
• all methods can be subjective,