Citation: Cronshaw, S. F., Best, R., Zugec, L., Warner, M. A., Hysong, S. J., & Pugh, J. A. (2007).  A five-component validation model for functional job analysis as used in job redesign. Ergometrika, 4, 12-31.

A Five-Component Validation Model for Functional Job Analysis as Used in Job Redesign

 

Five Strategies for Validating FJA Task Banks

We will begin our development of a model for job analysis validation with the definition of validity from the Standards for Educational and Psychological Testing (American Educational Research Association, et al., 1999), modified to address the validation problem for job analysis:  Validity refers to the degree to which evidence, as informed by theory, supports the inferences made from task bank data that are required to justify and support the proposed human resource management (HRM) uses of this information.  This definition of job analysis validity entails certain assumptions that we now summarize.

An FJA task bank cannot be said to be valid for all purposes.  As a result, the purpose to which the job analysis will be put must be explicated before meaningful statements about the “validation” or “validity” of the task bank can be made.

The job analysis must have a theoretical basis if we are to marshal validation evidence for the given purpose.   That is, a theory of work must undergird the job analysis methodology and its application in a human resources (HR) intervention.  This statement may be disputed by some, but we are persuaded by the example of Kurt Lewin who stated that “There is nothing so practical as a good theory”  (1951, p. 169).  Theory lifts job analysis out of the conceptual wasteland of dust bowl empiricism and forges rich linkages with science and practice across the wide domain of management, organizational, and social sciences.  The use of theory gives the organizational client the assurance that the job analysis intervention is not a “stab in the dark”; that the information generated through it has meaning, substance, and permanence.   

Sufficient evidence, of the empirical sort, must be adduced as part of the validation process to support a given interpretation of job analysis information in its eventual HR application.  Another question now
arises: What do we count as empirical evidence? – Numerical/ statistical data, narrative/descriptive material, or both? (We in fact will admit both as evidence for the purpose of job analysis validation.)

It is the inferences made from the information in the task bank that are validated.  It is technically incorrect to refer to the “validity” of a task bank or of job analysis data, either narratively- or ratings-based.

Many applications of FJA will require that three inferences be validated: Accuracy, comprehensiveness, and usability.  By referring to Webster’s Dictionary, we now define these inferences to correspond with the everyday use of these terms.  Job analysis data (both narrative and ratings) are accurate to the extent that they are free from mistakes; that they are correct and precise.  The data are comprehensive to the extent that they are inclusive, they include much of the relevant work domain.  They are usable if they are capable of being used; if they are suitable or fit for use.  These three inferences are given on the left-hand side of Table 1 and then cross-referenced in the table with the five validation strategies that are introduced and discussed throughout the remainder of the introduction to this paper.  As discussed and illustrated in detail by Fine and Cronshaw (1999), FJA can support, and inferences from it can! be validated for, a broad range of HR applications, including recruitment, selection, training, performance appraisal, job design, career development, and compensation.

We now describe five validation strategies used in validating inferences made from FJA task banks in a given HR application. These strategies draw on a synthesis of literatures in the areas of psychological measurement of objects and stimuli (e.g., Brunswik, 1956; Dunn-Rankin, 1983), linguistic philosophy (e.g., Wittgenstein, 1958; Baghramian, 1998), qualitative research methods (Lincoln & Guba, 1985), and conventional testing and assessment (e.g., Nunnally & Bernstein, 1994).  Our brief discussion of each validation strategy begins with a statement of the overall focus of the validation effort and presents a rationale for pursuing that validation strategy.  As well, each of the five validation strategies is referenced back to FJA. 

FJA relies heavily on the establishment and maintenance of a positive interpersonal dynamic over the two-day focus group.  The FJA focus group is highly interactive and data collection by the group is theory-driven; therefore, the FJA facilitator must skillfully bring four fundamental processes into alignment as shown in Figure 1.  Without the proper interaction of these components, it is unlikely that a task bank can be adequately validated for the intended HRM purpose.  As we discuss the five validation strategies, the logic for each strategy is referenced to the six FJA process linkages illustrated in Figure 1 and explained in more detail.

Linguistic validation assesses the question of “Is the job language in the task bank adequately controlled as required by FJA theory and methodology?”.  Careful control over job language is a fundamental FJA principle (Fine & Cronshaw, 1999).  The job language question in turn can be broken down into three aspects: Semantics, syntax, and pragmatics (Baghramian, 1998).  Semantics deals with the meaning and meaningfulness of terms and sentences in the FJA task bank.  Syntax is concerned with rules for linking together terms (words) into correctly constructed task statements.  Pragmatics involves the use of language by “flesh and blood” speakers (Baghramian, 2002) to achieve their goals when developing and using the FJA task bank.   All three aspects – the semantics, syntax, and pragmatics of job descriptive language – are addressed systematically, and in a theory-driven fashion, through a! pplication of the FJA model and methodology.

Two linkages in Figure 1 must be strengthened if the linguistic validation strategy is to provide support for the intended FJA application.  First, there must be correspondence between the conceptual aspect of FJA (its underlying theory) and the written task statements that comprise the FJA task bank.  This logical connection is represented by linkage 1 in Figure 1.  An indispensable part of this linkage is that the FJA task statement is written according to the FJA principles that underlie the controlled use of job language.  These semantic, syntactic, and pragmatic principles have evolved as part of the FJA method and process and are far too extensive to describe in detail here.  Interested readers are referred to Fine and Cronshaw (1999) for a detailed discussion of the controlled use of job language in FJA. 

A second connection in Figure 1 (i.e., linkage 2) must be attended to if FJA task data are to be valid for the intended purpose.  The task statements in the FJA task bank must be written with sufficient precision (i.e., freedom from mistakes in semantics, syntactics, and pragmatics) that they can be rated on the FJA rating scales with a high degree of interrater agreement.  The detailed definitions for the FJA scales that are used for that purpose are given in Fine and Getkate (1995).   Research by Cronshaw, Chung-Yan, and Schat (2006) shows that generally very good interrater agreement on the FJA scales can be expected if task statements are well written and the FJA scales are properly used.  The rating of task statements using the FJA scales (whether or not estimates of interrater agreement are obtained) contributes to linguistic validation because it forces additional discipline on the careful writing of the task statements and a! lso serves as a quality control check on the accuracy of the written task statement in the task bank (i.e., task statements that cannot be reliably rated must be revised and rewritten until good to very good interrater agreement is achieved).

Experiential validation is directed toward answering the question: Does the task bank describe the workers’ experience on the job?  The most well-informed source for experiential validation of an FJA task bank is the worker who performs the job.  In proposing experiential validation for FJA task banks, we must contend with the widespread belief that the self-evident facts deriving from individual experience are inferior to more formalized types of knowledge based on consensual thought systems (Husserl, 1973) including quantitative ratings.  Many job analysts will be convinced of the superiority of predicative self-evidence (e.g., that obtained after application of psychological measurement and research design) over reports based on immediate experience.  However, phenomenologists such as Husserl (1973) give sufficiently persuasive arguments for the substantiality of prepredicative (i.e., direct) experience that we believe it should ! be considered as important validation evidence.

Research evidence can be found to support the rationale for experiential validation.  Rousseau (1982) correlated FJA-based TDP ratings from the Dictionary of Occupational Titles with perceived job characteristics reported by individual workers and found that these perceptions (based on workers’ direct experience with their jobs) were empirically related to occupationally-based indicators of Data and People complexity.  Empirical evidence, as well as logical argumentation, suggests that the experiences of job holders should be directly related to job analysis data and that a poor fit between the job analysis data and the workers’ direct experience on the job is likely to compromise the validity of inferences based on FJA task banks, especially where workers are involved in developing the HR interventions which draw on the task bank.

The linkages in Figure 1 that assist in experiential validation are 3, 4, and 5.  The task bank must summarize and describe the experiences of job incumbents, as reported by the members of the focus group under the facilitation of a job analyst who is well versed in FJA theory and methodology (linkage 4).  As indicated in linkage 5, the FJA task statements, again based on the job experiences of the focus group members, must be written in conformance with the required generic structure of the task statement (as described by Fine and Cronshaw, 1999, p. 75).  Linkage 3 in Figure 1 reflects the need for the facilitator to mentally cross-check the task statements with the FJA scales to ensure accuracy as he or she writes and revises them on behalf of the focus group.  The need for greater linguistic precision that is required by this exercise motivates the facilitator to probe deeper into the work experiences with the focus group members.&nbs! p;
A similar purpose is served by rating the task statements after the focus group completes the task bank.  To the extent that linkages 3, 4, and 5 are attended to during the two-day FJA focus group, group members will be able to accurately home in on and summarize their work experiences in the grammatical form of task statements. 

Experiential validation evidence is accumulated for the FJA task bank by the above means and, at the same time, the inference of task bank accuracy is strengthened through linguistic validation.  Linguistic and experiential validation are in fact closely interrelated.   Lack of linguistic validation will lower the likelihood of experiential validation and vice versa.  This occurs because FJA represents the experience of work through carefully worded linguistic constructions (i.e., task statements).  Consequently, the two types of validation will interact to improve the accuracy of the FJA task information.

Ecological validation is directed at ensuring that the FJA task bank is sufficiently complete to cover the set of tasks performed in the relevant context.  Context can be defined relatively narrowly (i.e., as the position held by one individual or a job performed by two or more individuals) or more broadly (e.g., as the activities that take place in a manufacturing cell within a single manufacturing plant, a larger unit of the organization that spans several locations, or a multinational company across many countries).  In this study, we will consider ecological validation that spans six occupations in the VA primary care function in hospitals across the U.S..  Ecological validation as we have conceptualized it here is in the Brunswickian tradition (Brunswik, 1956) and so fundamentally differs from test validation in that it requires the examination of, and accounting for, the sampling adequacy of situations, rather than of subjects.  The ! sampling of situations (in this case, situations as tasks), rather than people, moves ecological validation outside the domain of the usual nomothetically-based measurement of individual (person) differences that has dominated psychology (Danziger, 1990).  We believe that this shift from a focus on people as the unit of analysis to a focus on situations is entirely consistent with, and in fact necessary to achieve, the proper validation of job analysis.  However, it will require a readjustment of perspective on the part of most personnel psychologists who are accustomed to treating persons (e.g., job analysis raters), and not situations, as their unit of analysis.

Hypothetico-criterial validation assesses the extent to which the information in the task bank is (i) generative of hypotheses regarding the nature and function of work measured at the task level as it is projected through a given HR intervention where (ii) these predictions are supported by empirical research.  To support hypothetico-criterial validation, the job analysis method must be predicated on strong theory (i.e., theory that generates non-trivial propositions that are testable through empirical research).  In this respect, we are fortunate, inasmuch as FJA is one of the few job analysis methods that is thoroughly grounded in a theory of work performance (Schmitt & Chan, 1998). As shown in linkage 6 in Figure 1, hypothetico-criterial validation tests propositions that are drawn from FJA theory mainly through operational definition via the FJA scales, especially Things, Data, and People (TDP) functional skills.  Hypothetico-criterial ! validation has a rich research tradition as shown by Cronshaw (April, 2004) who summarized the results from over 50 empirical studies across the social and organizational sciences that have used TDP ratings of functional complexity as antecedent and/or outcome variables.  Hypothetico-criterial validation parallels, but is not synonymous with, the conventional categories of construct and criterion-related validation used throughout the psychological testing literature.
Social-organizational validation assesses how well the user community reacts to and accepts the information in the FJA task bank in its eventual HR uses.  This FJA validation strategy is a variant of social validation that has been in long-standing use in the field of applied behavior analysis or ABA (social validation was originally introduced into that literature by Wolf, 1978).  In ABA, social validation evaluates the acceptability or viability of a programmed intervention to the eventual user (Schwartz & Baer, 1991).  We believe that social-organizational validation evidence will be strongest when the job analysis data spark interest and even excitement in the user community.  We do not agree with the statements of commentators who believe that job analysis must result in disinterest, or even boredom, on the part of users.  We take it as a positive sign if organizational decision makers become genuinely interested in job analys! is information and committed to its eventual use.  In turn, the extent of user interest in and support for the FJA task bank can and should be assessed through social-organizational validation as the job analysis information is used to structure and the inform HR intervention(s).

The points in this introduction are summarized in diagrammatic form in Table 1, where the three validation inferences of accuracy, comprehensiveness, and usability are cross-referenced to the five FJA validation strategies.  As well, Table 1 explicitly relates the logical linkages between the validation model inputs in Figure 1, the three validation inferences, and the five validation strategies.  To help in further clarifying the somewhat complex arguments made in this preceding introduction to this paper the general questions addressed by the five validation strategies are summarized in Table 2.

FJA Validation in Veteran’s Affairs

FJA was recently used as the primary job analytic methodology in a project sponsored by the U.S. Department of Veterans Affairs (VA).  The Veterans Health Administration, a branch of the Department of Veterans Affairs and the largest vertically integrated health care system in the United States, is transforming its mode of care delivery from a predominantly tertiary care model to one emphasizing primary care as the principal point of access. This organization change has resulted in the need for a fundamental redesign of VA healthcare work processes.  In response to this need, VA project staff developed an FJA task dictionary drawing on work by Moore (1972, 1999) who previously used FJA for task reallocation in designing improved and more cost-effective health services delivery in Papua New Guinea, the People’s Republic of China, Columbia, and the United States.  The VA task dictionary, as described in this paper, covered the six major occ! upations in VA primary care and was developed to guide decisions about task reallocation during anticipated job redesign projects in VA hospital facilities across the U.S. (Best, Hysong, Pugh, Ghosh, & Moore, 2006).  It was believed that this major FJA project would benefit from the validation of the FJA task data across the six primary care occupations and so the five-component validation model proposed in this introduction was used for that purpose.

 

 

Validation Checks and Measurements Built into This Study 
           
FJA validation in the main is accomplished by the many process checks and balances built into the FJA method, which serve to strengthen the linkages 1-5 in Figure 1.  These will be explained in more detail as we present the study results under the respective validation strategies.  In addition to applying these checks and balances that are built into the FJA method and focus group interaction, we collected empirical data (i.e., ratings) from both focus group participants and job incumbents.   These rating metrics are described below under the headings of task clarity and verification ratings as well as ratings on the FJA scales (Fine 1988; Fine & Getkate, 1995) and sociotechnical demand scales (Cronshaw & Alfieri, 2003).

Task clarity ratings.  The focus group participants received back the task bank for their jobs after it was edited by the project team.  They were asked to judge the clarity of each task statement in the task bank for their job on a 100 point scale.  They were asked to give the task the full 100 points if it described what they did on the job “with perfect clarity”, zero (“0”) points if the task was “completely vague” in describing their experience on the job; otherwise, if the task was neither perfectly clear nor perfectly vague in describing their experience on the job they were to give it a point value in the range between 0 and 100 corresponding to the clarity or vagueness of the task statement.  Respondents were asked to try to use the full range of the 0 to 100 scale as needed and to rate in increments of 5.  A mean task clarity rating for each occupation was computed by averaging the ! clarity ratings over all tasks for focus group participants completing the validation questionnaire for that occupation.

Task verification percentage.   When receiving back their task bank for review the focus group participants were asked to indicate whether they performed each of the tasks contained in the task bank (“Yes” or “No”).  The mean task verification percentage was computed by tabulating the number of tasks endorsed “Yes” by each respondent, dividing this total by the number of tasks in the task bank for that occupation, and then taking an average percentage over the respondents for that occupation.    

FJA scale ratings.   VA project staff rated all 243 tasks in the task dictionary on the seven FJA scales as described in more detail below: Three of worker function (Things, Data, and People which are presented in taxonomic form in Figure 2), one of Worker Instructions, and three of General Educational Development (Reasoning, Mathematics, and Language).  For illustrative purposes the first task statement in the VA task dictionary is presented in Table 4, along with its ratings on the FJA and sociotechnical demand scales described below.

Seven FJA scales are used in this paper.  The Things Function scale assesses complexity of worker involvement with tangibles such as office equipment, factory tools, and motor vehicles across four levels as shown in Figure 2 (detailed descriptions of these scale levels, as well as the levels of the other FJA scales, are contained in Fine and Getkate (1995) and Fine and Cronshaw (1999)).   The Data Function scale assesses complexity of worker involvement with information, ideas, facts, and statistics across six levels (see Figure 2).  The People Function scale assesses complexity of worker involvement in interaction with people and animals across eight levels (see Figure 2).  The Worker Instructions scale assesses the relative mix of prescription and discretion in the performance of a given task across eight levels, the lowest level (1) having the greatest amount of prescription relative to discretion (e.g., an off-bearing task in rou! tinized factory work) and the highest level (8) having the greatest amount of discretion relative to prescription (e.g., a research task conducted by a senior university faculty member).  The Reasoning Development scale assesses the required capability to deal with simple vs. difficult reasoning tasks from low to high complexity across six scale levels.  The Mathematics Development scale assesses the required capability to deal with mathematical problems and operations from low to high complexity across five levels.  The Language Development scale assesses the required capability to deal with oral or written communications and materials from simple instructions (level 1) to highly elaborated sources of written information and ideas (level 6).

Ratings of sociotechnical demand.  The theoretical rationale for these two sociotechnical or ST scales is contained in Cronshaw and Alfieri (2003) who present detailed operational definitions of them in the appendix to their article.  The Work Technology scale assesses the intensity of demand of the technical means and methods employed in completing a task across six levels ranging from automated application of technology (Level 1) to pure innovation of new technology (Level 6).  The Work Interaction scale assesses the intensity of the demand on the worker to assist others, coordinate efforts with them, and, if necessary, adapt style and behavior to accommodate others.  The scale metric ranges from a low of solitary work (Level 1) to a high of systems adaptation (level 6).

The Job Analysis Inferences Validated in this Study
 
Three specific inferences are to be validated before the task dictionary is used as a resource in task reallocation across the VA primary care function in U.S. hospitals operated by the VA:

• The tasks in the task dictionary accurately describe the tasks performed by the present job incumbents;

• The dictionary is comprehensive in its coverage of tasks presently performed across VA primary care settings in the U.S.; and

• The dictionary is usable for task reallocation as a part of job redesign in the VA primary care function. 

Results
           
The results of the FJA validation are presented below under each of the five validation strategies.  The results are referenced back to the text, tables, and figures in the introduction of this paper as needed to clarify their interpretation as they apply to the eventual use of the VA task dictionary in task reallocation/ job redesign in the VA primary care function.

Linguistic Validation Evidence for the Accuracy of the Task Statements in the VA Task Dictionary

The basic validation question addressed here is: Is the job language in the task bank adequately controlled as required by FJA theory and methodology?  As shown in Linkages 1 and 2 of Figure 1, linguistic validation requires that FJA theory, methodology, and measurement be drawn together as the facilitator writes accurate task statements on behalf of the focus group.  The procedures followed by the VA project group to strengthen the validity of the job analysis data under linguistic validation were:

• Task statements were written on behalf of the focus group members by certified FJ analysts or with a certified FJ analyst present.

• Three or more individuals well-trained in the FJA methodology edited each of the task statements with strict adherence to FJA semantics, syntax, and pragmatics.  The editors discussed the task statement wording and structure until they reached consensus.  One or more of these individuals also was present during the generation of the task statements by the focus groups.

• Throughout the two-day focus group, the facilitator carefully adhered to the FJA guidelines for the wording and structure of FJA task statements.

• The focus group participants received the task bank for revalidation after it was edited by VA project personnel.  Wording changes recommended by the focus group participants were discussed by the project personnel and incorporated into the task bank as appropriate.

The focus group participants for each primary care job were asked to rate the clarity of the task statements as an empirical means of linguistic validation. The results of these analyses are reported in Table 3.  The clarity ratings over the six VA primary care jobs were quite high, with the exception of the Health Technician occupation.   The Health Technician occupation contains a somewhat more heterogeneous set of positions than the other five occupations, accounting for the somewhat low clarity rating in Table 3.

Experiential Validation Evidence for the Accuracy of the VHA Task Dictionary

The basic validation question addressed is: Does the task bank describe the workers’ experience on the job?  If the job language is properly controlled (see linguistic validation above) and an open and frank discussion occurs during the two-day focus group,  then the task statements in the VA dictionary should be accurate in describing the workers’ experience on the job.  The procedures followed by the VA project personnel to strengthen linkages 3, 4, and 5 in Figure 1 as part of experiential validation were:

• All focus group facilitators were trained and experienced in FJA theory and methodology as set out in Fine and Cronshaw (1999, Appendix C).

• The VA project personnel edited the wording and grammar in the task statements contained in the individual occupational task banks to best describe the incumbents’ experience on the job.

• The individual occupational task banks were sent back to the original focus group participants for revalidation.  During revalidation, the participant checked off whether he/she did or did not perform the task and signed off to certify that the task bank covered at least 95% of the work that they performed.   The participants also were asked to rate the clarity of the task at this time.

• The VA project personnel rated the individual occupational task banks on the FJA scales and, where they were unable to each consensus on the assignment of a scale rating for a given task statement, the task statement was taken back to a job incumbent (not in the original focus group) for rewording to achieve better accuracy.

As an empirical means of experiential validation the focus group participants for each primary care job were asked to verify whether they performed the tasks in the revalidation task bank for their occupation. The results of these analyses are reported in the far right column of Table 3.  The task verification percentages over the six VA primary care jobs were high, again with the exception of the Health Technician occupation, the greater heterogeneity of the Health Technician occupation probably accounting for its lower endorsement rate compared to that of the other five primary care occupations.

Ecological Validation Evidence for the Comprehensiveness of the VHA Task Dictionary
           
Ecological validation asks the question:  Do the tasks contained in the task bank describe the work performed by job incumbents across the relevant context(s) (e.g., within job; across department, function, or organization) to which the job analysis results and application(s) are meant to generalize?  Ecological validation is important because
the VA is a very large, nationwide  system of loosely integrated health care facilities and geographically dispersed outpatient clinics. To be used in job design across this system, the task statements had to be proven to reflect the range of the primary care job demands, across geographically and operationally different locations.  Although it is reasonable to assume that the work content of primary care is relatively consistent across locations (i.e., because clinical guidelines largely prescribe VA medical practice), we cannot assume that who does the work of primary care is consistent across VA facilities nation wide (see Best, et al., 2003).

The procedures followed in ecological validation were:

• An ecological sampling frame was identified across the hospitals in the VA system.  We consulted an expert advisory panel of VHA primary care leadership to identify organizational characteristics that potentially influence how primary health care gets delivered (i.e., who does the work).   Our main objective was to maximize the sampling adequacy of organizational situations or contexts as required in ecological validation.   We selected facilities for inclusion in the study on the basis of three key characteristics identified by our advisory council that might cause systematic differences in the task data collected in the VA.  These three characteristics were: (1) Size of hospital because larger facilities support additional staff, including sub-specialists, whereas smaller facilities rely on fewer providers to perform healthcare work in general: (2) Presence of a medical school teaching facility because the mix of staff chan! ges when medical residents are practicing in a hospital; and (3) Differences in management structure and communication flow due to differences in degree of service line implementation and exposure to advanced clinic access initiatives (e.g., to reduce patient wait times) across hospitals (see Yano, Simon, Canelo, Mittman, and Rubenstein, 2000) .

• A “roll-out” process was developed and implemented whereby the task bank for a  specific primary care occupation, after the initial task bank was developed in a given hospital, was reviewed and augmented by the next occupationally-specific focus group in the ecological sampling plan.    

• Task statements resulting from the roll-out process that overlapped in content across multiple job titles were removed from the final task dictionary.

• The remaining task statements, totaling 243, were merged into a comprehensive primary care task dictionary.

Hypothetico-criterial Validation Evidence for the Usability of the VHA Task Dictionary

The question addressed under this validation strategy is: Do the task bank data empirically relate to other variables, processes, and dynamics as predicted by theory and previous research findings?  Two theories are of central importance here.  The first is the FJA theory that serves as the framework for the development and use of the task dictionary. The second theory is sociotechnical systems theory that helps inform the task reallocation comprising the core methodology in the job redesign proposed by the VA (Nadin, Waterson, & Parker, 2001; Older, Waterson, & Clegg, 1997).  To the extent that FJA and sociotechnical theory are co-extensive and can be integrated for job redesign purposes, the usability of the task dictionary in task reallocation will be strengthened considerably on both theoretical and practical levels.       

FJA theory uses the task as the basic unit of work to understand how worker skills can be best brought into alignment with the content of the work being done and the context of the work-doing system that surrounds and envelopes the worker.   Sociotechnical theory takes a task-based approach to conceptualization and analysis, and revolves around the proposition that “organizational objectives are best met…by the joint optimization of the technical and social system” (Cherns, 1978, p. 63).  In both the FJA and sociotechnical systems theories, workers as whole persons must be integrated into the technical and organizational arrangements of the workplace. Strong support for the usability of the FJA task dictionary in job redesign would be obtained if we could show that cross-over propositions can be generated from an integration of the FJA and sociotechnical theories.  Recent research has provided support for the idea that an! integration of the FJA and sociotechnical theories can produce scientifically-testable hypotheses.  Cronshaw and Alfieri (2003) found that sociotechnical demands (conceptualized from sociotechnical systems theory) impact on the FJA-based measures of the mental and interpersonal skills needed to do the work (i.e., on the content of the work done) and that these impacts are partially mediated through the exercise of worker discretion (i.e., an aspect of the context in which the work is done).   This was the first research to demonstrate empirical support for theoretical propositions coming out of an integration of the FJA and sociotechnical systems theories.

The intended use of the FJA task dictionary is for task reallocation within the VA primary care function, an activity that has an extensive history in both FJA applications (e.g., Fine, 1988) and sociotechnical interventions (e.g., Nadin, Waterson, & Parker, 2001).  The usability of the FJA task dictionary for this purpose would be strengthened (and evidence for hypothetical-criterial validation would be provided) if the predictions of Cronshaw and Alfieri (2003) were replicated from the data in the dictionary.  We used LISREL to test the path model proposed by those authors as a means to relate sociotechnical demands to worker skills in the VA task dictionary.  The result of our replication of their analysis using ratings from the full 243 tasks in the VA task dictionary is presented in Figure 3. The intercorrelation matrix of the variables used in the analysis is presented in Table 5.  Addition of a path between decisional discretio! n and People functional skill (the only path different from the Cronshaw and Alfieri results) produces a very good fit of the path model to the observed data (Adjusted Goodness of Fit Index = .90).   The additional path appearing when the VA data are used may be due to SOP in the form of clinical practice guidelines that have a pervasive influence on medical practice in the VA.  As a result, additional worker use of decisional discretion may be required to reconcile the clinical practice guidelines with the exercise of People functional skill in VA primary care occupations so resulting in the added path in Figure 2.  This would not be the case in the Cronshaw and Alfieri (2003) study that studied tasks from a broad range of occupations across the general economy.

The results of the path analysis show that social demand would serve as an excellent criterion for task reallocation in job redesign for VA primary care.  One concern with this strategy (i.e., the reallocation of tasks for given occupations into a single level of social demand as a means of rationalizing primary care duties across VA occupations and sites) is the possible loss of interest and challenge in the work for those workers performing tasks at the lower two levels of social demand.  To assess this possibility, cross-tabulations were done between levels of social demand and levels of General Educational Development for the 243 tasks in the primary care database (see Table 6).   The results in this table show considerable spread among instructional levels at each level of social demand, indicating that variety, interest, and challenge will still be available to workers if social demand is used as a fundamental criterion for task rea! llocation.

The complexity of People functional skill has been found to increase exponentially as a function of social demand (Cronshaw & Alfieri, 2005).  The same result was found here when People functional skill was regressed against social demand for the 243 tasks in the VA task dictionary (R2 for linear regression equation = .32; R2 for non-linear, i.e., exponential, regression equation = .38; difference in R2’s significant at p < .001).  This replication of the Cronshaw and Alfieri findings provides additional support for the use of social demand as a criterion in task reallocation so that the interpersonal capabilities of workers are not outstripped for given primary care occupations by an exponentially increasing requirement of People functional skill complexity in response to increases in social demands of tasks.

Social-Organizational Validation Evidence for the Usability of the VA Task Dictionary
           
No results on social-validation validation are presented in this study.  This validation evidence must be collected as the FJA task dictionary is used in job redesign.  The VA is particularly interested in using a participatory approach to task reallocation and job redesign.  This interest converges with efforts in other organizations to involve the workers performing the work (and who know it best) as an important partner in job redesign, including sociotechnically-based interventions (Klein, 1991; Nadin, Waterson, & Parker, 2001).  The validation evidence presented in this paper strongly suggests that users, especially the workers participating in the VA job redesign, should find the FJA task dictionary to be highly usable.  However, this prediction should be corroborated through social/ organizational validation as job redesign via task reallocation is carried out in local VA hospitals.  These validation efforts could become a! part of the formative evaluation that is recommended by Cronshaw and Fine (2003) as a means of sharpening and improving the task reallocation/ job redesign process.

Discussion
           
In the introduction to this paper we defined validity of FJA data as referring to the degree to which job analysis evidence, as informed by theory, support the interpretations of those data required for given HRM uses.  In this case, are the FJA task dictionary and its component 243 task statements sufficiently accurate, comprehensive, and usable for the intended use for task reallocation in the VA?  Based on our validation results, we conclude that the VA task dictionary is valid for the intended job redesign purpose.

The relevance of the validation study is better appreciated if the reader has a better understanding of the approach and purpose of the proposed VA job redesign.  The VA approach to the job redesign relies heavily on task reallocation, a common and well-established practice in the job redesign field (Older, Waterson, & Clegg, 1997).  Older et al. (1997) review the literature on task reallocation methods and present a list of 20 requirements for task reallocation (e.g., first, the method must be usable early in the design process).  The task reallocation process proposed in the VA, properly applied, can meet most of these 20 requirements but only if the underlying FJA task data are valid (i.e., accurate, comprehensive, and usable).  If the FJA data are not valid, we can expect major difficulties during task reallocation in the VA primary care function as well as in meeting the more general requirements for task reallocation as set out b! y Older et al. (1997).  

Having a validated FJA task bank provides additional assurance that the job redesign proposed by the VA will yield the intended results.  This point is illustrated vividly in a report on the analysis of the VA task dictionary in preparation for further task reallocation efforts in the VA.  Best, Hysong, Pugh, Ghosh, & Moore (2006) found that physicians, nurse practitioners/ physicians assistants, registered nurses, and licensed vocational nurses perform 60% and 97% of the same tasks.  This finding, along with a reported underutilization of VA clerks and health technicians, points to a “tremendous opportunity” to rationalize VA primary health care while substantially reducing costs to the VA system. Of course the validity of these inferences depends crucially on the accuracy and comprehensiveness of the task data in the primary care task dictionary.  If, for example, very similar tasks were represented separately in the task! dictionary – which could happen if only minor wording differences between them were mistaken for substantive task differences because of a failure in linguistic validation – the entire basis for the proposed task reallocation/ job redesign could come into doubt.   Validation of the job analysis procedure thereby provides an extra “margin of safety” that the organization should have built into its extensive and expensive job redesign interventions.  

The question of job redesign purpose is central to the proposed VA intervention based on FJA.  Seven purposes to be achieved in the VA task reallocation process are summarized in Table 7.  As pointed out by Older et al. (1997), the trade-offs between these decision criteria must be explicitly considered during the job redesign.  They must be weighted and balanced in a task reallocation solution that is appropriate to the local context and management needs in individual VA facilities.  In turn, the availability of a valid task dictionary (i.e., one that is accurate, comprehensive, and usable) is crucially important to making the right decision trade-offs during task reallocation.

The complexity of trade-offs during FJA task reallocation is best illustrated by emphasizing the system perspective taken by Functional Job Analysis.  Figure 4 shows the dynamics of the work-doing system that must be addressed and brought into balance during task reallocation.  For example, HR policies and procedures will have to be revised to keep the needs of the work organization in alignment with the efforts of the workers in the

redesigned primary care function.  In turn, an accurate and comprehensive task dictionary over the primary care function – and one that is validated as such – will assist in the exact wording of new HR policy and procedure (because the data are linguistically validated) and help ensure that rewritten policies and procedures documentation applies across all VA facilities (because the data are ecologically validated).

The larger corpus of job analysis literature has given some attention to problem of validating job analysis data.  Most of these researchers have focused their attention on validating job analysis ratings, rather than job analysis inferences investigated in this study.  Tasks, jobs, and occupations can be rated on a number of scale metrics, including importance, criticality-of-error, criticality, and/or time spent (Harvey, 1991) and, more recently, improvability of career-relevant skills (Maurer, Wrenn, Pierce, Tross, & Collins, 2003).   Sometimes, the rating data from individual scales are aggregated into overall dimension scores on the basis of factor analysis (e.g., Harvey, Friedman, Hakel, & Cornelius, 1988; McCormick, Jeanneret, & Mecham, 1974).  This widespread practice of collecting and analyzing scale ratings for validation of job analysis results would fall under our category of hypothetical-criterial validation.! There is much less attention given in the literature to the validation of narrative data from job analysis.  In this study, we have paid at least an equal amount of attention to narrative data in the validation process as compared to job analysis ratings per se (we devoted three validation strategies to the evaluation of narrative data from FJA, namely linguistic, experiential, and ecological validation).  In this, we try to take a more balanced approach to the validation of job analysis inferences, not by ignoring job analysis ratings, but by giving at least equal weighting to narratively-based data.  We believe this to be the right approach to take in validating inferences from job analyses of the qualitative type and recommend that other personnel psychologists using qualitatively-based job analysis similarly validate the narrative data generated from their job analysis methods.

The more comprehensive model of job validation presented here helps to resolve a recent controversy in the job analysis literature. Two groups of job analysis researchers squared off in the 1990 Journal of Organizational Behavior.  Sanchez and Levine (2000) argued that traditional measures of job analysis accuracy (via rating metrics) is of limited value and should be replaced with a consideration of the ‘consequential validity’ of the job analysis data.  Harvey and Wilson (2000) took the opposing view by arguing that rating accuracy (again via ratings) is a meaningful metric and that this comprises all the validation evidence needed for job analysis per se.  In our validation model, the consequential validity of Sanchez and Levine is subsumed under the inference of job analysis usefulness and would be addressed by both hypothetical-criterial and social-organizational validation evidence.  The accuracy criterion proposed by Har! vey and Wilson is of course subsumed under the inference of job analysis accuracy and is addressed by linguistic and experiential validation.   By giving full consideration to both ratings and narrative data from the job analysis, the validation question is expanded and enriched enough to show that both consequential validity (usability) and accuracy should be addressed, at least as far as FJA is concerned.  These two types of validation evidence are not mutually exclusive as might be concluded from the Sanchez-Levine vs. Harvey-Wilson debate; in fact, both sources are important and they depend on each other in supporting the use of FJA in VA job redesign.      

Conclusion

In a recent paper Prien, Prien, & Gamble (2004) ask the question:  “Is validity an issue in job analysis? – and, if so, in which way?”.  These are fair questions to ask, given the somewhat contentious history of validation as it has been applied to job analysis.  We answer their first question with a resounding “yes” – Validation of job analysis is (or at least should be) an issue of central importance to personnel psychologists and HR managers.  We answer their second question by picking up on the crucial point made by Morgeson and Campion (2000) who point out that inferences inevitably are and must be made by users of job analysis data and that these inferences should be validated, not job analysis ratings per se.  Personnel psychologists would not proceed in personnel selection without validating their selection instruments – as professionals we should able to give the same assurances ! about our job analysis procedures.   The five-component validation model presented here for Functional Job Analysis is meant to help move personnel psychology toward a comprehensive, practical, and theoretically-defensible means of validating qualitatively-based job analysis procedures.   We believe that at least some of the FJA validation model presented here can transfer to the validation of other job analysis methods and so become a useful part of those approaches.  Further work in pursuit of that objective would make a valuable contribution to the literatures of task, job, and occupational analysis.

Author Note

The research reported here was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (IIR #01-185 & HFP #98-002/REAP #05-129). The five junior authors’ salaries were supported in part by the Department of Veterans Affairs during the conduct of the research reported here. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the present employers of the five junior authors.  An earlier version of this paper was presented at the 2006 Annual Meeting of the Academy of Management in Atlanta, GA.
Correspondence and requests for reprints can be addressed to Steven F. Cronshaw, School of Business, College of Science and Management, University of Northern British Columbia, Prince George, BC, Canada V2N 4Z9.

References

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999).  Standards for educational and psychological testing.  Washington, DC: American Educational Research Association.

Baghramian, M. (1998) (ed.). Modern philosophy of language.   London: J.M. Dent.

Best, R. G., Hysong, S. J., Pugh, J. A., Ghosh, S., & Moore, F.I. (2006). Task overlap among primary care team members: Opportunity for system design?  Paper submitted for publication.

Brunswik, E. (1956). Perception and the representative design of psychological experiments.  Berkeley: University of California Press.

Cherns, A. (1978). The principles of sociotechnical design pp. 61-71). In W. Pasmore & J. Sherwood (Eds), Sociotechnical systems: A source book. La Jolla, CA: University Associates.

Cronshaw, S. F. (April, 2004). Use of TDP Concepts in Theory Development, Research, and Practice.   Paper Presented in E.A. Fleishman (Chair), Symposium:  “Things, Data, and People: Fifty Years of a Seminal Theory.”  19th Annual Conference of the Society for Industrial and Organizational Psychology in Chicago, Illinois.

Cronshaw, S. F. (1998). Job analysis: The changing nature of work (In Special Issue, Industrial-Organizational Psychology and emerging needs of the Canadian workplace: Traversing the next millennium). Canadian Psychology, 39, 5-13.

Cronshaw, S. F., & Alfieri, A. J. (2003). The impact of sociotechnical task demands on use of worker discretion and functional skill. Human Relations, 56, 1107-1130.

Cronshaw, S. F., Chung-Yan, G. A., & Schat, A. C. H. (2006). Comparative reliabilities of Functional Job Analysis scale ratings obtained through independent rating and consensus discussion. Unpublished manuscript.

Cronshaw, S. F., & Fine, S. A. (2003). The evaluation of job redesign processes. In J.E. Edwards, J.C. Scott, & N.S. Raju (Chapter 15, pp. 301-321), The human resources program-evaluation handbook. Thousand Oaks, CA: Sage.  

Cummings, L. L. & Worley, C. (1993). Organization Development and Change. St. Paul, Minn.: West.

Danziger, K. (1990). Constructing the subject: Historical origins of psychological research. New York: Cambridge University Press.

Dunn-Rankin, P. (1983). Scaling methods. Hillsdale, NJ: Erlbaum. 

Fine, S. A. (1988). Human service workers (Chapter 10.2, pp. 1163-1180). In S. Gael (ed.), The job analysis handbook for business, industry, and government, Vol. 2. New York: Wiley. 

Fine, S. A., & Cronshaw, S.F. (1999). Functional job analysis; A foundation for human resources management. Mahwah, NJ: Erlbaum.

Fine, S. A. & Getkate, M. (1995). Benchmark tasks for job analysis: A guide for functional job analysis (FJA) scales. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358.

Gael, S. (ed.) (1988). The job analysis handbook for business, industry, and government. New York: John Wiley.

Harvey, R. J. (1991). Job analysis (Chapter 2),  In M.D. Dunnette & L.M. Hough (eds.), Handbook of industrial and organizational psychology (Volume 2).  Palo Alto, CA: Consulting Psychologists Press.

Harvey, R. J., Friedman, L., Hakel, M. D., & Cornelius, E. T. (1988). Dimensionality of the Job Element Inventory, a simplified worker-oriented job analysis questionnaire. Journal of Applied Psychology, 73, 639-646.  

Harvey, R. J., & Wilson, M. A. (2000). Yes Virginia, there is an objective reality in job analysis. Journal of Organizational Behavior, 21, 829-854.

Husserl, E. (1973). Experience and judgment: Investigations in a genealogy of logic. Evanston, IL: Northwestern University Press.

Klein, J. A. (1991). A re-examination of autonomy in light of new manufacturing practices. Human Relations, 44, 21-38.

Lewin, K. (1951). Field theory in social science; selected theoretical papers. In D. Cartwright (ed.). New York: Harper & Row.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

McCormick, E. J., Jeanneret, P. R., & Mecham, R. C. (1972). A study of job characteristics and job dimensions as based on the Position Analysis Questionnaire (PAQ). Journal of Applied Psychology, 56, 347-368.

Maurer, T. J., Wrenn, K. A., Pierce, H. R., Tross, S. A., & Collins, W. C. (2003). Beliefs about ‘improvability’ of career-relevant skills: relevance to job/task analysis, competency modelling, and learning orientation. Journal of Organizational Behavior, 24, 107-131.

Moore, F. I. (1972). Dictionary of family planning job tasks. New Orleans: Tulane School of Public Health.

Moore, F. I. (1999). Guidelines for task analysis and job redesign. Geneva: World Health Organization.  Available at http://www.who.int/hrh/tools/job_analysis.pdf.

Morgeson, F. P., & Campion, M. A. (2000).  Accuracy in job analysis: Toward an inference-based model. Journal of Organizational Behavior, 21, 819-827.

Nadin, S. J., Waterson, P. E., & Parker, S. K. (2001).  Participation in job redesign: An evaluation of the use of a sociotechnical tool and its impact. Human Factors and Ergonomics in Manufacturing, 11, 53-69.

Nunnally, J. C., & Bernstein, I. H. (1994).  Psychometric theory (3rd ed.). New York: McGraw-Hill.

Older, M. T., Waterson, P. E., & Clegg, C. W. (1997). A critical reassessment of task allocation methods and their applicability. Ergonomics, 40, 151-171. 

Prien, E. P., Prien, K. O., & Gamble, L. G. (2004).  Perspectives on nonconventional job analysis methodologies. Journal of Business and Psychology, 18, 337-348.

Rousseau, D. M. (1982). Job perceptions when working with data, people and things. Journal of Occupational Psychology, 55, 43-52.

Ryan, A. M., & Sackett, P. R. (1992). Relationships between graduate training, professional affiliation, and individual psychological assessment practices for personnel decisions. Personnel Psychology, 45, 363-387.

Sanchez, J. I., & Levine, E. L. (2000). Accuracy or consequential validity: Which is the better standard for job analysis data? Journal of Organizational Behavior, 21, 809-818.

Schmitt, N., & Chan, D. (1998). Personnel selection: A theoretical approach. Thousand Oaks, CA:Sage.

Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments: Is current practice state of the art?  Journal of Applied Behavior Analysis. Special Issue: Social Validity: Multiple perspectives, 24, 189-204.

Yano, E. M., Simon, B., Canelo, I, Mittman, B., & Rubenstein, L. V. (2000). 1999 VHA Survey of Primary Care Practices. Technical Monograph #00-MC12; VA HSR&D Project  #MPC-970121.

Wittgenstein, L. (1958). The blue and brown books: Preliminary studies for the philosophical investigations. New York: Harper & Row.

Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214