Editors · Sponsors · Topics · Submissions · Reviews · Free Subscription
View Journal Contents by: Volume · Topic · Author

Citation:
Mulqueen, C. M., Stetz, T. A., Beaubien, J. M, & O'Connell, B. J. (2001).  Developing Dynamic Work Roles Using Jaccard Similarity Indices of Employee Competency Data. Ergometrika, 2, 26-37.

Developing Dynamic Work Roles Using Jaccard Similarity Indices of Employee Competency Data
 

Abstract

This paper describes a method for developing work roles that employs a quantitative and qualitative approach. Competency information from an electronic Knowledge Management System (KMS) was analyzed using the Jaccard (1912) similarity coefficient to assess skill and knowledge overlap among work roles. Based on the results of these analyses, groups of Subject Matter Experts (SMEs) determined how to categorize the work roles. Over 160 work roles were developed that describe the work performed within an organization. The work roles are stored in an electronic KMS, resulting in a dynamic system that allows the work roles to be efficiently updated as competencies change (i.e., real-time updates). The various uses of dynamic work roles is discussed, as well as the advantages and challenges of this approach to work role development and maintenance.

Developing Dynamic Work Roles Using

To make effective personnel decisions, organizations must first describe the work requirements of individual positions (Cascio, 1991). A typical job description lists the primary duties and tasks to be performed within the position. The development of job descriptions usually requires a significant amount of effort from both Human Resources (HR) professionals and job incumbents. Even with such efforts, job descriptions tend to be extremely general and do not adequately distinguish one job from another. As a result, they are useful for only the most basic of purposes (e.g., recruitment). Job descriptions have also been criticized for being static snapshots of dynamic jobs (Arvey & Murphy, 1998; Cascio, 1995).

Task-oriented job descriptions would have greater utility if they included not only a description of duties, but also the specific competencies that are required to function in the position. The descriptions would have further utility if the list of competencies was dynamically based. That is, the competency requirements should be easily updated as the positions evolve without undo time and resource expenditure. Job descriptions can be readily updated if the position competencies are generated from an electronic knowledge management system. Within a knowledge management system, employees can develop personal competency profiles that describe the competencies necessary to perform in their positions. When updated on a regular basis, this data can be easily analyzed across time in order to make necessary adjustments to job descriptions.

In recent years, a number of Industrial and Organizational Psychologists have begun to re-conceptualize the world of work from the narrow view of jobs that are based on specific tasks and duties to the broader view of work roles that are based on the competencies required to function in the position (Cascio, 1995; Peterson, Mumford, Borman, & Jeanneret, 1991). Work roles are distinct from job descriptions in that they not only define the job tasks but also describe the required competencies. Drawing on this body of work, this paper describes a scientific procedure for developing a multitude of work roles within an organization. This procedure allows the work role descriptions to be easily revised as the work roles change over time. Moreover, it allows the organization to easily re-organize the work roles, thereby helping it adapt to environmental threats. Finally, the procedure draws on the strengths of both qualitative and quantitative analyses.

Work Role Uses

Scientifically developed work roles can be used for a variety of purposes. First, work roles can aid in strategic planning initiatives. Because employee competency information is captured at the work role level, managers can efficiently project not only the types of work that will be necessary in the future, but also the competencies required to perform this work. This information feeds naturally into a variety of HR processes, thereby providing a basis for developing selection and promotion instruments.

Second, work roles are uniquely suited for organizational re-alignment purposes. Because work roles fit logically into organizational functions or units, reorganization along functional lines is relatively straightforward. Alternatively, if management decides to re-organize based on work process flow, the work roles can be easily organized based on their chain in the product cycle.

Finally, employees can use work roles for career path planning and personal development. For example, employees can read the work role descriptions that are of interest to them, and identify specific competencies that they need to develop in order to progress into those roles. Armed with this information, the employees can actively seek out training and broaden current assignments to develop the necessary skills. As a result, work roles put career planning and personal development within the control of individual employees.

Study Background

Founded in October 1996, the National Imagery and Mapping Agency (NIMA) is America’s newest intelligence agency. NIMA’s mission is to provide timely, relevant, and accurate geospatial intelligence in support of all national security initiatives. NIMA was formed by consolidating employees from several federal agencies, including the Defense Mapping Agency, the National Photographic Interpretation Center, the Central Imagery Office, the National Reconnaissance Office, and the Defense Airborne Reconnaissance Office.

Recognizing the need to efficiently manage their human capital and to promote a single organizational identity, the Federal Government authorized NIMA to create a new, integrated HR management system from the outset. This system was designed to ensure that NIMA had a flexible, highly skilled workforce that could respond to the rapidly changing environment in which they operate. To achieve this goal, NIMA needed a system that would be strategically oriented and person-based. This system is currently being used to align several of NIMA’s HR practices with their strategic mission. Eventually, it will serve as the basis for all their HR functions, including recruitment, manpower planning, compensation, promotion, training and development, and career path planning.

This paper describes the procedure used to develop dynamic work roles at NIMA. The work role concept emerged out of two primary needs. First, the agency did not have a convenient method for describing the work performed by employees or the work requirements of job assignments. Prior to the development of work roles, employees were assigned to broad occupations (e.g., job families). Within each occupation, however, many distinct types of work existed. In order to more efficiently make personnel decisions, NIMA needed more thorough descriptions of work at the employee level. Second, work roles were necessary for the agency to be in compliance with Federal regulations, such as those involving reductions in force (RIF). For example, in the event of a RIF, work roles would provide the necessary information for making workforce decisions.

Method

Design and Procedure

Work roles were developed through four distinct steps. First, panels of Subject Matter Experts (SMEs) described and defined different kinds of work within their occupations. Second, competency data from work role representatives was collected. Third, the competency data were analyzed for relevance and similarity among work role incumbents. Finally, SMEs from the parent occupations evaluated the information from the analyses and finalized the work role descriptions and competency requirements. These steps are described in more detail below.

Step One: Define Work Roles. Initially, groups of 3-6 SMEs from each of the agency’s 24 occupations met to identify meaningful distinctions among jobs within their occupation. These SME panels developed preliminary titles and general descriptions for work roles. The panels were facilitated by an Industrial and Organizational Psychologist as well as a member of the agency’s HR staff. These facilitators provided guidance for developing the preliminary work roles. Work roles were defined as "a group of work assignments with common duties and responsibilities that use common skills, knowledge, and tools." The SMEs were instructed that each work role should:

1. Define a different "kind" of work (e.g., systems engineer vs. security officer) and each should require a unique set of skills, knowledge, and tools (SKTs).

2. Be narrow enough to describe the efforts of a single employee (not the collective efforts of many employees) yet broad enough to describe all the duties of that employee (not just a subset of the tasks that employee performs).

3. Define a set of employees who are essentially interchangeable (i.e., all of the employees described by a work role could pick up the duties of any other employee described by the same work role and be minimally proficient within 90 days).

At the conclusion of the SME meetings, each occupation had a defined set of work roles. At this stage, a total of 198 preliminary work roles were identified and defined. Each occupation then identified a sample of employee representatives for each work role. Representatives were chosen based on their current work duties; they were believed to be working in the roles that were described at this step. There was an average of six representatives per work role, with a range from one to twelve.

Step Two: Collect Competency Data. The work role representatives were tasked with completing a competency profile of the skills, knowledge, and tools (SKTs) that are currently used to perform the work role. These profiles were created using a web-based hierarchical knowledge management (HKM) system. The HKM system is based on the Occupational Information Network (O*NET) that was developed for the U.S. Department of Labor to describe the world of work using a series of taxonomies for describing job and worker characteristics (Peterson, et al., 1999). At the heart of the HKM system is a relational database that links SKTs to tasks, jobs, and career paths in a hierarchical format.

Step Three: Analyze Work Role Data. Using the representatives’ competency data, a role profile report was created for each work role. The profile listed each skill, knowledge and tool identified by representatives as currently used in the work role. The profile also indicated the number of representatives that currently use each SKT. The SKT profiles were analyzed using the Jaccard similarity coefficient (Jaccard, 1912) to assess the degree of overlap among individuals within each work role, as well as the overlap across work roles within each occupation.

If work roles were found to be very similar to one another in terms of the competency requirements, these work roles were considered not distinct, and managers were encouraged to combine them into a single role. Twenty-four sets of analyses were conducted, one for each occupation. Tables 1 through 3 contain similarity matrices, and Table 4 lists the SKTs identified by the representatives for a particular work role. These four tables were included as part of work role profile reports that were used by SMEs. Each table is described more fully below.

The Jaccard index was originally developed to assess similarity among distributions of flora in different geographic areas (Jaccard, 1912). The procedure results in a matching coefficient for binary variables in which joint absences are excluded from both the denominator and the numerator and equal weight is given to matches and non-matches:

Since the list of SKTs for each work role consisted of binary data (i.e., 1 = currently used in work role, 0 = not currently used), the Jaccard index was an ideal choice. For our analyses, the number of groups equaled the number of work roles within an occupation. Thus for the example analysis shown in Tables 1 through 4, 12 groups were analyzed.

Three separate Jaccard similarity analyses were conducted. The first analysis computed the degree of similarity among representatives within each work role (see Table 1). This essentially produced a measure of agreement among representatives regarding individual work role requirements. In order to be included in the analysis representatives were required to rate at least one skill, one knowledge, and one tool. Following this initial screen, the Jaccard coefficient was calculated among representatives within each work role. Tables 1 through 4 display work role information for the occupation entitled Information Services. Table 1 displays for each work role within the occupation, the number of representatives and the average percent similarity.

The second Jaccard analysis computed the degree of similarity between each individual work role and the pool of all other work roles within the occupation. For the example shown in Table 2, this matrix presents the results of each work role with the combination of the remaining eleven work roles. The third and final Jaccard analysis computed the degree of competency similarity between each pair of work roles. This pairwise matrix is displayed in Table 3.

Step Four: Refine Work Roles. Following data collection and analysis, SMEs from each of the 24 occupations met to refine the work roles. The purpose of this step was to evaluate the work role competency profiles that were created based on representatives’ competency data and similarity analyses. This information was used to create final sets of work roles. These meetings were again facilitated by an Industrial and Organizational psychologist and an HR representative, who explained the analyses and profiles, and guided the SMEs through the process of reviewing and revising the work roles.

First, the SMEs reviewed the similarity matrices in order to determine whether there was unusual redundancy of competency requirements among work roles. For these determinations, the pairwise similarity matrix was found to be more helpful than the pooled matrix. If a high degree of overlap among two or more work roles was found, this might indicate that the roles were too similar to be considered separate, and as a result, should be combined into a single work role. A criterion of 40% or greater similarity among work roles was utilized. This criterion, however, was not absolute. It merely served as an initiator for the SMEs to discuss the affected work roles and their requirements. Factors other than competency similarity, such as critical mission function and staffing requirements, were also used to determine whether work roles should be combined or remain separate. The final determinations were made by the SMEs based on their knowledge of the role requirements and any other reasons for keeping similar work roles separate.

In addition to the Jaccard similarity indices, each role profile contained the list of associated SKTs organized according to the number of representatives who indicated that they currently use each (see Table 4). If a majority of representatives used a SKT, it was identified as a "core competency." This organization of the information facilitated decisions about the final competency profiles by outlining which SKTs were used by a majority of representatives. The SMEs reached consensus on final lists of SKTs for each work role by noting which SKTs were core competencies and if any SKTs were redundant or missing. Once again, this was a matter of expert judgment guided by empirical information from the work role analyses.
 

Table 4
Competency Summary for the Database Administrator Work Role

As a final step in work role refinement, SMEs reviewed the work role description as it was originally written. Based on the analyses and revision of competency profiles, SMEs determined whether the descriptions required editing. The SMEs also indicated whether any specific education or licensure requirements were necessary for the work role, and if there were any special environmental or physical requirements for performing the work.

Results

The review and revision process (Step 4) resulted in the finalized work roles. The completed work role description for Database Administrator is included as an Appendix. In addition to the work role description, associated competencies, and education and environmental/physical requirements, a band level descriptor was also included. The band descriptors were developed to generally describe the level of performance required at different pay bands within the organization. Thus, all Band 4 descriptions are identical across work roles. The work role profiles themselves are independent of pay band level.

For the Information Services occupation, all 12 work roles were retained. This was in spite of the rather high degree of competency similarity between pairs of work roles, notably work role #6 "Database Administrator" and work role #2 "Data Management Specialist" (53.6% similarity). The SMEs determined that these functions were sufficiently distinct to maintain as separate work roles. Likewise, work role #11 "Gateway Customer Representative" shared a moderate degree of overlap with several other work roles. This would be expected since this work role involves assisting customers with a number of issues that are pertinent across the spectrum of functions that exist within the occupation. Thus it was logical to maintain as a separate work role.

Overall, 168 work roles were developed across 24 occupations. Following this development phase, all employees within the agency were assigned to the most appropriate work role. Work role information is currently being used in several workforce processes, such as strategic planning, organizational restructuring, creation of workforce standards, recruitment and selection, training needs analysis, and career planning.

Discussion

The methodology presented in this paper offers several advantages. First, it seems clear that in the modern world of work, job descriptions must include the competencies that are necessary to perform the work. Simple lists of tasks and duties will not suffice for the complex skills and knowledge necessary for many positions within the U.S. economy. This procedure identifies the important competencies associated with various work roles, thus overcoming the narrow focus of traditional job descriptions. The increased breadth of work roles allows them to be used for multiple purposes such as recruitment, selection, performance appraisal, training needs analysis, organizational redesign, and career development.

Second, many work roles are highly dynamic. Often the general tasks and duties may change only slightly, however the competencies necessary to complete those duties may constantly change. The methodology presented here provides a scientific means for creating work roles that are dynamic and highly functional. Work roles can be easily maintained and updated as the work itself, and associated competencies, changes. This will be accomplished by an annual analysis of work role representative competency data. Employees at NIMA regularly update their competency profiles for the purposes of career planning and to aid in mission accomplishment. The web-based design of the knowledge management system enables employees to easily access their competency profiles from their desktop computers. Thus it will not be burdensome to collect and analyze this data. A fairly brief process of reviewing the data and making adjustments to the work role descriptions and profiles will be all that is required.

Third, the methodology uses both qualitative and quantitative techniques. The advantage of qualitative and quantitative methods for developing jobs and job families has been a much-debated topic (Guion, 1998). This method draws on the strengths of both. The qualitative work role descriptions help to ensure that the work roles are in vertical alignment with the vision and competitive strategy of NIMA (Schipmann, 1999). The empirical analyses using the Jaccard Index serves to verify SME judgments and provides objective data to support the work roles, thereby increasing legal defensibility.

There are, however, certain limitations to this study. First, while the methodology is rigorous and results in a set of useful work roles, it is quite resource intensive. This procedure for work role development required a firm commitment from senior leadership at NIMA. There was a particularly intense investment of SME time for the project. Although the ongoing maintenance of the work roles is fairly simple, many organizations may find the initial resource expenditure prohibitive.

Second, the method relies on a pre-existing set of competencies that was specifically developed for NIMA. It would not be possible to develop work roles similar to those at NIMA without first having established the competencies that are important within the organization. The procedure utilized at NIMA was possible because of the hierarchical knowledge management system that had been developed and is being used at the agency. The web-based application eased the burden of data collection significantly, both for the employee representatives and for the researchers.

References

Appendix

Work Role Description for Database Administrator

WORK ROLE TITLE: Database Administrator

PAYBAND: 04; FLSA STATUS: Non-Exempt

WORK ROLE DESCRIPTION:

Database Administrators oversee the operation and maintenance of databases. They establish, enforce, and apply policy to ensure proper functioning and optimal performance of the database management system. They perform database backup and recovery, schema maintenance, and file management.

BAND LEVEL DESCRIPTOR:

Band 4 employees have duties that may involve nonstandard or inter-related assignments requiring analytical, research, or negotiating skills. Assignments may require specialized training and considerable experience in the specific field of specialty area. Band 4 employees may direct the work of similar or lower level personnel or serve as project leader on large complex tasks. Employees work with others to exchange information, identify problems, propose solutions, and defend recommendations on issues to all levels within NIMA and, at times, to other organizations and agencies. Supervision is generally administrative in nature, with the supervisor outlining the assignments’ critical or novel features and establishing important parameters. Completed work is reviewed to ensure that objectives were attained. Typical functions include representing NIMA to external organizations, policy interpretation, briefing large audiences, mentoring junior staff, managing budgets, managing projects, and managing contracts.

COMPETENCIES:

Skills

Access Control Computer Systems Administration Customer Service

Database Development Database Management Implementation Management

Quality Assurance Information Exchange Information Research

Information Retrieval Interpersonal Relations Oral Communication

Knowledges

System administration techniques Backup procedures System monitoring principles

System admin/mgt techniques Database file & system relationships

Tools

Advanced computer workstation Database compiler Software development tools

Informix database system Oracle database management system

EDUCATION: No formal requirements.

LICENSES/CERTIFICATIONS: No formal requirements.

ENVIRONMENTAL/PHYSICAL REQUIREMENTS: No formal requirements.

Presented at the 18^th Annual Conference of the Society for Industrial/Organizational Psychology, Orlando, FL, April 2003.

Independent Job
Analysis
Consulting Services

Metrica Inc.
Generating Solutions

Workforce Dynamics
Human Resourced
Redefined

Otago Polytechnic
School of Applied
Business
 

Copyright © 2009 The Mitchell Project, LLC. All rights reserved.