|
|
|
|
|
|
|
Citation: Warren, C. R., & Landis, R. S. (2007). One is the loneliest number: A meta-analytic investigation on single-item measure fidelity. Ergometrika, 4, 32-53. One is the loneliest number:
A meta-analytic investigation on Christopher R. Warren Ronald S. Landis ABSTRACT Introduction In response to concerns about reliability, Wanous and colleagues proposed several methods for assessing single-item reliability, including Spearman’s classic correction for attenuation formula (Wanous & Reichers, 1996; Nunnally, 1970), meta-analysis (Wanous, Reichers, & Hudy, 1997), and factor analysis (Wanous & Hudy, 2001). These attempts have primarily concentrated on demonstrating that one type of measure (i.e., single items) is psychometrically equivalent to, and may replace, alternative versions (i.e., multiple items). Because these arguments are grounded in statistical formulae and mathematically sophisticated techniques, arguments favoring the use of single-item measures may result in increased use of such measures. Unfortunately, such a practice may lead to a conceptual and substantive literature that is based on psychometrically questionable measures. Such outcomes are possible given that researchers may afford greater weight to the! se arguments owing to the relatively sophisticated statistical concepts and techniques used to support the use of single item measures. Because the rigor of our measures solely determines whether the construct being investigated has been captured (Nunnally, 1967), the use of single-item measures should be sparse and well defended. For instance, Jaccoby (1978) illustrated the importance of this decision by questioning how comfortable people would be with the use of single-item measures of intelligence. In short, citing one of the aforementioned studies may take the place of a well thought out defense as to why a single-item measure was utilized rather than a multi-item scale. Perhaps too much attention has already been focused on the reliability estimates, when fundamentally, the primary purpose for calculating reliability is to provide support for the validity of the measurement instrument under question. As widely known, reliability is a necessary, but not sufficient for validity. Therefore, the assessment of validity should necessarily follow any determination of reliability. With this in mind, the purpose of the current paper was to evaluate the effect of single-item measures, as compared to multi-item scales, on resulting correlations with other variables. If single-item measures are associated with equivalent, or even stronger, correlations than multi-item scales, both researchers and practitioners would likely prefer to use such scales for practical reasons such as shorter surveys, lower administration costs, and potentially higher face validity (Wanous, Reicher, & Hudy, 1997). Alternatively, there would be few compelli! ng theoretical or practical arguments in favor of single-item measures if correlations between variables measured using single items are substantially lower than their multi-item counterparts. The following section summarizes the literature regarding single-item test reliability and its estimation. In addition, the implications of having low reliability are discussed in order to fully explain the importance of accurately portraying single-item measure fidelity. Finally, a demonstration of the implications of choosing a single-item versus multiple-item measurement scale is offered as a means to reemphasize the importance of avoiding measures of poorer quality with the same vigor we hold towards ill-defined constructs, inappropriate operationalizations, or lack of validity. Why Use Single Items? At first glance, there appear to be numerous advantages to using single-item measures to assess constructs in research and applied domains. First, the time involved in judging a single-item is much less when compared to a multi-item measure. Second, single-item measures are practically efficient insofar as they require less time to construct and take up less physical space on a survey than do multi-item measures. Third, the face validity of single item measures is argued to be much higher than measures with multiple items and facets (Wanous & Hudy, 2001). In addition, single-item measures might be easily altered for different populations or constructs of interest by changing a referent in the stem of the item (Nagy, 2002). For instance, when switching the construct of interest, a researcher or practitioner utilizing a single item may only need to perform minor editing to the item stem, rather than rewrite an entire scale. Finally, single-item measures all! ow the respondent to uniquely weight any relevant underlying dimensions into the single, overall judgment. For example, the Faces scale allows the respondent to determine what aspects of the job are most central and to what extent each aspect influences an overall judgment of job satisfaction. Alternatively, multi-item measures such as the Job Descriptive Index (Smith et al., 1989) choose the relevant dimensions and impose a psychometrically derived weighting scheme for all test takers. Although multiple-item global assessments of constructs are also weighted by the respondent, these scales are less practical in every other sense discussed. Clearly, single-item measures have many possible benefits that make their use very attractive to both researchers and practitioners (Nagy, 2002). In fact, Scarpello and Campbell (1983) concluded that the best possible measure of job satisfaction was a single item asking individuals to rate overall how satisfied they are with their job. Likewise, others argue that when a construct is “sufficiently narrow” or is unambiguous to respondents, measures with single items may be appropriate and useful (Sackett & Larson, 1990). Further, within individual investigations there seem to be acceptable correlations between global single-items and multiple-item scale scores. For example, Van den Berg and Feij (2003) found a strong relationship (r = .68) between a global job satisfaction single-item, “All in all, how satisfied are you with your current job?”, and a comparable 15-item scale. Reliability Estimation for Single-Item Measures Wanous and Reichers (1996) asserted that single-item reliability could be estimated by using Spearman’s correction for attenuation formula. This formula, reported in nearly all measurement texts, is
where rtrue is the estimated correlation having removed the influence of measurement error in both variables, rxy is the observed correlation between two variables, and rxx and ryy are the reliabilities for the two variables, respectively. Spearman’s formula has frequently been used to estimate the “true” relationship between two variables by controlling for measurement error. Wanous and Reichers (1996) and Wanous, Reichers, & Hudy (1997) modified the original Spearman formula (1) to estimate the reliability of single-item measures by relying on the observed correlation between a multi-item measure and single-item measure of the same construct. Because the reliability of the multi-item scale (the y-variable in the current example) can be computed and the “true” correlation between measures is assumed to be perfect (i.e., rtrue = 1.0), the only unknown is the reliability of the single item measure. Thus, (1) may be written as
Rearranging terms in order to isolate the single-item reliability (rxx ) yields
Finally, squaring both sides produces
To illustrate this method, Wanous and Reichers (1996) evaluated single-item measures of global and facet-based job satisfaction, perceived amount of participation, and desired amount of participation. These measures were administered to 506 members of a manufacturing organization at two time periods. For each scale, the correlations between global and facet-based measures were obtained, along with the internal consistency of the facet-based scale at both Time 1 and Time 2. Results indicated that single item measures correlated with multi-item, facet-based measures in a range from .60 to .72 for job satisfaction, perceived amount of participation, and desired amount of participation. Using these observed correlations and the modified Spearman formula, Wanous and Reichers (1996) reported estimated reliabilities ranging from .45 to .62 for single-item measures. Importantly, in the Wanous and Reichers (1996) approach, estimated single-item reliabilities were calculated under the assumption that the correlations between scores on two measures of the same construct were from the exact same domain and perfectly correlated, and thus a value of 1.0 was used as the hypothesized value of rcorr. Although this seems like a tenable assumption, when looking at bivariate correlations between single and multiple item scales, the values rarely approach 1.0. When this assumption was relaxed (i.e. the assumed true correlation between x and y was set to .90), the average single-item reliability rose to .70. On the basis of these analyses, Wanous and Reichers (1996) concluded that although researchers have traditionally avoided the use of single-item measures, they appear to demonstrate adequate reliability in some instances. Further, when the assumption that both global and facet-based scales are from the s! ame domain is relaxed, the reliability for the single-item measures is even higher than previously assumed possible. Wanous et al. (1997) replicated the preceding results using meta-analysis. Specifically, studies were gathered that contained a single and multiple-item measure of the same construct, job satisfaction, and the mean correlations (corrected for reliability) were utilized with the modified Spearman formula to calculate the minimum-level reliability estimates, which ranged from .45 to .69 (Wanous et al., 1997). Additionally, Nagy (2002) demonstrated that single-item facet measures were comparable to facet-scores on job satisfaction measures with reliabilities in the range from .62 to .76. Finally, Wanous and Hudy (2001) concluded that estimated single-item reliabilities could be as large as .80. In short, several independent investigations have obtained results supporting the use of single-item measures. There are, however, other instances in which the results have not been as encouraging for single-item reliability estimates. For example, Loo and Kells (1998) tested the method proposed by Wanous and colleagues with quality of life data gathered from 562 teachers and administrators in a community college. Reliability estimates were not strong for overall job satisfaction (.45) and job dissatisfaction (.21), but were more favorable for satisfaction with supervision (.80) and work group (.57). On the basis of these findings, Loo and Kells (1998) cautioned researchers on the use of single-item indicators, especially with complex constructs such as quality of work life. In sum, although the literature provides cautious support for the use of single-item measures, little attention has been drawn to the implications of low reliability for estimates of validity in primary investigations. According to classical test theory, variance not attributable to true variance is considered error (importantly, random error). Because error is defined as random, error variance should not correlate systematically with scores on any measure. Classical test theory can also be extended to the distribution level, where the observed variance for a set of scores is proposed to be due to true variance plus error variance. To the extent that a measure demonstrates high reliability the proportional contribution of error is smaller than when a measure has low reliability. Conversely, true score variance is proportionally reduced as reliability decreases, and amplified as reliability increases. Because only systematic variance in one variable can be related to systematic variance in another variable, low reliability serves to place a limit on observed correlation coefficients. As a result, the correlation coefficients associated with certain measures will alwa! ys be constrained by the reliability of those measures. Obviously, the extent to which observed correlations are attenuated due to low reliability has serious implications for understanding relationships between variables. The Current Study As previously stated, the purpose of the current investigation was to examine the use of single-item measures from an alternative perspective. Because a measure’s reliability places a cap on the extent to which scores on that measure might correlate with other variables, and because of the difficulty associated with assessing the reliability of scores on single item measures, we examined correlations as indices of the usefulness and fidelity of single-item measures. That is, a comparison of estimated population correlation coefficients for both single and multiple item scales was performed in order to provide evidence regarding how single-item scales function in practice, across studies. Fundamental to this argument is the fact that reliability is one component of relationship strength, and is inconsequential if adequate and stable correlations between measures cannot be obtained. To this end, a meta-analysis was conducted to test the fidelity of measur! ement scales consisting of multiple items, or simply a single item. In order to evaluate possible differences in correlations between constructs measured with single- versus multiple-items, a body of literature was selected in which both types of measures have appeared with some frequency. Specifically, two frequently studied job attitudes, turnover intentions (TOI) and job satisfaction (JS), along with two well-established personality dimensions, positive and negative affectivity (PA and NA, respectively), were selected. The primary goal was to determine if relationships (i.e., correlations) between predictor variables (e.g., personality) and criteria of interest (e.g., job attitudes) differed by the type of measurement scale utilized (i.e., single versus multiple items). Again, these particular outcome variables were selected to serve as an illustration because they represent commonly included variables in organizational studies and have historically been measured through both single and multiple-item scales. A brief summar! y of the relevant literature with respect to these variables and the expected relationships serves as a substantive foundation upon which to evaluate the key measurement question of interest. Variables Turnover Intentions. Turnover intentions (TOI) generally refer to an individual’s intent or willingness to quit or voluntarily leave an organization. This specific job attitude was utilized in the current investigation for several reasons. First, TOI has been argued to be a critical variable in terms of both costs to the organization and the disruption of organizational functioning (e.g., Cascio, 1991). Second, previous studies that have examined the correlations between turnover intentions and affect-related variables using meta-analysis have found statistically significant estimated correlations (e.g., Thoresen, Kaplan, Barsky, Warren, & de Chermont, 2003). Finally, previous recommendations warrant that a variable must be theoretically unidimensional and sufficiently narrow in order to assess it with a single item (e.g., Sackett & Larson, 1990; Scarpello and Campbell, 1983), criteria that should easily be met by TOI. Job Satisfaction. Locke (1976) defined job satisfaction (JS) as, “…a pleasurable or positive emotional state resulting from the appraisal of one’s job or experiences (p. 1300).” Typical dimensions assessed in facet-based JS scales include work, pay, promotions, recognition, benefits, working conditions, supervision, co-workers, and the company and management (e.g. Locke, 1976; Smith et al., 1989). These dimensions often are measured with several related items. For instance, the general category of work might include items related to the control, autonomy, variety, opportunity, and intrinsic interest with the job tasks. An alternative way to measure job satisfaction is through global assessments, in which questions assess reactions to the job as a whole, rather than priming specific facets. For instance, a popular single-item measure for JS is known as the Faces scale(Kunin, 1955), in which respondents rate their! reaction according to a continuum of emotionally expressive faces. Positive and Negative Affect. Watson (2000) defined affect as the phenomenological state of feeling positive emotions as well as negative emotions, such as happiness or depression, respectively. Researchers have offered differing perspectives on the distinctiveness of positive versus negative emotions, as either two separate poles (e.g. Barsade, Brief, & Spataro, 2003) or opposite ends of the same spectrum (Watson, Wiese, Vaidya, & Tellegen, 1999). The majority of investigations in the organizational sciences have followed the conceptualization of distinct constructs representing positive and negative affectivity. Therefore the current study followed this framework in order to collect as many individual correlations as possible for the meta-analysis. Expected Relationships between Study Variables Several studies have provided evidence regarding the nature of the relationships between the affect variables (PA and NA) and job attitudes (TOI and JS) chosen for the current investigation. For instance, Cropanzano, James, & Konovsky (1993) illustrated that PA was related to both job satisfaction and organizational commitment. Likewise, Chan (2001) found NA to be negatively related to both job satisfaction and commitment. In addition, several large scale meta-analyses have been conducted in which trait affect (Connolly & Viswesvaran, 2000) and traits within the Big Five typology (Judge, Hller, & Mount, 2002) have both been related to job satisfaction in the predicted directions, with positive traits (e.g. PA, extraversion) leading to higher levels of satisfaction, and negative traits (e.g. NA, neuroticism) leading to lower levels of satisfaction. Finally, a recent, comprehensive quantitative review of the literature has also provided support for p! ositive relationships between PA and job satisfaction and commitment and negative relationships between NA and these criteria (Thoresen et al., 2003). Based on the relatively strong theoretical and empirical support, the following relationships were expected between the study variables. First, there should be a negative, non-zero, relationship between measures of positive affect and turnover intentions. Second, there should be a positive, non-zero, relationship between measures of negative affect and turnover intentions. Third, there should be a positive, non-zero, relationship between measures of positive affect and job satisfaction. Finally, there should be a negative, non-zero, relationship between measures of negative affect and job satisfaction. Importantly, the preceding expectations were not considered as formal hypotheses in the current study. As a result, support was not deemed necessary to evaluate questions regarding the fidelity of single versus multiple-item measures. Instead, these expectations served as guidelines to evaluate the results of the various moderator analyses for each relationship.! The primary study hypothesis was generated based on several considerations dealing with single-item assessments of constructs. First, as described in classical test theory, the observed scores on any given construct are made up partially of error, and the degree of error vacillates with the quality of the indicator. Further, the reliability of measures places an upper limit on the magnitude of correlations between variables, and although numerous studies have demonstrated “acceptable” coefficients, there seems to be other instances where the stability and consistency of single-item measures is less than desirable (e.g., below the accepted .70 cutoff). Therefore, the primary hypothesis was:
METHOD Identification and Selection of Studies Studies were located through a variety of methods in order to maximize the accuracy of the estimates derived through meta-analysis. The first step was an electronic search using a number of internet-based search engines including PsychINFO (1877-July 2005) using the search terms of the three primary variables: positive affect*, negative affect*, neuroticism, emotional stability, extraversion and intentions to leave or turnover, and job or occupational satisfaction. This initial search resulted in approximately 218 studies for TOI, and 1875 for JS. In addition, manual searches were performed for the major industrial/organizational psychology and organizational behavior journals (e.g., Academy of Management Journal, Journal of Applied Psychology, Personnel Psychology) along with major review pieces (e.g., Brief, 1998). The total set of studies gathered was coded at least twice by four independent, trained, graduate students and faculty in an industrial/organizational psychology program who reached consensus on whether each study was to be included, along with the information to be utilized in the meta-analysis (e.g., effect sizes). To be included in the analyses, a requirement was that a primary study included at least one affective variable (either PA or NA) and one or more of the job attitudes (TOI or JS). The literature produced large variability in the measurement of both affect variables and most measures were accepted for the current purposes, including state and trait measures. However, several other conditions must have been met for the study to be included in the analyses. For instance, each study must have described the measure of TOI or JS with enough detail in order to classify the scale as consisting of a single item or multiple-items. Further, the relationship! between the affect variable and the outcome must have been presented as a correlation coefficient or some other effect size (e.g., d statistic). This process resulted in a total of 18 independent samples including PA and TOI, and 34 for NA and TOI. In terms of JS, a total of 76 independent samples were found for PA and 169 for NA. The sample sizes for each moderator condition, along with the observed correlations and the type of measure utilized in each study, are listed in Appendix A. Coding of Studies Each study’s abstract was first analyzed for content including the constructs and measures of interest. Next, the methodology of each study was investigated to determine if a measure of PA or NA and an outcome was included, along with an effect size. All the studies subsequently included reported correlation coefficients with known sample sizes for which they were obtained. The measure, or type of measure, for each scale was recorded along with the author and scale properties. Differences between state and trait measures of PA and NA were not investigated for the current analyses. Previous work has shown that the magnitude of population estimates between PA, NA, and various job attitudes do not differ (along with overlapping confidence intervals) for the state-trait distinction (see Thoresen et al., 2003). Finally, each outcome measure, whether TOI or JS, was coded as a single or multiple item measure solely based on the information provided within the ! study. As stated earlier, in those cases in which this level of description was not given, the study was omitted. Meta-Analytic Calculations Because the purpose of the current study was to evaluate differences in observed correlation coefficients across single and multiple item scales and not to generally estimate the affectivity-job attitude relationships, no corrections were made based on predictor or criterion reliabilities. This procedure provides a more accurate illustration of the difference in measurement scales for two reasons. First, the estimation procedures used to determine single-item scale reliabilities are not widely accepted, nor widely employed, and are central to the current investigation. Further, when tests of single item reliabilities have taken place, the estimates have not always been favorable (Loo & Kells, 1998). Second, the most conservative estimate of true score correlations between constructs within meta-analysis would be a “bare bones” analysis, correcting only for sampling error. Therefore, the two most commonly utilized procedures for estimating popu! lation-level relationships Raju, Burke, Normand, and Langlois (1991) and Pearlman, Schmidt, and Hunter (1980) would result in no differences for the current analyses, because the difference between these methods lies in how to produce missing reliability artifacts for correction purposes beyond sampling error. The analyses were conducted using MetaWin 1.6 (Rosenberg, Adams, & Gurevitch, 1997), which quantitatively combines studies and calculates the meta-analytic estimates for each of the conditions. Specifically, each group of studies with either PA or NA and correlations with TOI and JS measured with single or multiple item scales were entered into the program. The data entered included the sample size for the correlation in the primary study, the effect size (all in correlations), and the reliability information (entered as 1.0). RESULTS Results are shown in Table 1 and Table 2 for TOI and JS, respectively. The values included in the meta-analyses were the sample size for each study and the effect size (correlation) between one of the affective variables (PA or NA) and each outcome. The program used to test the hypotheses (MetaWin; Rosenberg et al., 1997) provided several values important to interpreting the results of the current investigation. First, the sample size weighted population value for the relationships, corrected for sampling error only, is indicated by Mp. Again, these estimates are “bare bones”, as no corrections for unreliability in the predictor or criterion were performed. Further, SDp indicates the variance of the population mean correlation (Rho), and SEM is the standard error of the population mean, both estimates of variability. Finally, the 95% C.I. refers to the 95% confidence interval around the population mean, which indicates th! e estimated population values as non-zero (not encompassing zero), along with the variability of the relationship. The general expectations concerning each affect variable and the outcome variables were supported by the data. Specifically, PA showed consistent negative average correlations with TOI, and positive relationships with JS, across moderator conditions. Further, NA showed positive correlations with TOI and negative relationships with JS. The remainder of the results will focus on the moderator conditions testing the fidelity of single-item versus multiple-item measures, by comparing the strength of each of these relationships. TOI Results Evaluation of the PA-TOI results reveals support of the prediction that single-item measures would show less strong relationships than multi-item measures. Specifically, the overall estimate of the population mean was smaller in magnitude for single item measures (Mp = -.03) than for multiple item measures (Mp = -.18) and the variability of the population mean was somewhat larger for single item measures (SDp = .13), than multiple item measures (SDp = .09). Along with these estimates, interpretation of the 95% confidence intervals was also generally consistent with expectations. First, the confidence interval for PA-TOI correlations using single item measures encompassed the zero point, (-.08↔.02) whereas the confidence interval for PA-TOI with multiple item measures did not (-.21↔.15). In addition to examining the confidence intervals associated with each estimated population parameter, a Z test was condu! cted to examine the sig nificance of the difference between comparable correlations using the following formula reported by Quinones, Ford, and Teachout (1995). (The formula provided in Quinones, Ford, & Teachout (1995) incorrectly contains a minus sign in the denominator.)
For the PA-TOI relationship, the Z test was significant (Z = 4.16, p < .01), indicating a statistically stronger relationship was observed when using multiple items to measure TOI as compared to single item measures of this criterion variable. For the case of NA, a similar pattern held whereby single item measures had weaker relationships with TOI (Mp = .18) than multiple item measures (Mp = .26). Again, the Z test was significant (Z = -3.58, p < .01). The variance of the population mean estimate was consistent with the primary study hypothesis in that the single item measures had much more variability (SDp = .21) than the multiple item measures (SDp = .07). For the NA-TOI confidence intervals, the interval for single-item measures (.14↔.22) was lower than that for multiple-item scales (.24↔.29), and showed no degree of overlap.
JS Results Results associated with JS as a criterion were also supportive of the hypothesis. Specifically, in the case of PA, the overall estimate of the population mean was smaller in magnitude for single item measures (Mp = .12) than for multiple item measures (Mp = .30), although the variance of the population mean was lower for single item measures (SDp = .00), than for multiple item measures (SDp = .13). The confidence intervals illustrated the difference in conditions with the interval for single-item JS measures very near the zero point (.05↔.19), whereas the interval for multiple item measures was much more desirable (.28↔.31). As with previous comparison, the Z test was significant (Z = -4.36, p < .01). For the case of NA, the same pattern or results was observed. Specifically, single-item measures were associated with a lower estimated population effect size (Mp = -.18) than multiple-item measures (Mp = -.29), and the standard deviation of the population estimate was less for the single-item measures (SDp = .05) than the multiple item measures (SDp = .11). The confidence intervals clearly illustrate the difference between the two measurement scales, whereby single-item measure estimates were lower in magnitude and more variable (-.21↔ -.15) than the multiple-item measure est! imates (-.29↔ -.28). Consistent with previous analyses, the Z test was significant (Z = 10.95, p < .01).
DISCUSSION The purpose of the current study was to evaluate the resulting criterion-related validities of single-item measures as compared to multi-item scales. In order to accomplish this, studies were gathered and categorized as utilizing single or multiple-item measures of typical job attitudes (i.e. TOI and JS) and two general personality dimensions (i.e. PA and NA). Meta-analytic techniques were used to estimate average population correlations between each of the personality-job attitude pairs. Results indicated that the overall population level relationships between personal characteristics and job attitudes were lower in magnitude with single-item versus multiple-item job attitude measures. Additionally, the 95% confidence intervals were consistently lower and even contained the zero point for the PA-TOI relationship. These differences between the population values for single and multiple item scales were consistent across all moderator conditions, although most pronounced for the PA-JS relationship. In sum, the data tend to support the prediction that the correlations associated with single-item measures are not as strong as compared to multiple-item scale correlations. Therefore, recently presented views that (a) estimates of single-item measure quality support their use, and that (b) these measures are largely comparable to multiple-item measures, are not warranted from these results. Limitations There are several limitations to the current study which should be noted. First, although large scale quantitative reviews are intended to speak to the generalizability of relationships, all have similar shortcomings, along with some more idiosyncratic to the question posed in this investigation. For instance, the search for and acquisition of studies always suffers from the possibility of non-inclusion for both systematic and random reasons. Even if multiple search options are utilized, such as in the current investigation, the possibility of publication bias (e.g. the file-drawer problem; Rosenthal, 1979) may limit the meta-analysis to include only significant results, either positive or negative in direction. Further, as evidenced in the Appendix, inconsistent and uneven reporting of the measures utilized, or reliability information, hinders research synthesis as well as interpretation. Another possible limitation involves the variables chosen to include in the meta-analysis. Whenever personality is investigated, researchers must choose from a plethora of conceptualizations, ranging from a two-factor model (e.g. EPI) to a 16-factor model (e.g. 16pf). Although the number of options may be advantageous to any individual research agenda, the number and relationship of various conceptualizations creates a myriad of problems when viewed from a synthesis standpoint, where generalization is the primary goal. For instance, PA and NA are often measured in a state and trait manner, sometimes simultaneously. Specifically, the State/Trait Anxiety Inventory (Spielberger et al., 1969), which measures a subfacet of NA in a temporally proximal and more general context, was reported in a number of included studies. Although the debate on the equivalence of affect and affectivity in measurement will most likely continue for its relevance to specific investiga! tions, previous meta-analytic results have illustrated relatively no difference for the variable included in the current study (Thoresen et al., 2003). There has been discussion as to the discrete nature of state versus trait measurements of affect, tapping into dispositional or situational aspects of individuals. However, there is a growing body of literature in organizational studies that speaks to the similarity of such conceptualizations for organizational measurement purposes. For instance, George and Jones (1996) suggested that trait PA leads to positive work outcomes through the induction of positive mood, or state PA. That is, state PA is posited to act as a mediator in the relationship between trait PA and the attainment of work-related goals and desire to leave the organization (George & Jones, 1996). NA has been argued to operate in a similar manner, with trait NA leading to state NA along with negative moods (George & Jones, 1996), and subsequently resulting in lower levels of job satisfaction and organizational commitment (Mowday, Steers, & Porter, 1979). Beyond the state versus trait! issue, global and facet-based scales may be assessing distinct constructs within respondents. Specifically, although scholars have emphasized the appropriateness of single item scales for sufficiently narrow constructs (e.g., Sackett & Larson, 1990), the operationalization of the construct in such a way may contaminate the results via method bias, or even assess a completely different concept or job attitude. Further research should address both the issues of contamination and construct equivalence between single and multiple item scales by looking beyond how the two types of measures correlate (e.g., Van den Berg & Feig, 2003). A more serious threat lies with equating the various typologies, dimensions, and subfacets of personality in general. In fact, the same large scale meta-analysis that found no differences between state and trait measurements (Thoresen et al., 2003), found large differences between pure measure of PA and NA (e.g. PANAS) and measures developed from the Big 5 typology (e.g. NEO, EPI), or those including extraversion and neuroticism. Unfortunately for the current investigation, because researchers tend to use both types of measures equally as often, omitting one entire group of measures would have substantially decreased overall sample sizes and studies. Importantly, the primary result of combining pure affect measures with other personality measures will be more conservative estimates of the population values and a possible attenuation of the true differences that could exist between the two measurement strategies. There are several statistical issues important to recognize when interpreting the present findings. Although scales measuring TOI and JS varied in a number of ways, such as focus (e.g. affective vs. cognitive), number of items, and dimensions, two dichotomous groups (single and multiple item) were evaluated in the current study. Obviously, there are many other possible moderators that are collapsed across in this process, clouding other important determinants of relationship strength. Further, this dichotomization of variables limits the statistical power associated with testing differences between the moderator conditions (Sagie & Koslowsky, 1993). Finally, the sample sizes in some moderator conditions were relatively small, which limits statistical power in detecting differences in moderator conditions and produced wider confidence intervals (Oswald & Johnson, 1998). Summary Current results may have been observed for a plethora of reasons, such as the role of state affect on single-item versus multiple item measures. For instance, the "cookie study" reported by Brief, Butcher, and Roberson (1995) illustrated that individuals reported higher job satisfaction after given treats to serve as mood inducements. In this instance, these authors utilized a single item measure of job satisfaction; the Faces scale (Kunin, 1955) with a multiple item scale, the JDI (Smith et al., 1989). The possibility exists that the induction utilized by Brief et al. (1995) might not have resulted in inflated job satisfaction responses if only the single-item scale had been used, or possibly greater differences due to the mood manipulation. This argument represents the possibility of contamination in assessment when using a single item measure, where less attention is directed towards specific facets of organizational life. Some support for this as!
sertion can be found in the correlations reported between single and multiple items scales, such as those offered by Wanous and Reichers (1996). Strong relationships were present, but not necessarily so high as to say the scales were measuring the same underlying construct domain. Until there is a more explicit conceptual understanding of how respondents interpret and respond to single-item measures, the debate on their use will almost certainly continue. However, the results of the current study seem to support the assertion, that in the world of measurement scales, one truly is the loneliest number. REFERENCES
*Abdel Halim, A. A. (1982). Social support and managerial affective responses to job stress. Journal of Occupational Behaviour, 3, 281-295.
|
|
|
|
|
|
Copyright © 2009 The Mitchell Project, LLC. All rights reserved. |
|