Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain

Assessment of individual differences in personality traits is arguably one of the hallmarks of psychological research. Testing the structural validity of trait measurements is paramount in this endeavor. In the current study, we investigated 30 facet traits in one of the accessible and comprehensive public-domain Five Factor Model (FFM) personality inventories, IPIP-NEO-120 (Johnson, 2014), using one of the largest US samples to date (N = 320,128). We present structural loadings for all trait facets organized into respective FFM-trait domain (Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness). Both hierarchical second-order and bi-factor models showed tolerable model fit indices, using confirmatory factor analysis in a structural equation modeling (SEM) framework. Some facet traits were substantially more representative than others for their respective trait domain, which facilitate further discussions on FFM-construct content. We conclude that IPIP-NEO is sufficiently structurally robust for future use, for the benefit of research and practice in personality assessment.

2016). The IPIP-NEO instrument makes use of this by including a number of facet traits, consisting of dispositions towards certain behaviors, affects, and cognitions within each factor domain (see Zillig, Hemenover, & Dienstbier, 2002). The 30 facet traits of IPIP-NEO are presented in Table 1, with 6 facet traits for each one of the 5 trait domains, arranged in columns, followed by example items. For instance, the first facet trait Friendliness in trait domain Extraversion (E1) attempts to capture the propensity of being friendly and experienced as socially warm person. One item example of the Friendliness facet trait is the extent one "makes friends easily".
Personality models are often constructed by items making up facet traits which in turn make up the broad trait domain, which aims at generating both content scope and precision (Johnson, 2014;McCrae, 2010). The extent to which this hierarchical structure is valid can be tested by assessing how items, facets, and trait factor are interlinked. Optimally, both trait and facet levels should be saturated by the variance provided by items. Note. Each one of the 30 facet traits in IPIP-NEO are exemplified with a typical item worded in italics.

The Present Study
The overall endeavor of the present study was to test the structure of one public domain version of the Five Factor Model (FFM) of personality, IPIP-NEO-120 (Johnson, 2014). The specific objective was to investigate how facet traits organize according to its proposed structure (  (Costa & McCrae, 1995). Confirming the structure of personality assessment in the public domain would give important information on scope and meaning of this publically available FFM to professionals who need a simple and accessible instrument in their research and practice.
Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain 262

Method Sample and Procedure
We were able to make use of one of the largest US samples to date (N = 320,128

Data Limitations
Despite the generous number of respondents this likely is not a representative nationwide sample. Due to the active volunteering of respondents, a reasonable assumption is that several of the trait facet levels such as in Openness to Emotionality, Openness to Intellect, and Altruism were likely unrepresentatively high (see Appendix), since these traits are known to characterize people interested in psychology (Vedel, 2016).
Another concern was that online surveys are often known for inconsistent responding or intentional misrepresentations. However, the quality of the present online survey data was analyzed according to guidelines in Johnson (2005). For example, duplicates (showing long strings of equivalent data) were removed. Also, repetitive patterns, which may indicate curiosity to see the items with no purpose of doing going through with the test, were removed. The missing data (1%) was finally corrected by imputing item means. Due to the present size of sample (N = 320,128), the item-correlations with and without the missing data imputed showed near perfect convergence (r = .998). The conclusion is that these commonly raised problems may not interfere with the purpose of the present study.

Instrument
IPIP-NEO-120 is a publically available representation of the FFM (Johnson, 2014), drawing 120 items from the International Personality Item Pool (IPIP; Goldberg et al., 2006). IPIP-NEO was built on open-source items correlating with the original NEO-PI-R (Costa & McCrae, 1995). IPIP-NEO was created seeking to optimize length, reliability, and validity in FFM measurement, and even surpassed the original in mean facet reliability (α > .80) (Johnson, 2014). The trait scales are built by four correlated items within each facet trait, while keeping bandwidth in meaning (i.e., removing repetitive items), as well as tending to balancing items (+ and -keyed items).
Each of the items are measured on a 1 (almost never) -5 (almost always) scale which then are summarized into facet traits (Min = 4, Max = 20). Six facet traits in turn are averaged into one trait factor. Overall, the mean reliability for facet scales in the present study was α = .78. Only four facets (13%) were below α = .70. See Table 2 for a summary of alphas.

Statistical Method
The analyses were based on the 30 facet traits in the FFM (IPIP-NEO-120). We first explored the unconstrained factor structure with facet correlations and exploratory factor analysis to see if facets organized into five proposed factors. The main objective was then finalized with testing hierarchical structural CFA models, one for each trait domain. The models were based on established FFM literature (Costa & McCrae, 1995), and tested the loadings between manifest items, latent trait facets, and respective latent trait domain. We utilized fit indices Note. Boldface italics indicates factor loading not aligning to expectation (i.e., higher loading on the wrong trait factor). 95% CI were less than ±0.01.
Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain 264 for two ways of modeling each FFM trait domain. The first way was the original and most used hierarchical second-order structure in two levels (Costa & McCrae, 1995). This is characterized by the general latent trait domain at the top, loaded by six facet traits, which in turn are loaded by 24 item measurement items. The second way of modeling was a bi-factor model, where both the general latent trait factor and specific latent facets are loaded onto directly by measurement items. The bi-factor approach is also helpful in revealing if any facet traits could be viewed as trait domains on their own-If items would not load on the general trait domain factor, but only on the facet trait, this could imply independence from the FFM domains. The CFA model was conducted in a SEM-framework, extracted with Maximum Likelihood, using AMOS v.23. The models were attempted as largely unconstrained, without covarying error terms. Due to the large sample size, all results were significant and standard errors were less than 0.01.  Note. The more intense blue color, the higher positive correlation, the more intense red color, the higher negative correlation. C1-C6 = Conscientiousness facets; E1-E6 = Extraversion facets; A1-A6 = Agreeableness facets; N1-N6 = Neuroticism facets; O1-O6 = Openness facets. Descriptions of facet abbreviations are found in Table 1. Kajonius & Johnson 265 Overall, the contrasting intensity of blue along the diagonal generally shows a high internal coherence within the five trait domains. Equally, the rows and columns of the Neuroticism factor (facets n1-n6) show intensity of red colors, indicating overall negative correlations with the other four trait domains. Apart from these patterns, most facets showed weak or no color with cross-facet traits, indicating fair, but far from perfect, discriminant qualities.

Exploring Five Factors in the IPIP-NEO
Before investigating the main objective of testing the five-factor structure as measured by IPIP-NEO, we checked whether Exploratory Factor Analysis (EFA) would confirm five factors. Utilizing Maximum Likelihood (ML) for extraction, and oblique rotation, assuming correlations between facets, the summary of loadings (structural pattern) is reported in Table 2. Only five factors had Eigenvalues above 1, which was an initial confirmation of the FFM IPIP-NEO structure. The goodness of fit was χ 2 (295) = 38,877, RMSEA = 0.07, and TLI = 0.84. The first factor extracted 21%, the second 11%, the third 8%, the fourth 6%, and the fifth 5% of total variance, making the total 51%. Overall, the facet traits revealed five clear trait structures (Table 2). However, there were also 4 facet discrepancies, as marked in italics in Table 2. Facet Trust (A1) loaded slightly more on Extraversion; Assertiveness (E3) and Activity (E4) organized mostly under Conscientiousness; and Emotionality (O3) under Neuroticism.

The Structure of the Five Factors in the IPIP-NEO
The main objective was to test the structures of the FFM trait domains in the public IPIP-NEO-120 instrument.
All five CFA trait structure models, one for each domain, were conducted in two ways: Second-order models; See Figure 2a, and bi-factor models; See Figure 2b). All parameters were unconstrained, and conducted without modifications (e.g., covariances between errors). Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain 266 Figure 2. a) The structure of the FFM second-order models. b) The structure of the FFM bi-factor models.
Note. FFM = Five Factor Model. Table 4 accounts for the bi-factor models (based on Figure 2), one for each FFM domain. The first observation here was that the bi-factor models, which control for a general trait factor, overall showed better model fits than second-order models. This is however expected since fit indices such as RMSEA reward parsimony (i.e., adding constraints). Nevertheless, CFI were also superior, and the difference was substantial for two of the trait structures: Extraversion (.90 compared to .87) and Agreeableness (.91 compared to .86). A second result was that ECV average was .47 on facet level and .50 on trait factor level. These results can overall be interpreted as unidimensionality in the components of the IPIP-NEO structure, which support a more nuanced facet structures as hypothesized. Along this line of reasoning, however, some trait structures had facets that seemed disconnected from the trait domain. For instance, the trait factor Agreeableness and facet Modesty showed a disconnection. Items "I think I am better than others" and "I think highly of myself" did not load on the general trait domain Agreeableness (β = .05 and β = .09), but highly on the facet trait (β = .88 and β = .90). (Recall that facet trait Modesty did not load satisfactorily on Agreeableness in the second-order model). Also, Self-discipline (in trait domain Conscientiousness) and Friendliness (in Extraversion) items even loaded negatively on respective facets, while loading strongly on respective general trait factors. This could be interpreted as these facets being identical with the general trait factor. i These mentioned examples may represent the core of respective general factors but may at the same time be too similar to optimally function as independent facet traits.
A last and final complexity was that all the trait domain models may be confounded by common variance bias, such as social desirability or acquiescence responding. This could potentially confound fit indices, as well as Kajonius & Johnson 267 loadings (See the extended discussion in Anusic, Schimmack, Pinkus, & Lockwood, 2009). Consequently, we also attempted to estimate to what extent a common method factor may be present. We created a so called bogus marker variable, which was made up by four uncorrelated fake items, which we added to the bi-factor model of the trait structure Agreeableness, chosen for its face appeal and social desirability. The average loadings from this fictive variable on the common factor, even though small, were not trivial (β = .18). With no com-  TLI = Tucker Lewis Index; CFI = Comparative Fit Index. 95% CI were less than ±0.01. a A standardized beta equal to or above 1.0 can legitimately occur, and indicates high multicollinearity (Deegan, 1978).
Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain 268 mon method variance present, this bogus variable should have been closer to zero. This result indicated that almost 4% of the variance in Agreeableness may be attributed to common method factors due to the respondent source.  Figure 2b)  , without improvement modifications (e.g., covariations between errors or between facets). a Did not converge in a first run with all parameters free, returning negative error variance or Heywood cases, which was amended by relaxing two constraints.

Discussion
We were able to confirm the established five factor structure in the public-domain version of the FFM, IPIP-NEO-120, in one of the largest US public samples to date. The inter-correlation matrix (Figure 1), as well as the exploratory factor analysis (Table 2) reported five clearly recognizable factor patterns. The main objective of testing hierarchical second-order models and bi-factor CFA models (Figures 2a and b) overall reported tolerable fit indices, and mostly sufficient structural loadings (Tables 3 and 4). However, these were at times not overly impressive. The results may be interpreted in a more optimistic light, seeing the often notoriously low model fit indices and weak model structures in complex personality models (Hopwood & Donnellan, 2010). It was clear that the five trait factors were supported by a substructure made up of facet traits, based on the ECVvalues (≈ .50; see Table 4). The ECV indicated that substantial common variance is present in the IPIP-NEOitems, assuming trait unidimensionality and thus supporting a more nuanced facet structure. Only trait factor Openness had a relatively low ECV of .43 and is the one factor in the IPIP-NEO that is more loosely structured, being composed of items constituting various facets such as Imagination, Liberalism, and Intellect (see Table   1). The IPIP-NEO-120 results encourage careful future use of the five trait domains and 30 facet traits in the public domain. This use is presently ongoing, such as being used in subclinical contexts (Kajonius, 2017), or nation-wide comparisons (Kajonius & Mac Giolla, 2017;Mac Giolla & Kajonius, 2018).
In line with previous research, the present study also identified some overall concerns regarding trait domain validity in the IPIP-NEO-120. Despite the FFM being an empirically impressive model, many critics have pointed out apparent limitations, such as lack of robust scope and meaning (see McCrae, 2010). One measurement objective is that facet traits should help define the broader trait domains, not confuse these. There may be both independent facet traits (e.g., Modesty) as well as perhaps domain-convergent facet traits (e.g., Self-discipline and Friendliness) in each of the FFM trait domains, as the present study indicates. The IPIP-NEO has undoubtedly inherited some problems from the original NEO-PI-R model (cf. Costa & McCrae, 1992;Furnham, Guenole, Levine, & Chamorro-Premuzic, 2013). One example in the present study is that facet traits Openness to Imagination, Emotionality, and Liberalism are weakly (β < .50) related to the general Openness domain. Another example is the Activity and Assertiveness facets in Extraversion. In the IPIP-NEO, Openness seems to be more characterized by artistic (esthetic) interests and intellectual endeavors, rather than emotions and politics (see facet loadings in Table 3), and Extraversion seems better characterized by social energy and positive temperaments, than being busy and assertive (which tended to sort under Conscientiousness, when unconstrained in EFA; see Table 1).
Moreover, one specific finding worth highlighting is that the facet trait Modesty was almost entirely disconnected from trait domain Agreeableness, and may surprisingly be the one facet currently not functioning sufficiently.
One explanation could be that Modesty should indeed be viewed as an independent trait apart from Agreeableness. Modesty may be more linked to a proposed sixth factor in an expanded FFM paradigm, Honesty-Humility (cf. Kajonius & Dåderman, 2014). Another explanation may be that Modesty contains a greater degree of social desirability than other facet traits (Allik et al., 2010). Not many would score an item such as "I think I am better than others" highly, and this may not provide enough variance to support a general Agreeableness trait. This anomaly may again fuel the debate concerning a sixth factor in the FFM, suggesting the addition of Honesty-Humility (e.g., DeYoung, Quilty, & Peterson, 2007). Such a factor may capture additional normative values, beyond Agreeableness. Either way, reporting the contents and structural validity of a trait domain is a crucial key factor for understanding what is being measured.
Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain 270 The other side of the coin is that a few IPIP-NEO facet traits also showed, not weak or disorganized loadings, The complexity and hierarchy of psychological personality traits may ultimately never be captured in a simple structural model (Hopwood & Donnellan, 2010). Nevertheless, the present IPIP-NEO seems to be reasonably robust compromise. The take-home message from the present study is the demonstrated structure of lower order FFM facet traits, which provides some of the needed scope and precision for future needs of measurements of individual differences.
Notes i) Note that the second-order models reported convergence between facet Self-discipline and trait Conscientiousness (β = .99) as well as between facet Friendliness and trait Extraversion (β = .96).

Funding
The authors have no funding to report.

About the Authors
Petri Kajonius is an active research within the fields of personality and social psychology, and an associate professor in work psychology at University West, Sweden. He is also an acknowledged speaker and communicator of science to the broad public.