The development and validation of the major life changing decision profile (MLCDP)

Background Chronic diseases may influence patients taking major life changing decisions (MLCDs) concerning for example education, career, relationships, having children and retirement. A validated measure is needed to evaluate the impact of chronic diseases on MLCDs, improving assessment of their life-long burden. The aims of this study were to develop a validated questionnaire, the “Major Life Changing Decision Profile” (MLCDP) and to evaluate its psychometric properties. Methods 50 interviews with dermatology patients and 258 questionnaires, completed by cardiology, rheumatology, nephrology, diabetes and respiratory disorder patients, were analysed for qualitative data using Nvivo8 software. Content validation was carried out by a panel of experts. The first version of the MLCDP was completed by 210 patients and an iterative process of multiple Exploratory Factor Analyses and item prevalence was used to guide item reduction. Face validity and practicability was assessed by patients. Results 48 MLCDs were selected from analysis of the transcripts and questionnaires for the first version of the MLCDP, and reduced to 45 by combination of similar themes. There was a high intraclass correlation coefficient (0.7) between the 13 members of the content validation panel. Four more items were deleted leaving a 41-item MLCDP that was completed by 210 patients. The most frequently recorded MLCDs were decisions to change eating habits (71.4%), to change smoking/drinking alcohol habits (58.5%) and not to travel or go for holidays abroad (50.9%). Factor analysis suggested item number reduction from 41 to 34, to 29, then 23 items. However after taking into account item prevalence data as well as factor analysis results, 32 items were retained. The 32-item MLCDP has five domains education (3 items), job/career (9), family/relationships (5), social (10) and physical (5). The MLCDP score is expressed as the absolute number of decisions that have been affected. Conclusions The 32-item (5 domains) MLCDP has been developed as an easy to complete generic tool for use in clinical practice and for quality of life and epidemiological research. Further validation is required.


Background
The focus of health-related quality of life (HRQoL) research has been on the assessment of current disease impact on patients' lives. However disease may have an additional major long-term impact by influencing major life changing decisions (MLCDs) [1,2], such as those concerning education, career, relationships or having children [3][4][5][6][7].
There is little information available about the impact of chronic diseases on MLCDs and there has been no defined measure to capture this information [1]. There are many published surveys assessing the impacts of chronic diseases on patients' HRQoL. However, these studies have not addressed the long term impact of chronic disease on critical life decisions taken by patients [1]. Instruments to measure HRQoL usually assess patient's current experiences and are not designed to assess the long term impact of disease, which may change over time. Even follow up studies, which might be expected to encompass more long term issues, usually compare current impacts before and after. In our review of HRQoL instruments we could not identify any instrument containing items to capture the influence of chronic disease on MLCDs or any long term impact on patients' lives [1].
A method to measure this impact is needed to draw attention to the issues, to identify specific patient needs, to allow a more complete assessment of the burden of disease and to permit comparison between the impacts of different diseases; with timely identification of the needs of patients, appropriate intervention may enable patients with chronic conditions to fulfil their full potential and aspirations. The aim of this study was to develop a standardised tool for use across all chronic conditions to record the influence of chronic disease on MLCDs and initially validate such a measure.

Methods
The South East Wales Research Ethics Committee approved this cross-sectional prospective study on 2nd June 2008. All participants gave informed written consent.

Study participants
Patients (aged 16 years and above) who had had chronic disease for > 1 year were recruited from the outpatient departments of dermatology, cardiology, rheumatology, nephrology, diabetes and respiratory disorders at the University Hospital of Wales, Cardiff and University Hospital Llandough. Patients were excluded if suffering from other illness or disability.

Data processing and analysis
Data was processed using SPSS16 statistical software for Windows. Descriptive statistics were used to record the judges' responses (%) on a four point scale. Statistically, Kappa coefficient and intraclass correlation coefficient (ICC) techniques were used to evaluate the agreement among the panel of judges for inter-rater reliability. For non-quantitative data (categorical/nominal), the kappa coefficient is the technique of choice. The ICC is more appropriate and better than the kappa technique for analysing data obtained from ratings using any scale (e.g. from 1 to 10) [8,9].
The level of agreement among the panel members for each item of the MLCDP was assessed statistically by using ICC for the quantitative and kappa technique for the qualitative phase to measure the level of agreement (inter-rater reliability) among the panel of judges. The SPSS software generally provides kappa statistics for only two judges. The Cohen Kappa macro syntax file was computed using SPSS12 software to calculate the multiple inter-rater agreement (>2 raters).

Procedure
There were four stages; conceptualisation and item generation, item reduction and development of the first version of a new tool, content validity and development of a revised version and construct validation using factor analysis. Using a formal check list [10], we conceptualised the development of a self-administered general health profile with multi-dichotomous close-ended simple statements.
Data for item generation were collected from a postal survey and individual interviews. Core themes and categories were identified using content analysis [3][4][5][6][7], and a generic MLCD questionnaire was developed, the "Major Life Changing Decisions Profile" (MLCDP). NVivo 8 and SPSS16 software were used.
The items generated were analysed using standard qualitative strategies [10][11][12] to reduce them to the core items identified from the interview transcripts. Some population and gender specific items were retained. After qualitative analysis, taxonomy techniques and rephrasing of items/ statements were introduced to fit the items into categories. Standard methods [10] to assess and modify language, reading age and item lengths were applied.
The 45-item MLCDP (version 1) was content validated by a team of experts (panel of judges). Their task was to review the profile, rate each item and to suggest appropriate changes to develop a profile that could accurately measure what it was intended to measure, with appropriate focus and emphasis for the target population. Consultant physicians, specialist nurses and academic experts from different disciplines were invited to join the panel of judges and to take part in this process. The content validation process was carried out in two phases, quantitative assessment and qualitative assessment.
For quantitative assessment, the MLCDP (version 1) and the questionnaire items rating sheets were sent out to the panel of judges to rate each item and for their expert opinion. The members of the panel were asked to bring their items rating sheets for discussion at the subsequent meeting. Instruction was provided for rating of the MLCDP (version 1) on a four point Likert type ordinal scale (1 = Strongly agree, 2 = Agree, 3 = Disagree and 4 = Strongly disagree) for each of the four criteria of language clarity, completeness, scaling and relevance. Under each item, a separate section was provided for the judges to write any suggestions.
In the qualitative assessment stage, all 13 panel members returned their completed item rating sheets and 6 panel members attended the discussion. The discussion was digitally recorded for later analysis. During the meeting, patients' comments given at the conceptualisation stage were consulted again where necessary to clarify the discussion. Various changes were made with consensus to the questionnaire (item inclusion, retraction, modification, and sequence) to reduce ambiguity and to make the MLCDP simpler, more user friendly and easier to understand.
This revised MLCDP was administered to another cohort of subjects for further construct validation using the Exploratory Factor Analysis technique of "principle component analysis" [13]. A correlation and component matrix was created for the assessment of data suitability followed by application of the Kaiser-Meyer-Olkin (KMO) measure and Bartlett's test of sphericity for adequate sampling. For factor extraction, Kaiser's criterion and Cattell's scree test were carried out and the varimax technique was used for factor rotation and final interpretation. Cronbach's alpha statistics were used to measure internal consistency [14].
Face validity and practicality of the new tool was assessed by using a separate simple tick box questionnaire to record patients' views on the new tool and by measuring time required to complete the MLCDP.

Item/Statement generation
The survey responses and interview transcriptions were cross-referenced during the decision-making process of using the themes for construction generation of each item. Items were kept short and phrased in the past tense. The core MLCD domains and statements were arranged sequentially to reflect life stages.
Two non-MLCD "life style" issues "to take part in other sports activities" and "to give up driving" were grouped under the "physical" domain. The items related to "community activities" and to "wear a wig/toupee" were included in the "social" domain. The gender specific MLCD "wearing make up" was addressed separately.

Item reduction
Initially, 48 items/statements were generated. Statements considered inappropriate, ambiguous or not universal were deleted. Several items were rephrased or combined. Items mentioned by less than 5% of respondents were not included.
The theme "not to mention illness on job application" was removed as this was considered an inappropriate action. The theme "not to seek employment" was removed because of similarity with another theme. One independent main category "to quit from professional sports" was excluded as it was reported by <5% of the patients. However some gender and speciality specific categories were retained, such as "major treatment decisions" and "to wear make up". The statement, "To leave a professional association/committee" was retained because of its generic nature. One life style MLCD related to "swimming" was put under the "social" rather than the "physical" domain: it was only reported by patients who were suffering from skin conditions, but this item may also be of relevance to patients with physical conditions. Another item "I decided not to take part in other sports activities" was added to the "physical" domain.
In the designing of the MLCDP we decided that we wished to measure the total impact of all of a patient's health conditions rather than focus on the effect of one specific disease. The reason for this is that if a patient has more than one condition affecting MLCDs, then it is highly likely that both conditions would influence in different additive ways these decisions and it would often be artificial to ask a patient to try to specify which MLCD was influenced by a specific disease.

Psychometric evaluation: content validity
The 45-item MLCDP was presented to a panel of experts for content validation [10,15]. The 13 panel members included clinicians from seven participating medical specialities: seven consultant physicians (5 males, 2 females) and six specialist nurses (all female). Eleven panel members were independent from the research team. Each panel member rated each of the 45 items on a 4-point scale (strongly agree to strongly disagree) for each of the four criteria, language clarity, completeness, scaling and relevance and suggested changes where they identified issues with any of these criteria. For each criterion, the percentage of panel members that strongly agreed, agreed, disagreed and strongly disagreed with that criterion having been fulfilled was calculated ( Table 2).
The median values of the panel members' ratings indicated good agreement between them in all four criteria ( Table 2). The items that did not gain agreement across all four criteria were "I decided not to take promotion", "I decided to stay in the same employment", "I decided to leave a professional association/committee" and "I decided not to go swimming". The following items failed on three criteria: "I decided to change my choice of career", "I decided to give up a job", "I decided to have IVF (In Vitro Fertilisation) treatment", and "I decided to change my eating habits".
The Intraclass Correlation Coefficient (ICC) analysis of absolute agreement showed an ICC of 0.707 (p = <0.0001; CI = 0.61 to 0.78) [8] indicating good inter-rater reliability among the panel and supporting the content validation.
Four clinicians (2 males, 2 females) and two academic QoL experts (both male), took part in the subsequent qualitative panel discussion. Each item was discussed until consensus was reached on their retraction, addition or modifications. The panel members' responses had a Cohen's Kappa [8] value for multiple raters (n = 6) of 0.81, (p = <0.0001, CI = 0.69 to 0.93). This value indicates "almost perfect agreement" [16] among the panel members on the 45 items, further supporting the content validity of the MLCDP.
Changes were implemented to produce the 41-item MLCDP. The six MLCD domains were reduced to five: education (4 items), job/career (10 items), family/relationships (7 items), social (15 items) and physical (5 items). This revised version was then examined by factor analysis.  (Tables 3 and 4) completed the 41-item MLCDP. The MLCDP was printed on thick blue paper to increase the response rate [17]. All 210 patients answered all of the items.
The internal consistency of the 41-item MLCDP measured by Cronbach's alpha was 0.84, indicating good reliability [18]. Seven items had corrected total-item correlations of <0.2 and were removed, increasing the Cronbach's alpha of the 34-item MLCDP to 0.85 (Appendix 1).
Exploratory factor analysis of the 34-item MLCDP was used to determine its construct validity [13] and to reduce the number of items if necessary [18,19] (Tables 5  and 6, Figure 1). Items that failed to load on any component or had a weak loading were removed, reducing the item number to 29 (Appendix 1).
Factor analysis was carried out on this 29-item MLCDP to determine whether the remaining items fitted well under the appropriate domains (Table 7, Figure 2). This suggested that a further 6 items could be deleted, reducing the MLCDP to 23 items (Appendix 1).
In order to ensure that all perspectives were considered in the decisions relating to retention and deletion For each criterion the percentage is given of the 13 panel members agreeing with that criterion being fulfilled.
of items, factor analysis of the full original 41-item MLCDP was also carried out. This analysis suggested that 11 items be deleted: these were compared with the 18 items suggested for deletion by the factor analysis of the 32-item MLCDP ( Table 8). Nine of these were the same, supporting the strength of the initial analysis (Appendix 1).

Scale refinement
The prevalence of deleted items reported in the qualitative phase of the study was examined to assess the appropriateness of reinstating any removed items and to assist item merger decisions. Final decisions on item deletion, rephrasing or regrouping were made on the basis of the factor analyses, on item prevalence and on statistical, conceptual and philosophical grounds. An item ranking table was used to compare the lowest ranked items with the items deleted as a result of factor analyses. If an item did not conceptually belong to a specific component, this also informed the decision whether or not to retain, remove or move that item to a more relevant component.
The level of disease influence on MLCDs was evaluated according to the percentage of patients who reported this influence. Items were ranked according to their prevalence (%) (   a result of the factor analysis of the 41-item MLCDP (with internal consistency reliability testing), 10 had a prevalence of >10%. Of the 11 items suggested for deletion as a result of factor analysis of the 41-item MLCDP (without internal consistency reliability testing), 6 had a prevalence of >10%: three of these items were first, second and fourth in the prevalence ranking (Table 10). This information was considered in the final selection of the MLCDP items.

Synthesis of the factor analysis findings
A strategy of item deletion, merger and application of appropriate phraseology was adopted for scale refinement. All 41 items were reviewed, with more attention given to those items with a prevalence of <10% and which were also suggested by one of the factor analyses to be removed (Table 10). On the basis of both factor analyses and examination of items of low and of high prevalence (items ranking), 30 items were discussed in detail. Seventeen items were retained, of which three were rephrased. Nine items were merged to create four new items and four items were completely removed. Following the scale refinement process the MLCDP items increased from 29 to 32 items. Figures 3, 4 and 5 shows the final draft of the MLCDP. The MLCDP score is expressed as the absolute number of decisions that have been affected, not as a percentage of the total number of items.

Scoring issues
We initially considered the scoring strategy from the perspective that if a MLCD had been influenced slightly, moderately, strongly or very strongly, then different scores should be placed on the answer to that question. The scores could then be added to the various questions to give an overall score that would reflect the totality of the strength of influence of disease on scores.
As we gained more experience with the MLCDP, our approach to the philosophy of scoring has changed. Our current view is that what matters to a patient is whether or not a particular major decision has or has not been influenced by disease. Even if a decision has been influenced only slightly, it has been influenced and so the course of that patient's life in relation to that decision has been affected by their disease. We have realised that under the graded scoring system originally proposed, if a single decision were influenced "very strongly", this would score, say, 4, the same score as having four decisions influenced "slightly". In reality the effect of having four decisions influenced slightly by a disease would be likely to have a much more widespread impact than having only one decision affected by a very strong influence.
Our current view is that the total number of decisions affected is the most critical information in comparing the overall impact of having a disease on MLCDs. However we currently retain the graded system of responses (without scoring them) in the MLCDP because we think that this graded information may be of value to a clinician or other carer when counselling a patient concerning this subject.

Face validity and practicality
When the data was collected for factor analysis, all 210 patients also answered a further 6 questions to assess the face validity and practicality of the 41-item MLCDP. The mean time taken by the patients to complete the MLCDP was 5.7 minutes (range = 2-15 minutes, median = 5 minutes). 204 (97.1%) patients reported that the MLCDP was easy to complete. 198 (94.3%) patients felt that "the response options for the statements in the questionnaire" were straight forward. 131 (62.4%) patients reported that the instructions and statements were "very clear" and 76 (36.2%) patients reported them as being "clear". 181 (86%) patients considered that the MLCDP was sufficiently comprehensive to measure the influence of disease on important life decisions. No new questions were suggested.

Discussion
We describe above the development and initial psychometric properties of a new tool designed to measure the novel domain of the impact of disease on a patient's MLCDs. The MLCDP was directly based on the experiences of patients. The MLCDP is the first instrument specifically developed to measure the influence of chronic diseases on MLCDs. A strict methodology [10] was followed at all developmental stages. The MLCDP is a generic questionnaire designed to be used across all medical specialities. An adequate number of subjects (n = 210) from seven medical specialities took part in the evaluation of its initial psychometric properties. A thorough process of multiple factor analyses was carried out. The MLCDP is easy to understand and can be completed in less than six minutes.
At the initial development stage a maximum possible number of items (48) were generated from patients' responses and grouped into six broad MLCD domains to make sure no aspect of MLCDs was overlooked. In the content validation of the MLCDP both quantitative and qualitative assessment techniques were applied [15].
"Exploratory factor analysis is a complex procedure, exacerbated by the lack of inferential statistics and the imperfections of "real world data" [20]. The researchers' judgment concerning the deletion and retention of specific items is important and should be based on the patients' responses and the research concept [20]. Indeed, the purpose of factor analysis was to see to what degree the mathematical approach confirms clinical intuition. It was therefore decided to keep in conceptually sound items at this stage of the instrument development, even though they may have been suggested by the factor analyses to be removed. It seemed appropriate to preserve those items for when other techniques, such as Rasch analysis and item response theory, could be applied to the data for further scale refinement.
At content validation stage we discussed with the panel of judges whether it was appropriate to include MLCDs that were confined to one particular gender or disease. This panel had decided to exclude any MLCD category that was reported by <5% of the patients. However the panel retained some gender and speciality specific categories to allow them to be tested for appropriateness and relevance at a later stage. The "make up" item came lowest in the item prevalence ranking and was also suggested by both factor analyses to be removed. But a closer consideration of this item and of related comments from patients at the scale refinement stage gave us further insight into the importance of this MLCD. The panel discussed in great detail the conundrum: does it really make sense to remove an item, if it genuinely represents from the patient's perspective an important MLCD? It was decided to retain the "make up" item at this stage and to postpone the decision as to whether or not to retain this item until after Rasch analysis and Item Response Theory are applied to MLCDP data for further scale refinement.
All the items came directly from patients and this patient origin had the paramount influence in our decisions concerning the development of the scale. Another reason to maintain items was to create a sense of concordance with the respondent by ensuring that items would be perceived as relevant. All items were subjected to a "factor extraction" process. Factor analysis should ideally be based on 300 cases, however 150 cases should be sufficient [18] and so the sample size of 210 was considered adequate.
When deciding scoring methods and their interpretation fundamental questions were raised such as what would be the perceived "value" of a patient reporting only one MLCD influenced by their chronic condition, if viewed within the perspective of a score system based on a maximum score of 32, should such a scoring system be devised. Does the MLCDP conceptually require any scoring system? What scoring system would assist meaningful interpretation of the data? Two patients reported that their disease had influenced >10 MLCDs, indicating an extraordinary level of long-term impact on those patients' lives. When we consider a patient in whom only one MLCD had been influenced, in the context of the 32 items identified, it might appear that the patient has experienced minimal impact on their life, but even one decision recorded as being influenced was a major decision which was life changing. There is virtually no possibility that all 32 MLCDs would be affected in any one patient and <1% patients reported more than 10 MLCDs affected. The profile is therefore more appropriately scored as an absolute value rather than as a percentage. Therefore the fundamental unit for expressing data from the MLCDP is the "total number of MLCDs that have been affected". Further psychometric analysis will determine whether for the final scoring model a single overall score, or a series of sub-domain scores, is most appropriate. It remains to be determined whether individual item weighting should be applied.
The MLCDP is not designed to detect changes over time or to be used in follow up studies, such as before and after intervention. It may help to identify which diseases have the biggest impacts on patients' lives; a total score based on adjectival data would be difficult to interpret in this context. However, the adjectival scoring was used for carrying out factor analysis as part of the development (item reduction) of the MLCDP in order to confirm grouping of the items based on clinical intuition through mathematical modelling. Therefore, the guidance provided for interpretation of the scores from the adjectival system has taken into account the reality of how the impact of a chronic disease on MLCDs should be managed.
Factor analysis indicated that some conceptually important items, which were high in the items prevalence ranking, should be deleted. However we decided that there should be a conservative approach to deleting items at the scale refinement stage and that decisions should be based primarily on patients' experiences, as shared during the detailed interviews and postal survey responses. It was reassuring that nine items were suggested for removal by two different factor analyses but it could be argued that all 41 of the original items are valid items, as they were all suggested by at least one actual patient and therefore reflect patients' reality. The tension between the outcome of factor analysis and patients' qualitative reports of their experiences is a common phenomenon in instrumental development. It is left to the research team to resolve this tension through careful review of both sources.

Uses of the MLCDP
The MLCDP may be helpful to identify which diseases have the biggest impact on patients' lives in terms of their MLCDs and which disease impacts most on

Conclusions
This study provides a method to evaluate a critical aspect of the long-term impact of chronic disease on patients' lives, the influence on MLCDs, allowing the use of the "MLCD" domain in the assessment of long-term disease impact and the overall assessment of HRQoL. The MLCDP data could be used to inform health, education and employment policies underpinned by a patient-centred approach. The use of the MLCDP could help with educating health providers about long-term consequences of chronic conditions, preparing them to help patients at an early disease stage and offer a longterm management plan. The use of the MLCDP for epidemiological surveys could identify which chronic conditions have the most influence on which type of MLCDs.

Limitations and further work
Further psychometric testing is required. As some items were deleted, merged or rephrased, further factor analysis and internal consistency reliability of the remaining items is required, followed by internal consistency reliability and test-retest reliability of the final remaining items. Full psychometric evaluation of the MLCDP should be carried out using both the classical test theory approach and novel techniques, such as item response theory. The MLCDP was not designed to detect change, but to be used to identify the number of major life decisions that a patient considers have been affected by their disease. We would therefore not expect there to be differences in the "behaviour" of the instrument according to different duration of disease, but this will need to be tested.
The current profile is not designed to detect change over time or to be used in follow up studies, such as before and after drug intervention. Further work is required to create a tool similar to the MLCDP that would be sensitive to clinical change. The issues surrounding MLCDs are very complex and highly subjective. There is therefore a need to study a control group to see whether there is a difference in approach between people who have or who do not have chronic disease and to confirm our assumption that MLCDs are part of the normal course of life. The MLCDP is an easy to complete generic tool. Although it requires further validation, it provides a means to measure this major impact of disease on patients' lives.

Ethical approval
The study was approved by the South East Wales Local Research Ethics committee (2nd June 2008). All participants consented to take part before starting the study.  For each statement please tick one box Figure 5 The 32-items MLCDP Part 3.
In order to ensure that all perspectives were considered in the decisions relating to retention and deletion of items, factor analysis of the full original 41-item MLCDP was also carried out. Items were deleted after the final varimax rotation based on the same criteria used for the previous analysis. The final rotated component matrix revealed that four items failed to load on any component, five items loaded weakly (<0.4) and two items loaded on two components with little difference between their values. These 11 deleted items were compared with the 18 items deleted as a result of the factor analysis of the 32-item MLCDP ( Table 8). Nine of the items were the same, which supported the initial factor analysis approach.