The breakdown of the response rate at each time point is presented in Table 1.
Questionnaires were completed at two time points, matched by cohorts, but not by person.
Table 1
Response rates across three countries and time points. (T1 = Time 1, T2 = Time 2)
Time point
|
Location
|
Pre- (T1)
Frequency/Percentage
|
Post- (T2)
Frequency/Percentage
|
|
England
|
133 (48.9%)
|
132 (49.8)
|
|
Scotland
|
98 (36.0%)
|
94 (35.5%)
|
|
Northern Ireland
|
41 (15.1%)
|
39 (14.7%)
|
Exploratory Factor Analysis
AWES
Initial analysis indicates the appropriateness for factor analysis of the items (KMO = 0.844, Barlett test for sphericity chi square, 2061.15, df = 210, p = 0.01). Exploratory factor analysis of the items show that a five-factor model emerged from the data. These include Care delivery (8 items); Feeling valued (4 items); Workload (3 items); Staff (3 items) and Role of manager (3 items). Overall model explained 52.49% of the variance: Factor 1 = 16.15%; factor 2 = 12.98%; factor 3 = 9.29%; factor 4 = 8.69%; factor 5 = 5.38%. Examination of the measures of homogeneity show that the factors were stable: Care delivery (time 1 = 0.85, time 2 = 0.85); Feeling valued (time 1 = 0.85, time 2 = 0.74); Workload (time 1 = 0.74, time 2 = 0.60); Staff (time 1 = 0.74, time 2 = 0.60) and Role of manager (time 1 = 0.56, time 2 = 0.18). It also included 14 additional questions that examined the workplace culture and context.
POWCS
Initial analysis indicates the appropriateness for factor analysis of the items (KMO = 0.948, Barlett test for sphericity chi square, 6138, df = 190, p = 0.01). Exploratory factor analysis identified an emergent four factor model that includes Impact on Manager (6 items, time 1 = 0.88; time 2 = 0.52); Impact on service user (6 item time ); Leadership (5 items) and Stress (4 items). Overall model explained 62% of the variance: Impact on Manager = 47.38%; Impact on service user = 7.69%; Leadership = 3.91%; Stress = 3.16%; Examination of the measures of homogeneity show that the factors were stable: Impact on Manager (time 1 = 0.88; time 2 = 0.52); Impact on service user (time 1 = 0.85; time 2 = 0.57); Leadership (time 1 = 0.83; time 2 = 0.79) and Stress (time 1 = 0.64; time 2 = 0.70). The questionnaire included 8 additional items that measured the work environment.
Perception of Workplace Change Schedule (POWCS)
The items of the questionnaire were examined across both time points. Examination of skewness and kurtosis showed no deviation from normality of distribution and the data was suitable for analysis using parametric tests. Examination of table 2 show that there were statistically significant changes across time points on 26 of the 28 items. For example, job satisfaction and satisfaction with working conditions and enthusiasm to work in the care sector changed positively and significantly. There were small, positive but statistically significant changes across the items measuring the workplace culture. Items with the largest changes included the amount of time staff spend with residents, managers’ own quality of life, and the morale of staff.
The results were categorised by factors identified by the exploratory factor analysis. Also included are additional statements which although not aligned with factor analysis provide additional insight.
Table 2
Perceived impact of LSP on various factors (managers, service users, leadership, stress, additional comments), according to findings from POWCS
During the last 12 months:
|
Pre-Post
Mean
|
SD
|
Skewness
|
Kurtosis
|
Factor 1: Impact on Manager
|
2.90
2.42
|
0.78
0.88
|
-0.12
-0.50
|
-0.59
-0.21
|
The sense of personal achievement I get from work has
|
2.87
2.11**
|
1.22
1.26
|
0.08
0.86
|
-0.96
-0.42
|
My feeling of being valued has
|
3.04
2.48**
|
1.03
1.12
|
-0.63
0.37
|
-0.48
-0.43
|
My job satisfaction has
|
2.97
2.39**
|
1.10
1.16
|
0.05
0.46
|
-0.60
-0.60
|
My feelings of job security have
|
2.95
2.63**
|
0.93
0.99
|
-0.13
-0.17
|
0.45
-0.14
|
Satisfaction with my overall working conditions has
|
2.92
2.52**
|
0.87
1.03
|
− .15
0.33
|
0.42
-0.28
|
My enthusiasm for working in the care sector has
|
2.84
2.29**
|
0.95
1.08
|
-0.16
0.43
|
-0.16
-0.49
|
Factor 2: Impact on service users
|
2.51
1.98
|
0.54
0.60
|
-0.32
0.26
|
0.21
-0.25
|
The amount of time staff actively talk with relatives and service users has
|
2.75
2.09**
|
0.83
0.82
|
-0.08
0.27
|
0.37
-0.60
|
The quality of life of my service users has
|
2.27
1.87 **
|
0.69
0.74
|
-0.11
0.49
|
-0.45
0.14
|
Staff’s desire to take the initiative in responding to service user’s needs has
|
2.49
1.98 **
|
0.76
0.69
|
0.10
0.17
|
0.05
-0.41
|
Service user’s active involvement in decisions that have affected them has
|
2.37
1.94 **
|
0.69
0.70
|
-0.18
0.21
|
-0.38
-0.53
|
The quality of interaction between staff and service users has
|
2.55
1.91**
|
0.69
0.76
|
-0.17
0.32
|
0.27
-0.74
|
The quality of interaction between staff and relatives has
|
2.66
2.08**
|
0.69
0.77
|
-0.36
-0.03
|
0.77
-1.10
|
Factor 3: Leadership
|
2.51
1.80
|
0.64
0.62
|
0.22
0.72
|
-0.30
0.02
|
The quality of management and leadership I am able to offer has
|
2.60
1.85**
|
0.95
0.91
|
0.36
0.93
|
0.08
0.20
|
My understanding of how to improve the culture of care has
|
2.38
1.68**
|
0.76
0.75
|
0.02
0.88
|
-0.36
0.19
|
My staff’s ability to take initiative has
|
2.56
2.06**
|
0.71
0.71
|
0.01
0.48
|
0.15
0.41
|
My leadership & communication skills have
|
2.37
1.58**
|
0.77
0.67
|
0.08
0.90
|
0.18
0.27
|
My confidence as a professional has
|
2.65
1.84**
|
0.96
0.90
|
0.26
1.01
|
-0.19
0.95
|
Factor 4: Stress
|
2.33
3.08
|
0.82
0.94
|
0.66
-0.05
|
0.80
-0.48
|
The levels of stress I feel has
|
2.40
3.35**
|
1.10
1.23
|
0.70
-0.31
|
-0.11
-0.84
|
My workload has
|
1.75
2.39**
|
0.94
1.09
|
1.29
0.38
|
1.41
-0.48
|
My own quality of life has
|
3.17
2.51**
|
1.19
1.12
|
0.12
0.50
|
-0.59
-0.21
|
Additional Statements
|
|
|
|
|
The morale of my staff has
|
2.82
2.07**
|
1.15
0.95
|
0.36
0.81
|
-0.54
0.35
|
The quality of my engagement with my staff has
|
2.46
1.73**
|
0.98
0.79
|
0.40
0.80
|
-0.20
-0.10
|
Satisfaction with practice in the care setting has
|
2.58
2.01**
|
0.87
0.83
|
0.03
0.59
|
-0.36
0.09
|
My satisfaction with the relationship I have with my line manager/ owner has
|
2.75
2.49**
|
1.00
1.02
|
0.08
0.14
|
-0.36
-0.34
|
The overall level of quality of practice in this care setting has
|
2.21
1.88**
|
0.71
0.81
|
0.22
0.93
|
0.34
1.40
|
Staff sickness levels have
|
3.00
3.40**
|
0.95
0.97
|
-0.06
-0.14
|
-0.19
-0.10
|
Staff retention levels have
|
2.88
2.97
|
0.79
0.90
|
-0.03
0.22
|
0.71
0.35
|
inappropriate hospital admissions appear to have
|
3.65
3.76
|
0.88
0.89
|
0.15
-0.05
|
-0.87
-0.60
|
N.B. Time 2 scores are in bold print; * =Significant at p < 0.05; **=significant at p < 0.01 |
Note: In this survey a decrease in score is equal to a positive impact (5= decreased a lot, 1 = increased a lot)
|
Additional statements
Whilst the additional statements in table 2 did not align with specific factors, they all improved at a statistically significant level, (two items - staff retention and inappropriate admissions, changed positively but not with a statistically significant change. Notable improvements were reported in statements pertaining to staff morale, and staff sickness.
The scores for each factor at both time points across countries are show in Table 3. Overall scores across the four constructs changed positively and at a statistically significant level.
Table 3
Summary of perceived impact of LSP across countries (England, Scotland, Northern Ireland), according to findings from POWCS
|
Impact on Managers
|
Impact on Service users
|
Leadership
|
Stress
|
|
Pre-
|
Post-
|
Pre-
|
Post-
|
Pre-
|
Post-
|
Pre-
|
Post-
|
England
|
2.84 (.76)
|
2.64**
(.95)
|
2.52 (.55)
|
2.15** (.61)
|
2.52 (.63)
|
2.01** (.66)
|
2.23 (0.80)
|
2.88** (0.93)
|
Scotland
|
2.88 (.79)
|
2.18**
(.74)
|
2.48 (.55)
|
1.81** (.56)
|
2.45 (.65)
|
1.59** (.51)
|
2.43 (.77)
|
3.30** (.88)
|
N. Ireland
|
3.31 (.79)
|
2.25** (.77)
|
2.53 (.52)
|
1.80** (.53)
|
2.67 (0.69)
|
1.62**
(.53)
|
2.45 (1.01)
|
3.21** (1.01)
|
Overall
|
2.90 (.78)
|
2.42** (.88)
|
2.51 (.54)
|
1.98** (.60)
|
2.51 (.65)
|
1.80** (.63)
|
2.33 (0.82)
|
3.08** (.94)
|
These findings demonstrate a significant positive change across all four factors in each of the three geographical areas (Impact on Managers, Impact on Service users, Leadership, and Stress).
Impact on Managers – Definition: “feeling of achievement and satisfaction with working in the sector and job”.
On the construct ‘Impact on Managers’, a scores of 2.9 reflects a sense of things staying the same at pre-intervention and there were statistically significant differences in scoring between the three geographical areas at time 1 (f = 4.57, df 257,2, p = 0.01) from 3.31 to 2.84 (see Table 3). After the intervention, all scores decreased at a statistically significant level (t = 6.69, df 527, p < 0.01) but this change was strongest among participants in Northern Ireland and Scotland (Impact on Managers - t = 6.694, df 527, p = 0.01; See table 2). At time 2, there were statistically significant differences in scores between countries ranging between 2.64–2.18.
Whist significant changes were seen across the factors, the most significant changes were in the statements on participants’ feeling valued with a significant increase of 0.97 and enthusiasm for working in the care sector showing a significant increase of 0.93.
Impact on Service Users – Definition: “level of engagement and interaction in care between staff and service users/relatives”.
Construct scores of 2.51 reflects a sense of things staying the same at pre-intervention. There was no significant difference in scoring across geographical locations in the pre-intervention data. Examination of the scores show that overall participants scored this construct to have improved across time points at a statistically significant level for the total sample (t = 10.59, df 527, p = 0.01). This change in scoring was statistically significant across time between counties (f = 5.05, df 5,2, p = 0.01). All scores decreased across all three locations, but England decreased at a lower rate when compared to Scotland and Northern Ireland Impact on service users. Within this factor the most notable change was around the amount of time staff actively talk with relatives and service users and the quality of life for service users.
Leadership – Definition: “positive transformative leadership through effective communication”.
Construct scores of 2.51 reflects a sense of things staying the same at pre-intervention. There were no significant differences in leadership across countries at the start of the programme. However, a significant difference over time for total sample (t = 12.79, df 527, p = 0.01). There was significant difference in scoring across time between counties (f = 7.63, df 5,2, p = 0.01). The overall outcome was that all scores decreased across all three locations. The statement with the most change pre and post was staff ability to take initiative. In addition, participants increased confidence as a professional health professional was noted. Managers’ perception of how to improve the culture of care had marginally decreased in T2.
Stress – Definition: “stress, workload and its negative impact on quality of life”.
Construct scores of 2.33 reflects a sense of stress had been increasing at pre-intervention. There was no significant difference across countries at the start though a significant difference was noted over time for total sample (t= -9.74, df 527, p = 0.01) as stress levels decreased. There was a significant difference in scoring across time between counties (f = 0.94, df 5,2, p = 0.01). The participants own quality of life had the largest shift amongst all the questionnaire statements aligned to factors, stress and workload.
Assessment of Work Environments Schedule (AWES)
Examination of the scores for the items of AWES (Table 4) show small but statistically significant changes in 29 of the 36 items, mostly in a significant manner. These changes were noted on items within the five factors (see Table 4) such as a sense of feeling valued and a more manageable workload.
The largest change was in ‘I lack confidence in my role as a care professional’, this is a negatively framed statement which demonstrates that more participants disagreed with this at the second time point so a significant result.
The other items with large changes include
-
The quality of life of my service users is positive
-
Staff are provided with sufficient time to provide the type of care they need
-
There is a positive feeling of morale among my staff
-
I feel that staff prioritise the service user’s quality of life before the tasks of the day
Table 4
Perceived impact of LSP on various factors (care delivery, feeling valued, workload, staff, role as manager, additional comments), according to findings from AWES
Thinking about the place in which I work, I feel that
(1 = strongly disagree – 5 = strongly agree)
|
Mean
|
SD
|
Skewness
|
Kurtosis
|
Care delivery
|
|
|
|
|
The environment of care for service users is good
|
4.25
4.32
|
0.78
0.71
|
-0.72
-1.13
|
0.90
2.27
|
Staff play an active role in decision-making about resident care
|
3.76
4.08**
|
0.78
0.95
|
-0.23
-1.11
|
-0.30
0.82
|
The overall quality of care provided is high
|
4.25
3.92**
|
0.64
1.21
|
-0.61
-0.96
|
1.03
-0.14
|
I am very satisfied with the level of care practice that staff offer to service users
|
3.87
4.02**
|
0.71
0.93
|
-0.40
-0.84
|
0.26
0.11
|
The quality of life of my service users is positive
|
3.93
3.33**
|
0.67
1.42
|
-0.31
-0.29
|
0.29
-1.36
|
I am content with the quality of interaction that staff have with service users
|
3.37
3.92**
|
0.89
0.79
|
-0.33
-0.80
|
-0.84
0.88
|
I am content with the quality of interaction that staff have with relatives
|
3.35
3.64**
|
0.89
0.79
|
-0.33
-0.80
|
-0.84
0.88
|
The care setting feels like a positive community where service users, staff and relatives enjoy spending time with one another
|
3.76
4.45**
|
0.75
0.66
|
-0.41
-1.10
|
0.61
1.34
|
Feeling valued
|
|
|
|
|
I am congratulated when I do things well
|
3.26
3.86**
|
0.97
0.85
|
-0.18
-0.50
|
-0.53
-0.03
|
I am given respect by my superiors
|
3.83
4.34**
|
0.88
0.66
|
-0.69
-0.90
|
0.56
1.91
|
I feel valued for the work I do
|
3.44
3.92**
|
0.91
0.85
|
-0.50
-0.80
|
0.24
0.85
|
I have a positive relationship with my line manager/ owner
|
3.98
4.03
|
0.77
0.71
|
-0.38
-0.88
|
-0.17
1.37
|
Workload
|
|
|
|
|
The amount of work I am given to do is realistic
|
3.01
3.52**
|
1.02
1.03
|
-0.09
0.45
|
-0.68
-0.48
|
I am able to make sufficient time to support staff to deliver care to service users
|
3.38
3.90**
|
0.86
0.91
|
-0.42
-0.58
|
-0.44
-0.24
|
The amount of time I have to talk to relatives and service users is acceptable
|
3.22
3.28
|
0.94
1.11
|
-0.25
-0.15
|
-0.88
-1.05
|
Staff
|
|
|
|
|
I actively provide space and time to listen to the views of staff
|
3.95
4.54**
|
0.72
0.55
|
-0.42
-0.77
|
0.16
0.36
|
I actively listen to the opinions of my staff
|
4.12
4.47**
|
0.63
0.65
|
-0.28
-1.26
|
0.27
2.65
|
My staff are congratulated when they do things well
|
4.12
4.22
|
0.78
0.95
|
0.89
-1.06
|
1.40
0.34
|
Role as a manager
|
|
|
|
|
I feel that I have the management and leadership skills required to undertake an effective role
|
3.93
4.28**
|
0.65
0.67
|
-0.43
-0.79
|
0.72
1.04
|
I lack confidence in my role as a care professional
|
2.36
3.06**
|
0.91
1.35
|
0.72
-0.13
|
0.41
-1.34
|
I have a positive quality of life
|
3.72
4.39**
|
0.83
2.63
|
-0.57
14.24
|
0.09
220.60
|
Additional Items
|
|
|
|
|
There is a good spirit of cooperation between managers and staff
|
3.62
4.31**
|
0.71
0.69
|
-0.70
-0.84
|
0.84
0.79
|
There is a good spirit of cooperation between staff
|
3.65
4.22**
|
0.77
0.67
|
-0.70
-0.75
|
0.41
1.15
|
Staff can try new ideas without criticism
|
3.83
4.15**
|
0.74
0.87
|
-0.17
-1.07
|
-0.32
1.09
|
Staff are provided with sufficient time to provide the type of care they need
|
3.41
4.21**
|
0.90
0.78
|
-0.40
-0.93
|
-0.41
0.76
|
Staff are actively encouraged to develop their skills
|
4.08
4.58**
|
0.71
0.61
|
-0.74
-1.34
|
1.08
1.75
|
My responsibilities as a care professional are too great
|
3.36
3.27**
|
0.87
0.93
|
0.11
0.05
|
-0.50
-0.62
|
Staffing levels are adequate for the workload
|
3.50
4.08**
|
0.98
0.86
|
-0.55
-1.07
|
-0.30
0.88
|
I typically experience high levels of stress
|
3.64
3.52
|
0.93
0.95
|
-0.38
-0.18
|
-0.19
-0.90
|
There is a positive feeling of morale among my staff
|
3.36
4.16**
|
0.77
0.71
|
-0.38
-0.88
|
-0.17
1.37
|
I currently get a positive sense of personal achievement from my work
|
3.93
4.33**
|
0.76
0.68
|
-0.79
-1.11
|
1.35
2.51
|
My understanding of how to change the culture of care is limited
|
2.92
3.10
|
0.89
1.80
|
0.11
0.12
|
-0.44
-1.28
|
I feel that I have developed effective influencing skills
|
3.66
4.06**
|
0.77
0.79
|
-0.33
-0.95
|
-0.16
1.26
|
My general feeling at present is that:
|
|
|
|
|
Staff sickness levels are an on-going problem
|
3.50
3.02**
|
1.14
1.13
|
-038
0.04
|
-0.77
-0.99
|
Staff retention levels are an on-going problem
|
3.16
3.20
|
1.08
1.12
|
-0.03
-0.27
|
-0.93
-0.90
|
I feel that staff prioritise the service user’s quality of life before the tasks of the day
|
3.19
3.99**
|
0.95
0.78
|
-0.13
-0.83
|
-0.39
1.06
|
Table 5
Summary of perceived impact of LSP across countries (England, Scotland, Northern Ireland), according to findings from AWES
|
Care Delivery
|
Feeling valued
|
Workload
|
Staff
|
Role of a manager
|
|
Pre
|
Post
|
Pre
|
Post
|
Pre
|
Post
|
Pre
|
Post
|
Pre
|
Post
|
England
|
3.76
|
3.54
|
3.67
|
3.97
|
3.11
|
3.29
|
4.12
|
4.16
|
3.72
|
3.42
|
Scotland
|
3.75
|
4.34
|
3.62
|
4.11
|
3.27
|
3.83
|
4.01
|
4.65
|
3.81
|
4.23
|
N. Ireland
|
3.82
|
4.36
|
3.48
|
4.10
|
3.37
|
3.85
|
4.02
|
4.68
|
3.85
|
4.56
|
Overall
|
3.77
|
3.94
|
3.63
|
4.04
|
3.21
|
3.57
|
4.06
|
4.41
|
3.77
|
3.88
|
There were no significant differences according to the country on all four construct scores prior to the intervention, indicating that all regions experienced similar work environment issues, see Table 5.
Care Delivery – Definition: “the positive work culture promoting the provision of quality care & interaction for service”.
There was a small, statistically significant difference in scores for total sample across both time points (t=-3.52, df 533, p = 0.01). Scores increased from 3.77 to 3.94, (Table 5). Examination of the scores according to country across time show significant difference in scoring across countries across time (f = 10.56 df 5 2, p = 0.00). Similar patterns in scoring were reported across the factor ‘feeling valued’ ‘workload’ ‘staff’ and ‘role of a manager’. Within this factor the scores went down for two statements – ‘quality of care provided is high’ and ‘quality of life for service users’ though the latter was a small decrease of 0.8. However, a significant shift was noted in the care setting ‘feels like a positive community’ and that the participant is content with the ‘quality of interaction between staff & service users’.
Feeling valued – Definition: “appreciation & recognition from my line management”.
There was a statistically significant difference for total sample across time (t=-6.93, df 533, p = 0.01) and a significant difference in scoring across countries across time (f = 2.095 df 5 2, p = 0.00), (Table 5) All scores increased at a similar rate across time points across all three locations. A feeling of being valued and respected scored higher after the intervention though a small decrease was noted in the positive relationship with line mangers.
Workload – Definition: “time to support residents & staff”.
A Statistically Significant difference for total sample across time (t=-5.44, df 533, p = 0.01) was noted and a significant difference in scoring across countries across time (f = 3.84 df 5 2, p = 0.02), (Table 5). All scores increased across all three locations. Within this factor a largest shift was ‘the amount of work I am given to do is realistic’ similar with ‘I am able to make sufficient time to support staff to deliver care to service users’
Staff – Definition: “my impact and interaction with staff”.
Statistically Significant difference for total sample across time (t=-7.07, df 533, p = 0.01) was recorded and a significant difference in scoring across countries across time (f = 21.43 df 5 2, p = 0.00), (Table 5). The most notable change in this factor was for the statement ‘I actively provide space & time to listen to the views of my staff’
Role of a manager – Definition: “skills, confidence and my own quality of life”.
No statistically Significant difference for total sample across time (t=-1.39, df 530, p = 0.17) but a significant difference in scoring across countries across time (f = 17.11 df 5 2, p = 0.00).
Whilst an increase was recorded in the score for the statement about managers confidence this is a negatively worded statement and so demonstrates a reduction in confidence. Additionally, a significant increase was noted for leadership skills and managers having a positive quality of life.
Additional items
Additional statement items which did not align with factors are included in Table 4 and add additional insight to results of the intervention. Whist staff retention problems continue to be a concern as do appropriate staffing levels staff sickness levels are less so. Participants report increased confidence in personal achievement, changing the culture of care and effective influencing skills. Also stress levels are lesser following the intervention.