The Research Behind CISCA
Posted on: 05/06/2020
CISCA Validity Reliability and Performance Analysis | May 2020
Following the creation of CISCA, a series of analyses were run to assess the reliability and validity of the tool to ensure it accurately and empirically measures care capability.
CISCA was administered to a sample of 339 staff and 354 consumers and family members across 14 teams of retirement living and aged care providers in Western Australia. Following data cleansing and initial testing an initial exploratory factor analysis was conducted on the data using a maximum likelihood extraction with direct oblimin rotation. Further regression analyses were conducted to further assess the tool’s validity.
The Exploratory Factor Analysis revealed the CISCA survey accurately and reliably measures 5 care capability areas that also model the same 5 leadership capability areas from CILCA as hypothesized.
Caring with purpose had the lowest reliability and while it’s still acceptable (above 0.7) we have written three new questions to be included to improve the quality of this factor. We will conduct further reliability assessments on future samples to ensure these three questions achieve the goal of improving the overall reliability of Caring with Purpose.
The Regression Analysis found significant causational relationships between CISCA performance and organizational data as follows:
Team performance on CISCA can explain employee workplace culture scores by 27%
Team performance on CISCA can explain employee job satisfaction by 23%
Comparing site comparison data, we found Engagement correlated with the consumer experience, measured by Net Promoter Score and Customer Satisfaction. Note: this result is only based on a sample of 5 sites and caution must be applied when making conclusions about these relationships until we have a larger sample.
The results indicate that the comprehensive CISCA tool accurately measures some key factors contributing to overall performance including engagement, culture and front line care capability.
It also provides evidence that investing in improving Care Capability is likely to improve staff experience which is also likely to improve the consumer experience and overall care outcomes.
Further research into this will be conducted with a bigger data set in the future.
Learn more about our CISCA Assessment here