- Research article
- Open Access
Psychometric properties of the Czech Integrated Palliative Outcome Scale: reliability and content validity analysis
BMC Palliative Care volume 19, Article number: 39 (2020)
Outcome measurement is an essential part of the evaluation of palliative care and the measurements need to be reliable, valid and adapted to the culture in which they are used. The Integrated Palliative Outcome Scale (IPOS) is a widely used tool for assessing personal-level outcomes in palliative care. The aim of this study was to provide Czech version of IPOS and assess its psychometric properties.
Patients receiving palliative care in hospice or hospitals completed the IPOS. The reliability of Czech IPOS was tested with Cronbach alpha (for internal consistency), the intraclass correlation coefficient for total IPOS score and weighted Kappa (for test-retest reliability of individual items). Factor analysis was used for elucidating the construct (Exploratory Factor Analysis). Convergent validity was tested with correlation analysis (Spearman correlation) in a part of the sample, who completed also the Edmonton Symptom Assessment System (ESAS) and the Palliative Performance Scale (PPS).
The sample consisted of 140 patients (mean age 72; 90 women; 81% oncological disease). The Cronbach alpha was 0.789; intraclass correlation was 0.88. The correlations of IPOS with ESAS was R = 0.4 and PPS R = − 0.2. Exploratory factor analysis revealed a 2-factor solution on our data. The first factor covers emotional and information needs and the second factor covers physical symptoms.
Czech IPOS has very good reliability regarding both internal consistency and test-retest reliability. Together with an item analysis results, we can conclude that the Czech adaptation of the tool was successful. The convergent validity needs to be assessed on the larger sample and the proposed 2-factor internal structure of the questionnaire has to be confirmed by using CFA.
The main goal of palliative care is to improve the quality of life of patients suffering from life-threatening illnesses and their families. Therefore, quality-of-life measurements are important for the evaluation of palliative care interventions and the needs of patients or quantifying the change in health status . A wide variety of measurements currently exists and they differ in the number of measured domains, number of items, mode of administration (questionnaire/interview, patient/proxy) and also in the level of validity and reliability . The Palliative Outcome Scale (POS) is one of the tools for comprehensive measurement of the patients´ main symptoms and concerns . POS is widely used in clinical care, audit, research, and training and it was validated in several languages [4, 5]. The POS measures have been used in different patients populations such as patients with cancer, respiratory, heart, renal or liver failure, and neurological diseases [6,7,8,9,10]. POS-S was developed as an addition to POS to be used as a brief tool specifically focused on physical symptoms . There are also specific variations of POS for dementia or renal failure patients, (POS S-Renal, POS S-Multiple Sclerosis, POS S-Parkinson Disease) . IPOS is the youngest instrument from the POS family which merges questions from POS and POS-S as it was requested from clinicians . IPOS consists of 10 questions which cover main symptoms, patient and family distress, well-being, sharing feelings with family, practical concerns and information needs .
IPOS was found to have excellent reliability [12,13,14,15,16] and face and content validity was also confirmed in several studies using cognitive interviews [11, 17, 18] Convergent validity has been confirmed for the original and German IPOS , Japanese version of IPOS  and French IPOS . In many other countries the process of validation is ongoing and all language version which are currently available, such as Portuguese, Polish, Greek etc., can be found online (www.pos-pal.org).This study aims to provide a valid version of IPOS in Czech and to report the psychometric properties of IPOS from this first pilot Czech study. During the standardization, we followed the manual created by authors of POS .
This was a mixed-method multicenter study conducted in 6 organizations in the Czech Republic (1 home hospice care, 2 hospices facilities and 3 hospitals). Data were obtained by trained clinical staff - nurses or social workers during the inpatient admission or home visit. The inclusion criteria were: being patient of hospice or home hospice care or palliative care team/unit in the hospital and able to give consent to participate. We excluded patients who had cognitive impairment (judged by the clinical team) and who did not understand the Czech language. Patients completed IPOS and a demographic questionnaire on their own or with help from the staff member. When appropriate, patients were asked to complete IPOS twice for testing of reliability. The second measurement was done when it was possible and feasible from the clinical point of view, predominantly during the next appointment. The instructions were to do it after minimum of 3 days.
IPOS consists of 10 questions with 17 items. Question 1 is about the main concerns and has open-ended options. Q2 addresses specific symptoms and there is also a place for adding any additional symptoms (Q2a-c). Q3-Q6 ask about psychological, spiritual, communication and practical concerns but Q6–8 address positive aspects and the direction of possible answers is opposite. Q10 is not scored and asks patients whether they filled IPOS with any help or by themselves. All questions except Q1 have a numerical scale from 0 to 4 and only one response is allowed for each question. The sum score can range from 0 to 68 and is computed from all items except Q1 and Q2a-c.
The Czech version was created clarifying conceptual definition equivalents in Czech followed by forward and backward translation which was done by independent translators as required by the Manual for the cross-cultural adaptation of the POS . The initial Czech version of IPOS was piloted through cognitive interviews with 5 patients and 5 health care providers from hospice and the face validity of the Czech IPOS was confirmed. The final Czech version of IPOS can be found in Additional file 1.
Part of the sample completed the Edmonton Symptom Assessment System or the Palliative Performance Scale for testing the construct validity of IPOS. Only those data collection sites which use ESAS and PPS as part of routine care were asked to provide both data. The Edmonton Symptom Assessment System (ESAS) is another questionnaire assessing the key patients´ symptoms and concerns and is commonly used in Czech hospices. ESAS consists of 10 items measuring physical symptoms and well-being and patients are asked to rate the symptoms severity from 0 to 10 on a numerical scale .
Palliative Performance Scale (PPS) is a tool for measuring performance status of patients in palliative care and it is usually recorded by nurses or by physicians with good inter-rater agreement . It was developed from the Karnofsky Performance Scale . It is oriented on physical functions and activities and can be used for prognostication and planning care . Patients’ performance is scored by percentage in 11 categories from fully ambulatory and healthy (100%) to death (0%). The ratings are based on observation of 5 categories: ambulation, level of activity and evidence of disease, ability to self-care, food/fluid intake and state of consciousness .
The Ethical Committee of the General University Hospital in Prague approved the study (Protocol Number 51/18 S-IV) and all participants gave written informed consent.
Internal consistency of the IPOS total score was investigated by using Cronbach ‘s alfa. Item difficulty was calculated using item mean and converted to interval < 0;1 > using formula mean-scale min/(scale max-scale min). Part of the sample (13%) completed the IPOS in two different times for confirmation of temporal stability (T1 and T2) with an average range of 15.6 days between the measures (SD = 9.0). Test-retest reliability of the IPOS total score was evaluated for the part of the sample (N = 14, see Table 1) using the intraclass correlation coefficient (ICC). An ICC range of 0.4–0.7 was considered moderate and > 0.75 was considered to represent high test-retest reliability . For each of 17 IPOS items, we also computed four metrics of test-retest reliability: level of agreement, level of agreement within one score, quadratic weighted kappa and Spearman correlation. A range of kappa from 0.41 to 0.60 was considered as moderate, 0.61–0.80 as substantial, and 0.81–1 as almost perfect [25, 26].
To test the influence of gender, place of care and age, we used parametric methods (t-test and Pearson correlation coefficient respectively) based on a sufficiently large sample and normal distribution of overall IPOS score.
Moreover, we used factor analysis to explore the possible dimensions of the Czech IPOS questionnaire and to elucidate the constructs. We applied Exploratory factor analysis (EFA) using principal axis factoring as the extraction method and Varimax rotations. The number of factors to be extracted derived from the combination of Kaiser’s criterion and Cattell’s scree plot method.
The Spearman correlations between the IPOS score and two other measures commonly used in palliative care (ESAS and PPS) were assessed to report preliminary results of convergent validity. We expected mid-range correlation between total IPOS score and ESAS total score and PPS (0.5–0.7) because these methods do not cover spiritual, practical and family issues similarly like Murtagh and her colleagues . The non-parametric method was chosen due to quite small sample sizes.
All missing values were excluded from the analysis. A significant p-value was set at 0.05. All analyses were conducted within SPSS v. 25.0 (IBM Corp., Armonk, NY, USA).
From November 2017 until August 2018, we collected IPOS data from 144 patients. However, 4 patients had to be excluded from the final sample because they did not complete full IPOS. Most of them were inpatients, only in 16% of patients the place of care was at home provided by the home hospice. The number of patients from the hospital and hospice were similar (43% vs 57%). In the sample, there were few more women (64%) and most of the patients suffered from oncological disease (81%). The detailed description of the sample is in Table 1. Most of the patients (88.6%) needed help in the completion of IPOS.
Table 2 presents descriptive statistics of all 17 IPOS items for the whole sample. We used the short names in the description of items, similarly as Sakurai et al.  and Sandham et al.  [14, 15]. As a part of the item analysis, we evaluated each item’s difficulty and correlation with the total IPOS score (item-total correlation). The minimum item difficulty was 0.13 (Vomiting), the maximum was 0.6 (Poor mobility). All item-total correlations were higher than 0.3, the highest predictor of the total score was item measuring Weakness with item-total correlation 0.66.
Influence of gender, age and place of care
The total IPOS score did not differ for men and women (t = − 1.537, p = 0.127) nor did it correlate with the age of patients (r = 0.141, p = 0.096). However, we found a significant difference in the total IPOS score when comparing patients from hospices and patients from hospitals (t = − 3.613, p < 0.001). More specifically, the average total IPOS score of patients from hospices was lower (38.75, SD = 9.11) than the average score of patients from hospitals (44.28, SD = 8.77).
Cronbach’s alpha for 17 IPOS items (which are used for calculation of the overall score) was 0.789. Temporal stability was evaluated for all items separately as well as for the overall score. A one-way intra-class correlation coefficient of IPOS total score indicated a high level of temporal stability (ICC = 0.88, 95% CI: 0.56–0.94). Sufficient test-retest reliability was also supported by significant Spearman correlation between two total IPOS scores in T1 and T2 (r = 0.88, p < 0.05). For most of the items significant Spearman correlations were found as well as fair to good levels of weighted kappa, however, several items showed rather low temporal stability, mainly items called Family anxiety, Practical problems, Drowsiness or Anxiety. For more detailed results, please see Table 3.
Exploratory factor analysis
Both Kaiser-Meyer-Olkin Measure of Sampling Adequacy (0.696) and Bartlett’s test of sphericity (p < 0.001) indicated that a factor analysis might be useful with our data. Based on the combination of Kaiser’s criterion and Cattell’s scree plot method, we decided to present the two-factor model (Table 4) as an output of EFA which explains 29.1% of the variance (Factor 1: 15.9%, Factor 2: 13.3%) and the factors showed a correlation of 0.316.
Spearman’s correlation of the sum score of IPOS and PPS was found to be weaker than was expected by our hypotheses and non-significant (Rs(40) = −0.249; p = 0.121), correlation with ESAS showed to be on a moderate level (Rs(14) = 0.414; p = 0.141), however, not significant due to a very small research sample. Data from PPS and ESAS were not available from many patients so these results have to be considered preliminary only.
This study aimed to provide a valid version of the Czech IPOS and to report the psychometric properties of IPOS. Item analysis results showed that the Czech adaptation of the tool was successful. This study showed also that the Czech IPOS has very good reliability regarding internal consistency and we preliminary assessed the validity of the Czech IPOS and temporal stability.
Items analysis showed that all of the items in IPOS meet the requirements for item difficulty and item-total correlation. The lowest discriminant ability was found in item Vomiting because 75% of patients did not report this symptom. This is not consistent with previous results . However, in Sandham et al. study only hospice patients were assessed which might have caused the difference . Another study with patients from hospitals and home-based palliative services found similar results when Vomiting, Practical matters and Having enough information did not have full range of responses .
Regarding influence of place, age or gender, in our sample, we found significant differences in the total IPOS score according to the place of care which was also confirmed in other countries for POS [27, 28]. This might be explained by the fact that patients in hospices are usually in the terminal stage of disease with well-controlled symptoms as the median of the length of stay in Czech home hospices is around 10 days . IPOS total score did not differ according to age or gender which is consistent with other studies .
The reliability of IPOS was measured in two ways with Cronbach alpha and test-rest reliability. The Cronbach alpha showed a high internal consistency of the Czech version of IPOS which is consistent with other studies [12, 13, 15]. IPOS was completed twice by 14 patients and test-retest reliability was confirmed by a sufficient intraclass-correlation coefficient. Some items showed low temporal stability, mainly items called Family anxiety, Practical problems, Drowsiness or Anxiety (0.02–0.33) which is not consistent with Japanese validation where items with the lowest temporal stability (0.522–0.622) were Share Feelings, Information and Practical Problems, for others items ICC was higher than 0.7 . This study is missing independent global change rating which would confirm stability of patients´ health condition. Condition of patients in palliative care is fast-changing which makes the interpretation of our results more difficult. The low temporal stability of these items in Czech IPOS might be also explained by the fact that time between measurement was longer than in previous studies and varied (M = 15.6, SD = 9). In other studies retest was conducted the next day [14, 30]. Therefore, we need to confirm the retest reliability for Czech IPOS in a shorter period. On the other hand, the second measurement should be done later than the next day to avoid bias that respondents may recall their previous responses . These results show that Practical Problems is an item on which we should focus our attention because it is unstable, and it can change even within 1 day.
The results of factor analysis showed the two-factor model could be applied to our data. The first factor consists of items associated with psychological concerns (Anxiety, Depression, Information etc.) and the second factor is composed of items assessing physical symptoms. Only the item Shortness of breath cannot be easily assigned to one of these factor groups because the loadings reached the low and almost equal level. Sandham and her colleagues identified unidimensionality in IPOS measuring palliative care needs of patients . Even though our data showed the possibility of applying the two-factor model for Czech IPOS, there is a significant correlation between both factors (R = 0.316). In our study, we were limited by the size of the overall sample not sufficient to apply Confirmatory factor analysis (CFA). Murtagh and her colleagues identified three factors in IPOS using CFA – Physical Symptoms, Emotional Symptoms and Communication/Practical Issues . This suggests that subscales could differ according to socio-cultural context or that we need more data for testing our two-factor model and the three-factor model using CFA and to compare which of these models is more precise for our population.
In terms of convergent validity, the overall score was correlated with PPS which is a tool measuring physical status  and the correlation was weaker than expected because this tool is only focused on physical symptoms. For correlation with ESAS, we found a moderate correlation which was not significant because of the small number of patients who completed IPOS and ESAS. Correlation with ESAS was also confirmed in other study . Sakurai and his colleagues also confirmed validity of IPOS using other instruments (EORTC QLQ-30, FACIT-Sp12, and STAS) and found strong to moderate correlations, except for the item Information . One possible explanation is that this item is rather unique as the only similar question from STAS is answered by a clinician . Correlation of APCA African POS and MVQoLI were found to be weak to moderate for which the explanation might be that different measures of quality of life use different conceptualizations of this term .
This study has several limitations. We found moderate but not significant correlation of IPOS and ESAS which means that we cannot confirm convergent of validity of Czech IPOS due to small sample who completed IPOS and ESAS. These results only imply trend which was confirmed in other studies. Due to logistical demand on participating staff it was not possible to get ESAS from every patient in the sample. Only those data collection sites which use ESAS and PPS provided both data. We also could not conduct confirmatory factor analysis on this data due to insufficient sample size. The interval of retest should be shorter with a low level of variability or instead of short time period we should use external criterion to judge stability of patients´ condition. The number of patients who completed the second measurement in this study was very low, therefore, more data for more precise retest reliability results are needed.
This study confirmed that the Czech version of IPOS might be used in the clinical setting and the cultural adaptation was successful. This study also further proved that IPOS is a reliable method for assessing the quality of life of patients in palliative care.
Availability of data and materials
The datasets used during the current study are available from the corresponding author on reasonable request.
Palliative outcome scale
Integrated palliative outcome scale
Edmonton symptom assessment system
Palliative performance scale
Intraclass correlation coefficient
Exploratory factor analysis
Confirmatory factor analysis
African palliative outcome scale
Missoula-Vitas quality of life index
- EORTC QLQ-30:
European organisation for research and treatment of cancer quality of life questionnaire
Functional assessment of chronic illness therapy – spiritual well-being
Support team assessment schedule
Addington-Hall J, Bruera E, Higginson IJ, Payne S. Research methods in palliative care. Oxford: Oxford University Press; 2007.
Albers G, Echteld MA, de Vet HC, Onwuteaka-Philipsen BD, van der Linden MH, Deliens L. Evaluation of quality-of-life measures for use in palliative care: a systematic review. Palliat Med. 2010;24(1):17–37.
Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Qual Saf Health Care. 1999;8(4):219–27.
Bausewein C, Le Grice C, Simon S, Higginson I. The use of two common palliative outcome measures in clinical care and research: a systematic review of POS and STAS. Palliat Med. 2011;25(4):304–13.
Collins ES, Witt J, Bausewein C, Daveson BA, Higginson IJ, Murtagh FEM. A systematic review of the use of the palliative care outcome scale and the support team assessment schedule in palliative care. J Pain Symptom Manag. 2015;50(6):842–853.e19.
Bausewein C, Booth S, Higginson IJ. Measurement of dyspnoea in the clinical rather than the research setting. Curr Opin Support Palliat Care. 2008;2(2):95–9.
Horton R. Differences in assessment of symptoms and quality of life between patients with advanced cancer and their specialist palliative care nurses in a home care setting. Palliat Med. 2002;16(6):488–94.
Kane PM, Daveson BA, Ryan K, Ellis-Smith CI, Mahon NG, McAdam B, et al. Feasibility and acceptability of a patient-reported outcome intervention in chronic heart failure. BMJ Support Palliat Care. 2017;7(4):470–9.
Raj R, Ahuja K, Frandsen M, Murtagh FEM, Jose M. Validation of the IPOS-renal symptom survey in advanced kidney disease: a cross-sectional study. J Pain Symptom Manag. 2018;56(2):281–7.
Saleem TZ, Higginson IJ, Chaudhuri KR, Martin A, Burman R, Leigh PN. Symptom prevalence, severity and palliative care needs assessment using the palliative outcome scale: a cross-sectional study of patients with Parkinson’s disease and related neurological conditions. Palliat Med. 2013;27(8):722–31.
Schildmann EK, Groeneveld EI, Denzel J, Brown A, Bernhardt F, Bailey K, et al. Discovering the hidden benefits of cognitive interviewing in two languages: the first phase of a validation study of the integrated palliative care outcome scale. Palliat Med. 2016;30(6):599–610.
Antunes B, Rodrigues PP, Higginson IJ, Ferreira PL. Validation and cultural adaptation of the integrated palliative care outcome scale (IPOS) for the Portuguese population. In: EAPC 2017 15th Wordl congress of the European Association for Palliative Care. Hayward: Newmarket; 2017. p. 697. Retrieved from: https://www.eapc-2017.org/files/EAPC17/dl/EJPC-Abstract-Book-2017.pdf.
Murtagh FE, Ramsenthaler C, Firth A, Groeneveld EI, Lovell N, Simon ST, et al. A brief, patient- and proxy-reported outcome measure in advanced illness: validity, reliability and responsiveness of the integrated palliative care outcome scale (IPOS). Palliat Med. 2019;33(8):1045–57.
Sakurai H, Miyashita M, Imai K, Miyamoto S, Otani H, Oishi A, et al. Validation of the integrated palliative care outcome scale (IPOS) – Japanese version. Jpn J Clin Oncol. 2019;49(3):257–62.
Sandham MH, Medvedev ON, Hedgecock E, Higginson IJ, Siegert RJ. A Rasch analysis of the integrated palliative care outcome scale. J Pain Symptom Manag. 2019;57(2):290–6.
Sterie A-C, Borasio GD, Bernard M. Validation of the French version of the integrated palliative care outcome scale. J Pain Symptom Manag. 2019;58(5):886–890.e5.
Beck I, Olsson Möller U, Malmström M, Klarare A, Samuelsson H, Lundh Hagelin C, et al. Translation and cultural adaptation of the integrated palliative care outcome scale including cognitive interviewing with patients and staff. BMC Palliat Care. 2017;16(1):49.
Veronese S, Rabitti E, Costantini M, Valle A, Higginson I. Translation and cognitive testing of the Italian integrated palliative outcome scale (IPOS) among patients and healthcare professionals. Frey R, editor. PLoS One. 2019;14(1):e0208536.
Antunes B, Brown A, Witt J, Daveson BA, Ramsenthaler C, Benalia H, et al. Manual for crosscultural adaptation and psychometric validation of the POS; 2019. Retrieved from: https://pos-pal.org/maix/resources.php.
Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton symptom assessment system (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care. 1991;7(2):6–9.
Zimmermann C, Burman D, Bandukwala S, Seccareccia D, Kaya E, Bryson J, et al. Nurse and physician inter-rater agreement of three performance status measures in palliative care outpatients. Support Care Cancer. 2010;18(5):609–16.
Anderson F, Downing GM, Hill J, Casorso L, Lerch N. Palliative performance scale (PPS): a new tool. J Palliat Care. 1996;12(1):5–11.
Baik D, Russell D, Jordan L, Dooley F, Bowles KH, Masterson Creber RM. Using the palliative performance scale to estimate survival for patients at the end of life: a systematic review of the literature. J Palliat Med. 2018;21(11):1651–61.
Koo TK, Li MY. A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159.
Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360–3.
Lerzynski GA, Allan A, Murray SA. Die Bewertung der palliativmedizinischen Patientenversorgung mithilfe der palliative care outcome scale (POS) in verschiedenen Versorgungsformen: die Anwendung eines palliativmedizinischen Messinstruments bei Krebspatienten im St. Columba’s Hospiz, Edinburgh, und bei Patienten mit Lungenkrebs oder Herzinsuffizienz in häuslicher Versorgung. Z Für Palliativmedizin. 2004;5(1):19–27.
Pidgeon T, Johnson CE, Currow D, Yates P, Banfield M, Lester L, et al. A survey of patients’ experience of pain and other symptoms while receiving care from palliative care services. BMJ Support Palliat Care. 2016;6(3):315–22.
Cesta domů. Výroční zpráva za rok 2018Cesta domů, z.ú. 2019. Retrieved from: https://www.cestadomu.cz/aktuality/vyrocni-zprava-2018.
Harding R, Selman L, Agupio G, Dinat N, Downing J, Gwyther L, et al. Validation of a core outcome measure for palliative care in Africa: the APCA African palliative outcome scale. Health Qual Life Outcomes. 2010;8(1):10.
We thank all women and men who took part in this study. We also thank the staff from hospice “Cesta domů”, Hospice of the Good Shepherd and Hospice of the St. Stephan and the staff from hospitals in Jihlava, General University Hospital in Prague and The Sisters of Mercy of St. Charles Borromeo who gave time helping recruit the participants and complete IPOS with them.
This study was supported by grant no. 17-26722Y Czech Science Foundation. The funder had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of data; the preparation, review, and approval of the manuscript; or the decision to submit the manuscript for publication.
Ethics approval and consent to participate
The Ethical Committee of the General University Hospital in Prague approved the study (Protocol Number 51/18 S-IV) and all participants gave written informed consent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Vlckova, K., Hoschlova, E., Chroustova, E. et al. Psychometric properties of the Czech Integrated Palliative Outcome Scale: reliability and content validity analysis. BMC Palliat Care 19, 39 (2020). https://doi.org/10.1186/s12904-020-00552-x
- Outcome measurement
- Patient-reported outcome measure
- Palliative care
- Symptom assessment