- Research article
- Open Access
Should analyses of large, national palliative care data sets with patient reported outcomes (PROs) be restricted to services with high patient participation? A register-based study
BMC Palliative Care volume 19, Article number: 89 (2020)
There is an increased interest in the analysis of large, national palliative care data sets including patient reported outcomes (PROs). No study has investigated if it was best to include or exclude data from services with low response rates in order to obtain the patient reported outcomes most representative of the national palliative care population. Thus, the aim of this study was to investigate whether services with low response rates should be excluded from analyses to prevent effects of possible selection bias.
Data from the Danish Palliative Care Database from 24,589 specialized palliative care admittances of cancer patients was included. Patients reported ten aspects of quality of life using the EORTC QLQ-C15-PAL-questionnaire. Multiple linear regression was performed to test if response rate was associated with the ten aspects of quality of life.
The score of six quality of life aspects were significantly associated with response rate. However, in only two cases patients from specialized palliative care services with lower response rates (< 20.0%, 20.0–29.9%, 30.0–39.9%, 40.0–49.9% or 50.0–59.9) were feeling better than patients from services with high response rates (≥60%) and in both cases it was less than 2 points on a 0–100 scale.
The study hypothesis, that patients from specialized palliative care services with lower response rates were reporting better quality of life than those from specialized palliative care services with high response rates, was not supported. This suggests that there is no reason to exclude data from specialized palliative care services with low response rates.
It has been shown that not all symptoms in advanced cancer patients admitted to palliative care are recognized by the health care professionals. Systematic symptom assessment has been proposed as a solution to this problem [1,2,3]. Therefore, starting in 2010, cancer patients in all specialized palliative care (SPC) services in Denmark have been invited to complete the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-Core-15-Palliative Care questionnaire (EORTC QLQ-C15-PAL) at the start of SPC. The questionnaire assesses different quality of life aspects of cancer patients in palliative care, i.e., nine symptoms and problems and overall QOL. There is increasing interest in the analysis of large, national data sets from palliative care with patient reported outcomes (PROs). Thus, we planned to analyze these patient reported QOL data to get a better understanding of cancer patients’ QOL at the start of SPC on a national level. However, the response rates varied to a large extent between the SPC services in Denmark. This led to a dilemma: we were concerned that the data from palliative services with low response rate might be affected by selection bias (e.g., if fewer of the most symptomatic patients were included, the resulting scores would not be representative). On the other hand, excluding some of the SPC services from the dataset, would lead to reduced generalizability of our study findings: we would not be reporting national data but data from ‘well-performing services’ only. And if excluding services, where should the cutoff for exclusion be?
Some studies on patients with advanced cancer and in palliative care have found that health affected study participation [4, 5]: lower performance score and more pain were associated with a lower probability of answering a questionnaire  and non-respondents had lower physical performance and shorter survival than respondents . However, a third study in patients with advanced cancer found no indication of clinically relevant differences in the quality of life scores when observed scores were compared to scores that included imputed data, and thus found no indication of bias due to non-participation . Thus, non-respondents would likely either be similar or in worse health compared to respondents.
Since patients admitted to SPC are severely ill and close to death, it is not feasible for all patients to report their symptoms/problems. However, if research is to give an understanding of different aspects of QOL (symptoms, problems and overall QOL, in the following generally referred to as QOL) at the start of SPC it is crucial to obtain QOL reporting from as representative a sample as possible. If better QOL-scores are reported in SPC services with low response rates it could be because only the most well patients (i.e. those who were easiest to obtain QOL reporting from) were asked to report their QOL. In this case, the QOL scores from SPC services with low response rates would probably be biased and thus overestimate QOL, and one could argue that a more correct estimate of QOL could be obtained if services with low response rates were excluded from future analyses.
Therefore, in this study, we wished to test the hypothesis: Patients from SPC services with low response rates report (on average) better on different QOL aspects than patients from services with high response rates, indicating a (larger) impact of selection bias in services with low response rates than in services with high response rates. Thus, although it is impossible to rule out selection bias even in services with quite high response rates, we assumed that differences in average scale scores between services with low and high response rates could reflect possible selection bias in the services with low response.
Thus, the aim of this study was to test whether response rate was associated with the level of different QOL aspects (i.e., with the level of the nine symptoms and problems and overall QOL from the EORTC QLQ-C15-PAL questionnaire).
If our study hypothesis was rejected and no indication of bias was found, there would not seem to be any reason to exclude data from the SPC services with low response rates in future studies.
Patients and data
The Danish palliative database
All SPC services in Denmark deliver data to the Danish Palliative Database (DPD) on patients referred to their service. DPD contains information on all patients referred to SPC in Denmark from 2010 and onwards. Patient information recorded in DPD includes diagnosis, socio-demographic factors, whether the patient has received SPC, and the symptom/problems and QOL among patients admitted to SPC. Data in this study was obtained from the DPD.
This study included data for patients who: 1) were admitted to SPC and died between January 1st, 2010 and December 31st, 2015, 2) had a cancer diagnosis, 3) were 18+ years of age and 4) answered the EORTC QLQ-C15-PAL questionnaire in the period from 3 days prior to admission to the day of admission to SPC.
The EORTC QLQ-C15-PAL questionnaire is a shortened version of the widely used EORTC QLQ-C30 developed for assessment of different aspects of quality of life in cancer patients in palliative care, i.e., assessment of nine symptoms and problems and overall QOL . EORTC QLQ-C15-PAL has four multi-item scales (physical functioning, emotional functioning, fatigue and pain) and six single-item scales (dyspnea, insomnia, appetite loss, constipation, nausea/vomiting and overall QOL) . The patients answer on a 4-point scale how much they have experienced the symptom/problem (not at all, a little, quite a bit, very much), except for overall QOL which is rated on a 7-point scale where 1 is very poor and 7 excellent. The time frame is the past week except for physical functioning for which no time-period is specified.
The analyses were performed using SAS statistical software version 9.4.
Conversion of scale scores and computation of response rate
The responses to the EORTC QLQ-C15-PAL were converted into 0–100 scales according to the scoring manual [8, 9]. For the two functional scales and QOL, higher scores represent better functioning/QOL, whereas for the seven symptom scales, higher scores represent worse symptoms [8, 9].
The response rate was computed for each SPC service for each calendar year. The response rate for a service a given year was computed as the number of patients admitted to the service that year who completed the questionnaire at admittance divided by all the patients who were admitted to the service the same year. A response rate was allocated to each patient in the study. Thus, a patient admitted to e.g., the Palliative Care Team in Århus in 2010 was allocated the response rate of Palliative Care Team in Århus in 2010. Response rate was grouped into; < 20.0%, 20.0–29.9%, 30.0–39.9%, 40.0–49.9%, 50.0–59.9% and ≥ 60.0%.
Multiple linear regression
Multiple linear regression analyses were performed to study the association between response rate and scale scores. The ≥60.0% response rate group was used as reference since it was expected to have the least selection bias.
The study hypothesis, i.e. that patients from SPC services with low response rates had better scale scores on the ten QOL aspects than patients from services with high response rates (indicating possible selection bias), was tested using four criteria:
The p-value for the overall association between response rate and scale score was < 0.05.
The p-value for at least one pairwise comparison of scale scores in lower response rates groups (< 20.0%, 20.0–29.9%, 30.0–39.9% 40.0–49.9%, 50.0–59.9%) with the ≥60.0% response rate group was < 0.05.
The difference in scale score was in the direction supporting our hypothesis (lower symptom score and higher functioning and QOL in the lower response rate groups compared to the ≥60.0% group), and
The mean difference in scale scores had to be 5 or more to be considered clinically relevant.
If one or more of the criteria were not fulfilled, the study hypothesis was rejected. This was tested for the scale scores of each of the ten QOL aspects.
The choice of 5 as a clinically relevant difference in scale scores was based on results and conclusions from previous studies [10,11,12,13,14]. In these studies, 10 is often used as the clinically relevant cut point . In this study, the more conservative cut point of 5 was chosen because it was important not to miss relevant differences.
Adjustment and random effects in the regression analyses
The patients in this study were from either hospices or palliative care teams. Previous studies have found more symptoms, worse performance status and shorter survival time in hospice patients compared to patients from other palliative care services [16,17,18,19]. We therefore controlled for type of SPC service (hospice/team) in the regression analyses. A random effect for SPC service was also included in the model because patients from the same SPC service were expected to be more similar than patients from different SPC services. Further, a random effect of patient id was included in the model to account for the fact that some patients filled in more than one questionnaire (if admitted to more than one service).
Choice of regression analysis method
The results of the association between response rate and scale scores from the linear regression analyses (mean differences in symptom score) are presented in this article due to their simple interpretation. Linear regression is, however, not the obvious choice for the scales with only four (dyspnea, sleep, appetite loss, constipation and nausea) or seven (pain, emotional functioning and QOL) possible scores because the normal distribution assumption is likely to be violated. Logistic regression (with symptom scores dichotomized at the median) with random effects was therefore performed as a sensitivity analysis to ensure that significant associations were not missed in the linear regression analyses.
Change in mean scale scores when excluding patients from SPC services with response rates < 60.0%
In addition to the regression analyses we compared the mean scale scores for the entire study sample with the mean scores obtained when patients from services with response rate < 20.0, < 30.0, < 40.0, < 50.0 and < 60.0%, respectively, were excluded. Higher symptom/lower functional scores in the reduced samples would support the hypothesis of selection bias.
The 40,316 adult Danish cancer patients in DPD who received specialized palliative care and died between 2010 and 2015 had 49,307 admittances to SPC services (some patients were admitted to more than one service). In 24,589 (49.9%) of these patient admittances, based on 22,420 patients, the EORTC QLQ-C15-PAL was answered. These 24,589 admittances were included in this study.
Of the 24,589 SPC admittances, 49.0% represented women and the average age was 68.5 years (Table 1). Three of four (74.0%) of the SPC admittances were in a palliative care team. The number of SPC admittances increased from 2010 to 2014 but decreased from 2014 to 2015 (patients who died later than 2015 were not included). The largest differences between patients responding to the EORTC QLQ-C15-PAL questionnaire (and thus included in the study) and non-respondents was that the study population were less likely to be hospice patients (26.0% vs. 47.9%) and their average survival time was longer (94.0 vs. 64.5 days) compared to non-respondents (Table 1).
The patients were from 44 services, and each service admitted between 2 and 591 patients each year. The overall response rate was 49.9% and varied between year and service from 0.0 to 93.2% (Table 2).
Most patients were categorized in the response rate group ≥60.0% but 42.3% were categorized in a lower group, i.e. were admitted to a service having a response rate below 60.0% in the year of admission (Table 1).
Are symptom scores biased in services with low response rates?
Criterion 1. Significant associations with response rate were found for six of the ten symptoms/problem scales (dyspnea, appetite loss, fatigue, nausea, emotional function and physical function).
Criterion 2. Mean scores for the six symptoms/problems were compared between the ≥60.0% response rate group and each of the five lower response rate groups (< 20.0%, 20.0–29.9%, 30.0–39.9%, 40.0–49.9% and 50.0–59.9%, respectively), i.e., in total 30 comparisons. In 13 of the 30 comparisons, a significant difference was found between the ≥60.0% response rate group and the lower response rate group.
Criterion 3. In two of these 13 comparisons, patients from the lower response rate groups (20.0–29.9% and 30.0–39.0%) had better scores (higher mean physical function) compared to the ≥60.0% response rate group, in accordance with the study hypothesis.
The remaining 11 of the 13 significant comparisons contradicted the study hypothesis as the patients from the lower response rate group had worse symptoms/problems. Thus, compared to the ≥60.0% response rate group, the 30.0–59.9% response rate groups had worse appetite loss, the 40.0–49.9% response rate group had worse fatigue, the < 20.0–49.9% response rate groups had worse nausea and the 20.0–59.9% response rate groups had lower emotional function.
Criterion 4. For physical function, the 20.0–29.9% and 30.0–39.0% response rate groups had 1.9 and 1.7 point higher mean physical function, respectively, compared to the ≥60.0% response rate group and these differences were therefore not clinically relevant. For the 11 significant comparisons that were not in accordance with the study hypothesis, only one was possibly clinically relevant (patients from the < 20.0% response rate group reported 5.7 point more nausea compared to patients from the ≥60.0% response rate group).
Choice of regression analysis method
The linear regression analyses found more significant associations between response rates and scale scores than the logistic regression analyses did, and thus the linear regression analyses did not generally miss significant associations, except for QOL but in that case the logistic regression analysis did not find systematically worse (or better) QOL in the lower response rate groups compared to the highest response rate group (Table 3).
Change in mean scale scores when patients from SPC services with response rates < 60.0% are excluded.
The mean scale scores were almost identical for the whole population and the sub-populations with exclusion of the < 20.0, < 30.0, < 40.0, < 50.0 and < 60.0% response rate groups, respectively. The largest change in mean score after exclusion of the lower response rate groups was 2.0 (in emotional function, range 0–100) (Fig. 3). Thus, removing patients from services with lower response rates from the mean scale score calculations had almost no effect on the mean scores.
Evidence supporting selection bias in SPC services with low response rates?
Our aim was to test whether the response rate was associated with scale scores of ten quality of life aspects. By doing this we could test the study hypothesis, i.e. that patients from SPC services with low response rates were reporting better QOL than patients from services with high response rates, indicating possible selection bias in services with low response rates. Such selection bias could happen if the health care professional in low response services had primarily asked the most well patients to answer the EORTC QLQ-C15-PAL questionnaire, leading to an overestimation of the QOL in the low response services which would result in better scores on the ten QOL aspects (lower symptom scores and higher function and overall QOL scores) in these low response services compared to high response services where patients were not selected (or less selected). On the other hand, if patients in low response services were randomly asked to answer the EORTC QLQ-C15-PAL questionnaire the QOL would be representative for the patients in the low response service. This would likely result in similar QOL scores in the low and high response services or perhaps worse scores in the low response services if the explanation to the lower response rate was a sicker patient population. Thus, to accept the study hypothesis, the score of one or more of the quality of life aspects should be significantly better (i.e., lower symptom score, higher function score, higher overall QOL score) in palliative care services with low response rates compared to patients from services with high response rates. However, results from the multiple linear regression analyses did not support the study hypothesis: there were overall associations between response rate and six out of ten QOL scale scores. However, within these six scales, only two out of 30 comparisons were significant and in the expected direction. The magnitude of these comparisons was very small (below 2 on a 0–100 scale) and thus not clinically relevant. Furthermore, mean scale scores in the whole population were almost identical to mean scale scores calculated after exclusion of the < 20.0, < 30.0, < 40.0, < 50.0 and < 60.0% response rate groups, respectively, supporting that the response rate had neglectable impact on the scale scores. This is in accordance with findings from a previous, much smaller study in patients with advanced cancer, where no evidence of clinically relevant difference in the quality of life scores due to non-participation was found .
There were some cases where patients from services with low response rates reported significantly worse QOL, and not better QOL as hypothesized. The differences were relatively small (1.8–5.7 on a 100-point scale) and due to the large number of statistical test performed, these findings may be due to chance. It is however not unlikely that non-response is associated with poor health [4, 5] and the slightly worse symptom/problem scores in SPC services with low response rates may reflect a slightly more ill patient population in SPC services with low response rates, i.e. not selection bias.
Strengths and weaknesses
There are several strengths in this study. First, it is based on a large dataset with national coverage including data from all SPC services in Denmark. Second, the possibility that different SPC services may not have the same composition of patients (due to socio-demographic or other differences across referral areas) was accounted for by including the random effect of specific SPC service.
No services had complete data, which is a limitation for at least two reasons. First, because our conclusion that there is not more selection bias in SPC services with low response rates than in those with ≥60.0% response rate is not a full investigation of the possibility of selection bias in SPC services with low response rates. However, due to the nature of SPC, which is provided to patients with very advanced disease and short survival, it will probably not be possible to achieve much higher response rates than achieved in the best performing services. Second, it is impossible to know whether differences in scale scores between SPC services with high and low response rates are in fact do to selection bias in SPC services with low response rates or whether they are caused by SPC services with low response rates actually having different scale scores, since we do not know the true scale scores. Thus, looking at difference in scale scores between SPC services with low and high response rates is only one way to attempt to decide whether selection bias seems plausible in services with low response rates, and the fact that we found no major impact is reassuring.
Only a few large studies (1000–2000 patients) have looked at symptom prevalence or symptom levels in patients at the start of palliative care [17, 20, 21] and to our knowledge this is the first European study to report data from all services in a whole country. Given that the present study indicates that our full data set can be used for analysis without any major risk of impact of selection bias, future analyses can safely be made at the national level to investigate the symptoms/problems and QOL among patients admitted to SPC.
The study hypothesis – that patients from SPC services with lower response rates have better scores on different QOL aspects than those from SPC services with high response rates due to selection bias in services with low response rates - was not supported. Therefore, there does not seem to be any reason to exclude data from SPC services with low response rates in future studies of national data sets.
Availability of data and materials
The data utilized in this study are available at Danish Palliative Care Database. Restrictions apply to the availability of these data.
Danish Palliative Care Database
Specialized palliative care
Quality of life
Stromgren AS, Groenvold M, Pedersen L, Olsen AK, Spile M, Sjogren P. Does the medical record cover the symptoms experienced by cancer patients receiving palliative care? A comparison of the record and patient self-rating. J Pain Symptom Manag. 2001;21(3):189–96.
Stromgren AS, Groenvold M, Sorensen A, Andersen L. Symptom recognition in advanced cancer. A comparison of nursing records against patient self-rating. Acta Anaesthesiol Scand. 2001;45(9):1080–5.
Homsi J, Walsh D, Rivera N, Rybicki LA, Nelson KA, Legrand SB, et al. Symptom evaluation in palliative medicine: patient report vs systematic assessment. Support Care Cancer. 2006;14(5):444–53.
Hjermstad MJ, Lie HC, Caraceni A, Currow DC, Fainsinger RL, Gundersen OE, et al. Computer-based symptom assessment is feasible in patients with advanced cancer: results from an international multicenter study, the EPCRC-CSA. J Pain Symptom Manag. 2012;44(5):639–54.
Stromgren AS, Goldschmidt D, Groenvold M, Petersen MA, Jensen PT, Pedersen L, et al. Self-assessment in cancer patients referred to palliative care: a study of feasibility and symptom epidemiology. Cancer. 2002;94(2):512–20.
Petersen MA, Pedersen L, Groenvold M. Does nonparticipation in studies of advanced cancer lead to biased quality-of-life scores? J Palliat Med. 2009;12(11):1023–8.
Groenvold M, Petersen MA, Aaronson NK, Arraras JI, Blazeby JM, Bottomley A, et al. The development of the EORTC QLQ-C15-PAL: a shortened questionnaire for cancer patients in palliative care. Eur J Cancer. 2006;42(1):55–64.
Groenvold M, Petersen MA. EORTC-QOL-Group. Addendum to the EORTC QLQ-C30 scoring manual: scoring of the EORTC QLQ-C15-PAL. Brussels: European Organization for Research and treatment of Cancer; 2006.
Fayers P, Aaronson N, Bjordal K, Groenvold M, Curran D, Bottomley A. EORTC QLQ-C30 scoring manual. 3rd ed. Brussel: EORTC Quality of Life Group; 2001.
Cocks K, King MT, Velikova G, de Castro G, Jr., Martyn St-James M, Fayers PM, et al. Evidence-based guidelines for interpreting change scores for the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30. Eur J Cancer. 2012;48(11):1713–1721.
Maringwa J, Quinten C, King M, Ringash J, Osoba D, Coens C, et al. Minimal clinically meaningful differences for the EORTC QLQ-C30 and EORTC QLQ-BN20 scales in brain cancer patients. Ann Oncol. 2011;22(9):2107–12.
Maringwa JT, Quinten C, King M, Ringash J, Osoba D, Coens C, et al. Minimal important differences for interpreting health-related quality of life scores from the EORTC QLQ-C30 in lung cancer patients participating in randomized controlled trials. Support Care Cancer. 2011;19(11):1753–60.
Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol. 1998;16(1):139–44.
Musoro JZ, Bottomley A, Coens C, Eggermont AM, King MT, Cocks K, et al. Interpreting European Organisation for Research and Treatment for Cancer Quality of life Questionnaire core 30 scores as minimally importantly different for patients with malignant melanoma. Eur J Cancer. 2018;104:169–81.
Cocks K, King MT, Velikova G, Fayers PM, Brown JM. Quality, interpretation and presentation of European Organisation for Research and Treatment of Cancer quality of life questionnaire core 30 data in randomised controlled trials. Eur J Cancer. 2008;44(13):1793–8.
Hansen MB, Groenvold M. Dansk Palliativ database: Årsrapport 2013. Copenhagen; 2014.
Hjermstad MJ, Aass N, Aielli F, Bennett M, Brunelli C, Caraceni A, et al. Characteristics of the case mix, organisation and delivery in cancer palliative care: a challenge for good-quality research. BMJ Support Palliat Care. 2018;8(4):456–67.
Stromgren AS, Groenvold M, Pedersen L, Olsen AK, Sjogren P. Symptomatology of cancer patients in palliative care: content validation of self-assessment questionnaires against medical records. Eur J Cancer. 2002;38(6):788–94.
Potter J, Hami F, Bryan T, Quigley C. Symptoms in 400 patients referred to palliative care services: prevalence and patterns. Palliat Med. 2003;17(4):310–4.
Vainio A, Auvinen A. Prevalence of symptoms among patients with advanced cancer: an international collaborative study. Symptom prevalence group. J Pain Symptom Manag. 1996;12(1):3–10.
Walsh D, Donnelly S, Rybicki L. The symptoms of advanced cancer: relationship to age, gender, and performance status in 1,000 patients. Support Care Cancer. 2000;8(3):175–9.
A great thanks to the Danish Palliative Care Database for access to the data used in the article and to all the specialized palliative care services in Denmark who delivered the data to the Danish Palliative Care Database.
This work was supported by the Department of Public Health at the University of Copenhagen, the Danish Cancer society (grant number R94-A5634–14-S9) and Clinical Quality Program of the Regions of Denmark (grant number: RKKP-0031/q/01). The funding was used for salary to the first author of this paper (MBH). Funding sources had no role in the conception of the study, analysis or interpretation of data, in writing the manuscript, or in any other part of the study.
Ethics approval and consent to participate
This study was based only on registers from the Danish Palliative Care Database; therefore, it had not impact on any individuals’ care and not required Ethics Committee approval according to Danish law. The study was conducted following the approval from the Danish Data Protection Agency (j.nr.: 2007-58-0015/local j.nr. BFH-2014-033 I-Suite no. 02953).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hansen, M.B., Petersen, M.A., Ross, L. et al. Should analyses of large, national palliative care data sets with patient reported outcomes (PROs) be restricted to services with high patient participation? A register-based study. BMC Palliat Care 19, 89 (2020). https://doi.org/10.1186/s12904-020-00596-z
- ‘Palliative care’
- ‘Quality of Life’
- ‘Sign and Symptoms’
- ‘Patient Reported Outcome Measures’
- ‘Needs assessment’
- ‘Symptom assessment’
- ‘Patient participation’
- ‘Response rate’
- ‘Selection Bias’