Skip to main content
Download PDF

Accuracy of clinical predictions of prognosis at the end-of-life: evidence from routinely collected data in urgent care records

BMC Palliative Care volume 22, Article number: 51 (2023) Cite this article

Abstract

Background

The accuracy of prognostication has important implications for patients, families, and health services since it may be linked to clinical decision-making, patient experience and outcomes and resource allocation. Study aim is to evaluate the accuracy of temporal predictions of survival in patients with cancer, dementia, heart, or respiratory disease.

Methods

Accuracy of clinical prediction was evaluated using retrospective, observational cohort study of 98,187 individuals with a Coordinate My Care record, the Electronic Palliative Care Coordination System serving London, 2010–2020. The survival times of patients were summarised using median and interquartile ranges. Kaplan Meier survival curves were created to describe and compare survival across prognostic categories and disease trajectories. The extent of agreement between estimated and actual prognosis was quantified using linear weighted Kappa statistic.

Results

Overall, 3% were predicted to live “days”; 13% “weeks”; 28% “months”; and 56% “year/years”. The agreement between estimated and actual prognosis using linear weighted Kappa statistic was highest for patients with dementia/frailty (0.75) and cancer (0.73). Clinicians’ estimates were able to discriminate (log-rank p < 0.001) between groups of patients with differing survival prospects. Across all disease groups, the accuracy of survival estimates was high for patients who were likely to live for fewer than 14 days (74% accuracy) or for more than one year (83% accuracy), but less accurate at predicting survival of “weeks” or “months” (32% accuracy).

Conclusion

Clinicians are good at identifying individuals who will die imminently and those who will live for much longer. The accuracy of prognostication for these time frames differs across major disease categories, but remains acceptable even in non-cancer patients, including patients with dementia. Advance Care Planning and timely access to palliative care based on individual patient needs may be beneficial for those where there is significant prognostic uncertainty; those who are neither imminently dying nor expected to live for “years”.

Peer Review reports

Introduction

Clinicians are often required to estimate the survival of patients with advanced conditions [1]. Survival prognostication has important implications for patients, families and health care resource utilisation. Prognoses can influence clinical decisions about care pathways and treatment options, family expectations and psychological burden [2]. For patients with advanced disease, prognosis may be used to inform triage decisions, either for appropriate intensive hospital care or for timely access to community and end-of-life care resources. Failure to recognise when patients are approaching the end of life may contribute to more intensive, but ultimately futile, care [3]. Furthermore, inaccurate predictions may result in a delay in introducing interventions that could otherwise improve the quality of life and support for patients and their families [4]. Accurate prognostication can help patients to make plans and achieve goal-concordant care.

The World Health Organisation estimates that each year more than 40 million people require palliative care globally [5]. The need for palliative care is increasing, especially in developed countries due to an ageing population and greater numbers of patients living with complex comorbidities [6, 7]. Improved identification of patients who are approaching end-of-life may better inform patient care decisions as well as system-wide approaches and policies regarding the equitable provision and access to palliative and end-of-life care [8].

Survival prognostication is challenging [9] and clinical predictions tend to be inaccurate [10, 11]. Studies suggest that clinicians are often overly optimistic and unreliable in their estimates [12, 13]. Several prognostic tools or scoring systems exist but none has consistently displayed superiority against clinician predictions of survival [14,15,16].

Temporal predictions of survival relate to estimates about how long a patient has left to live [17, 18]. These can either be expressed as a continuous variable (e.g., a specific number of days, weeks, or months before death), or using discrete categories (e.g., between 1–2 weeks, etc.). Previous studies evaluating temporal predictions have been relatively small with samples sizes in the hundreds or few thousands [18].

Most studies examining prognostic accuracy have focused on hospital and hospice/palliative care patients, and those with cancer diagnoses. Prognostication appears to be even more inaccurate in patients with a non-cancer diagnosis [12, 19] but there have been few robust studies in this area. As the predicted trajectories of disease vary according to diagnostic category [20], a better understanding of prognostic accuracy in cohorts of patients according to diagnosis is needed to inform clinical training and appropriate application of prognostic estimates in clinical care.

This study aimed to evaluate clinicians’ accuracy of temporal predictions of survival for both cancer and non-cancer patients using routinely collected data from an Electronic Palliative Care Coordination System; an urgent clinical record system where information relevant to the delivery of a patient’s care can be recorded and accessed by health care professionals to improve coordination of care for patients, especially for those nearing the end of life [21, 22].

Methods

Setting and data source

The study used anonymised data routinely collected as part of Coordinate My Care, the largest Electronic Palliative Care Coordination System in UK, serving London 2010 -2022 [22]. Coordinate My Care is designed to facilitate patients and their clinicians in the making of advance care decisions and to share them digitally in real-time with relevant healthcare professionals involved in patients’ care. Coordinate My Care includes clinical information; patients’ preferences about end-of-life care such as their “preferred place of care”, “preferred place of death”; ceiling of treatment and resuscitation decisions. Since 2017, Coordinate My Care plans have contained a mandatory field that requires a temporal clinical prediction of survival. Clinicians are asked to provide a “likely prognosis” using a predetermined list of outcomes including the following categories: “days”, “weeks, “months”, “year”, “years” and “undefined”.

Patients consent to the use of their anonymised information for research at the time of consenting to the creation of a Coordinate My Care record. This service evaluation was approved by the Royal Marsden Committee for Clinical Research (1002, approved 14/09/2020). This study was co-produced with input from patients and members of the public and supported through a Biomedical Research Centre at the Royal Marsden and Institute for Cancer Research grant for Patient and Public Involvement.

Sample

We included all individuals aged 18 or older with a Coordinate My Care plan registered between January 2010 and December 2020. The Coordinate My Care platform is connected to the NHS Spine, a digital NHS information exchange platform via which recorded dates of death are updated. Anonymised data were extracted in April 2021.

Analyses

Stata (Version 14) was used for analyses. Demographics and clinical details were summarised for the study population and stratified according to whether the patient was alive or dead at the time of analysis. The overall cohort was also stratified into those with and without (missing) a defined survival estimate. Records containing a prognosis estimate were included in the analysis of accuracy (“Full prognosis cohort”). Records without a defined prognosis estimate were examined for differences compared to those with a documented prognosis.

Because the temporal estimates that clinicians used to make their prognostic estimates lacked clear boundaries (e.g., “weeks” is not precisely defined on the Coordinate My Care record) it was not possible to analyse the accuracy of predictions without making some assumptions about the boundaries of these categories. We defined a prognosis of “days” to mean 0–13 days, on the basis that if a patient survived for 14 days or more then they had already survived for “weeks”. Using similar logic, we defined “weeks” to mean ≥ 14 days and < 60 days (i.e., fewer than two months); “months” to mean ≥ 60 days and < 365 days (i.e., less than one year); and “years” to mean more than one year. As end-of-life care is typically focussed on patients in their last year of life, the fields “Year” and “Years” were combined to create a cohort of patients who had a longer prognosis. As well as having some face-validity, these boundary limits are like those used to define “days”; “weeks”, “months” and “years” in the well-validated Prognosis in Palliative care Study (PiPS) predictor models [9, 23, 24].

When more than one prognosis was recorded, the initial prognostic estimate was used for analysis. Survival was calculated as the number of days between the date of the initial survival prediction and the date of death if recorded. Survival predictions (“days”, “weeks”, “months”, “year/years”) were compared to individuals’ actual survival for the total study population and for groups of patients hypothesised to have distinct end-of-life illness trajectories [20]: cancer, dementia and frailty, heart disease and respiratory disease.

The accuracy of clinical prognostic estimates was defined as the frequency with which the clinician selected the correct survival category and was described as a percentage of the total number of estimates [9]. The extent of agreement between estimated and actual prognosis was quantified using linear weighted Kappa statistic. Kappa denotes the agreement between prognostic estimate and survival and considers that some prognoses could be expected to be correct purely based on chance. The kappa statistic result can be interpreted as follows: values ≤ 0 indicate no agreement; 0.01–0.20 none to slight; 0.21–0.40 fair; 0.41– 0.60 moderate; 0.61–0.80 substantial; and 0.81–1.00 as almost perfect agreement [25]. The survival times of patients were summarised using median and interquartile (IQ) ranges. Kaplan Meier survival curves were created to describe and compare survival across prognostic categories and disease trajectories. Patients for whom survival since prognosis was longer than a year were censored at day 400 in the Kaplan Meir curves. The log-rank test was used to compare survival between defined groups.

The characteristics of patients for whom prognosis was overestimated or underestimated were examined.

A sensitivity analysis excluding individuals who had a CMC record created after the start of the COVID-19 pandemic (March 2020) was carried out on the primary outcome i.e., the accuracy of the survival estimates to examine the hypothesis that prognostic accuracy may have been inflated, especially during the early months of the pandemic.

The study was reported using the RECORD (REporting of studies Conducted using Observational Routinely-collected Data) statement – extended from the STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) statement (Supplementary Table 1).

Results

One hundred eight thousand four hundred twenty-four eligible Coordinate My Care records were created during the study period. 10,237 (9%) records were excluded from the analysis of accuracy because they were missing information on prognosis prediction. The total analysed sample comprised 98,187 individuals (Table 1). During the observed period, 49,940 (51%) of the study cohort had died and 48,247 (49%) were alive at the time of the extract. Fifty seven percent of the cohort was female and 58% older than 80 years. Cancer and dementia/frailty were the most common diagnoses, 28% and 21% respectively. Cancer (43%) and dementia/frailty (21%) were also the most common diseases at the time of death. Most of the study population were frail (42% had WHO performance status 4, indicating the individual is incapable of performing any self-care). For those that had a place of death recorded, most died outside of hospital (79%). The preferred place of death and preferred place of care at the end of life were home and care home. Many patients had a recorded “do not attempt cardiopulmonary resuscitation” order (63%).

Table 1 Characteristics of the study sample containing prognostic estimates N (%)
Full size table

Out of a total cohort of individuals without a documented prognosis (n = 10,237), 81.5% had died at the time of data extraction. There were differences between these patients and the sample included in the analysis in terms of diagnosis, place of death and DNACPR status. In the cohort with missing prognosis information there was a slightly higher proportion of patients with cancer and dementia (34.4% versus 28.1% and 27.2% versus 20.9% respectively compared with the accuracy study cohort) and a lower proportion of patients with respiratory and cardiac disease. In the cohort with missing prognosis information, a much lower proportion died at home (29% versus 40%) and a higher proportion died in hospital (28.1% versus 21.5% in the cohort with prognostic information). There was a lower proportion of patients in the cohort without prognosis information with a DNACPR order in place (57.6% versus 63%) (Supplementary Table 2).

In the full prognosis cohort, 3% were predicted to live “days”; 13% “weeks”; 28% “months”; and 56% “year/years” (Table 2). The absolute agreement between prognostication and survival was 62% for the entire sample with an overall linear weighted Kappa of 0.634. Overall, clinicians “overestimated” prognosis (i.e., the patient died sooner than expected) in 23% of cases and underestimated prognosis (i.e., the patient lived longer than expected) in 15% of cases.

Table 2 Prognostication accuracy in the full cohort and stratified across major disease groups. Full prognosis cohort includes individuals with a primary diagnosis classified as “cancer”, “dementia/frailty”, “heart disease”, “respiratory disease” and “other”. Shaded cells for the “Prediction” and “Survival” denote those for whom the prediction was accurate. Weighted Kappa denotes the agreement between prognostication outcome and survival, the higher the value the better the prognostic ability. Q1 First quartile, the value in the dataset that holds 25% of the values below it. Q3 Third Quartile, the value in the dataset that holds 25% of the values above it
Full size table

The agreement between estimated and actual prognosis using linear weighted Kappa statistic was substantial for all disease groups and highest for patients with dementia/frailty (0.75) and cancer (0.73).The relatively good overall agreement between prognostic estimates and survival was influenced by the high accuracy of estimates that patients were likely to die within days (74%) or to live for more than a year (83%). The absolute agreement between prognostic estimates and survival was highest for patients expected to live “days” (74% in full cohort) or “year/years” (83% in full cohort) and this was mirrored in all disease categories. On the other hand, across all disease groups, clinicians were less accurate at predicting who would live for weeks or months (32% in both categories in the full cohort).

Clinicians’ estimates were able to discriminate (log-rank p < 0.001) between groups of patients with differing survival prospects as illustrated by the Kaplan–Meier survival curves for the general cohort (Fig. 1). The median survival for each group fell within the expected upper and lower boundary limits. That is: patients expected to survive for “days” (< 14 days) survived for a median of 4 days; those expected to survive for “weeks” (≥ 14 days and < 60 days) survived for a median of 24 days; those expected to survive for “months” (≥ 60 days and < 365 days) survived for a median of 146 days, and those expected to survive for “years” (≥ 365 days) survived for a median of 639 days. Similar patterns were observed across all disease groups (Fig. 2).

Fig. 1
figure 1

Kaplan–Meier survival curves for the full cohort across different prognostication categories. Notes: Survival is censored at day 400. Log ran test p<0.001

Full size image
Fig. 2
figure 2

Kaplan–Meier survival curves for the major disease groups across different prognostication categories. Notes: Survival is censored at day 400. Log rank test for each disease group p<0.001. Additional data including numbers at risk are included in Additional File 2

Full size image

There was no difference in the accuracy of prognostic estimates when individuals who had a CMC record created after the start of the COVID-19 pandemic (March 2020) were excluded (Supplementary Table 3).

A higher proportion of patients for whom providers overestimated prognosis were older, female, with a diagnosis of dementia and of those who died, a higher proportion died in a care home, as compared to those for whom providers underestimated prognosis (Supplementary Table 4).

Discussion

Main findings/results of the study:

Using a large set of routinely collected data we evaluated the accuracy of temporal predictions of survival in patients with a variety of advanced illnesses.

Our analysis demonstrates that the overall clinical accuracy of categorising patients into broad prognostic groups was 62%, mirroring prognostication estimates in other clinical studies [9, 14]. Clinicians were particularly good at identifying patients who were likely to live for fewer than 14 days. In a recent large multi-centre study of 1896 patients with cancer admitted to a palliative care unit, as in our study, survival estimates (using clinician’s predictions of survival or prognostic scales) were most accurate in patients with days to live. In that study however, all methods of prognostication demonstrated good performance for up to 60 days of survival, which is likely explained by the underlying diagnosis and the short overall median survival time (19 days), suggesting that patients were likely to have had more predictable trajectories [26]. In our study of patients with a variety of diagnoses and likely disease trajectories, clinicians were also good at identifying patients who were likely to live for more than one year but were less accurate at predicting survival of “weeks” or “months”. The greatest extent of agreement between estimated and actual prognosis was found for patients with dementia/frailty or cancer. The median survival of patients predicted to live for days, weeks, months, or years, fell within the expected range for the whole cohort and each disease category.

What this study adds:

In this study, the accuracy of predictions about imminent death (within two weeks) was 74.2% across the full patient cohort. The best accuracy for this time frame was in cancer patients (79.5%) and the worst was in patients with respiratory disease (63.5%). Our findings align with the more typically predictable short period of rapid decline which is often described in patients with cancer, as compared to the gradual and progressive deterioration in patients with other diagnoses [20, 27, 28]. For patients with heart or lung disease, frequent exacerbations in their clinical condition on the background of a gradual decline in health and functional status make identification of the dying phase more challenging [20]. Nonetheless, our results provide reassurance that for the majority of patients, regardless of diagnosis, clinicians are reasonably accurate at identifying the last few days of life.

Clinicians were also good at recognising which patients were likely to live for more or less than one year. Of the 55,347 patients predicted to survive for more than one year, 45,920 (83%) did so. Conversely, of the 39,840 people predicted to live for less than one year, 30,726 (77%) did so.

This study demonstrates the significant difference in the accuracy of survival estimates depending on the prognostic category. Factors that influence the accuracy of survival estimates are yet to be defined but are likely to be complex and multiple [18]. Whilst prognostic models vary in levels of sophistication [29], despite its central importance, clinicians are rarely trained for prognostication [30]. Clinicians usually consider a number of patients’ characteristics when providing a prognosis, such as performance status, underlying diagnosis, clinical observations, symptoms, comorbidities and the known overall disease trajectory [24, 31,32,33]. Clinicians also often rely on their own clinical experience and intuition. This may, however, be subject to explicit or implicit cognitive bias [17]. Clinicians acknowledge the challenges of estimating survival and discussing estimates with patients and families [23]. In addition, there is an inherent human desire to preserve hope, which may consciously or unconsciously affect clinicians’ communications about prognosis. Timing of prognosis is relevant, as evidence suggests that survival estimates may become more accurate when people are nearing end-of-life, a phenomenon known as “the horizon effect” [1, 34]. This finding is mirrored in the data presented in this study. However, survival predictions remain a challenge for many clinicians, even for patients who are imminently dying [35].

Our study shows that it is more difficult to identify patients who will die within “weeks” or “months” than “days” or “years”. Although many of these patients died within or earlier than the upper bound of the predicted period (e.g., most patients predicted to die within “weeks” died in fewer than 60 days [8,387/12,457 = 67%]), these data highlight the significant prognostic uncertainty faced by patients, who are neither imminently dying nor expected to live for “years”. Approaches and interventions to support patients whose situations are clinical uncertain assume greater importance [36,37,38] greatly amplified during the COVID-19 pandemic [39]. Advance care planning (i.e., enabling patients to define goals and preferences for future medical treatment and care, and to record and share these discussions and decisions with relevant health and social care providers) [40] attempts to ensure that care is patient-centred even when their situation is uncertain. Early palliative care may also have a role here. The introduction of palliative care early in a patient’s disease journey has been shown to support communication and improve quality of life [3, 41, 42].

The majority of the records contained within this Electronic Palliative Care Coordination System included a prognostic estimate, which is reflective of the nature of such systems. However, in our cohort, patients with missing information on prognosis were much more likely to die in hospital and much less likely to die at home, despite the finding that a similar proportion wanted to die either at home or in a care home. Sensitive and timely discussions around prognosis may be a helpful part of the process of advance care planning for some patients, acknowledging that this is not something that every patient wants to engage with.

In this study, we observed high levels of prognostic accuracy for patients with dementia who were expected to die within days and for those with a life expectancy of year/years but poor accuracy of survival estimates of “weeks” or “months”. Patients with dementia however often experience an unpredictable disease trajectory. This can make identification of the end-of-life and prognostication challenging [43]. Our data support international recommendations that the initiation of palliative care for patients with dementia should be based on need rather than prognostication [44].

Strengths and weaknesses/limitations of the study:

This was a large study involving patients with varying diagnoses. As a result, we were able to make robust estimates of the accuracy of survival for different groups of patients and evaluate prognostic accuracy. These data provide a real-life insight into survival predictions incorporated into individual patients’ clinical care decisions and care preferences. This study was large enough to enable stratification and comparison across major disease trajectories, facilitating detailed insight into disease-specific prognostication. Having a temporal estimate of survival as a mandatory data field ensured that prognosis information was available for 91% of the overall cohort in the observed period.

Our analyses were limited due to the format of the available data. The survival prediction was based on categorical values, consequently we are unable to calculate the area under the receiver operating characteristics curve (AUROC). Linear weighted kappa coefficients were used to determine “true” agreement between estimates and actual survival, considering the agreement that would be expected by chance [45] and giving an average of the kappa coefficients across the different survival categories [46]. Our use of LWK and percentage of accurate estimates supports comparison with other prognostic studies in which this approach was used [9, 18].

Patients with a Coordinate My Care record are expected to have a limited life expectancy as the service is designed for patients with advanced conditions and therefore our results cannot be generalised to other patient cohorts. This cohort was based in London, so results may not be generalisable to non-urban settings. Although the role of the clinician in documenting prognosis in Coordinate My Care is recorded, it is not possible to determine which health care professional made the prognosis or the role of the multi-disciplinary team in this process [47]. A previous systematic review concluded that there was no difference in the accuracy of prognostic estimates between clinician sub-groups [18]. Prognostic estimates may be revised over time, and subsequent estimates may be more or less accurate than initial ones. We did not include longitudinal changes in prognostication in our analysis. Our data included some Coordinate My Care records that were created during the initial months of the COVID-19 pandemic. It is possible that the onset of a new and unknown disease made accurate prognoses even more difficult, Our data did not however demonstrate any difference in prognostic accuracy when prognostic estimates from records created after the start of the pandemic were included.

Our study was based on the retrospective analysis of routinely collected data and does not give any information about the information needs of individual patients about prognostic estimates.

Conclusion

Prognosticating in clinical practice is challenging, however, our results suggest that clinician predictions of survival are generally good at identifying individuals at either end of the survival spectrum (i.e., those who will die imminently and those who will live for much longer). The accuracy of prognostication for these time frames differs across major disease categories, but remains acceptable even in non-cancer patients, including those with dementia. Identifying patients who will live more than “days” but less than “months”, or more than “weeks” but less than “years” is more challenging and further research to improve prognostic accuracy for these patients is needed. For these patients with more uncertain outcomes, advance care planning and early introduction to palliative care may be particularly beneficial. We suggest that training for clinicians in having sensitive discussions around prognosis [33], should include an acknowledgement of the significant level of prognostic uncertainty for patients who are not thought to be imminently dying with an emphasis on the need for appropriate and individualised support according to how much information the patient and their family wants to know [48].

Availability of data and materials

Anonymised data from this study are stored on the Biomedical Research Informatics Digital Environment (BRIDgE), a Trusted Research Environment (TRE) and informatics platform at The Royal Marsden Biomedical Research Centre. Data sharing requests and access to the protocol and supplementary information would be available on reasonable request after completion of existing studies and whenever legally and ethically possible. Data access requests should be directed to Dr Joanne Droney joanne.droney@rmh.nhs.uk.

Access requests will be reviewed and authorised by The Royal Marsden Committee for Clinical Research and Information Governance Committee.

References

  1. Selby D, Chakraborty A, Lilien T, Stacey E, Zhang L, Myers J. Clinician accuracy when estimating survival duration: The role of the patient’s performance status and time-based prognostic categories. J Pain Symptom Manage. 2011;42:578–88. https://doi.org/10.1016/j.jpainsymman.2011.01.012.

    Article  PubMed  Google Scholar 

  2. Glare PA, Sinclair CT. Palliative Medicine Review: Prognostication. J Palliat Med. 2008;11:84–103. https://doi.org/10.1089/jpm.2008.9992.

    Article  PubMed  Google Scholar 

  3. Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, et al. Early palliative care for patients with metastatic non–small-cell lung cancer. N Engl J Med. 2010;36(8):733–42. https://doi.org/10.1056/nejmoa1000678.

  4. Fairchild A, Debenham B, Danielson B, Huang F, Ghosh S. Comparative multidisciplinary prediction of survival in patients with advanced cancer. Support Care Cancer. 2014;22:611–7. https://doi.org/10.1007/s00520-013-2013-2.

    Article  CAS  PubMed  Google Scholar 

  5. World Health Organisation. Planning and implementing palliative care services: a guide for programme managers. 2016. ISBN 978 92 4 156541 7

  6. Bone AE, Gomes B, Etkind SN, Verne J, Murtagh FE, Evans CJ, et al. What is the impact of population ageing on the future provision of end-of-life care? Population-based projections of place of death. Palliat Med. 2018;32:329–36. https://doi.org/10.1177/0269216317734435.

    Article  PubMed  Google Scholar 

  7. Etkind SN, Bone AE, Gomes B, Lovell N, Evans CJ, Higginson IJ, et al. How many people will need palliative care in 2040? Past trends, future projections and implications for services. BMC Med. 2017;15:102. https://doi.org/10.1186/s12916-017-0860-2.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Department of Health and Social Care (DHSC) England. End of Life Care Strategy – Promoting High Quality Care for All Adults at the End of Life. 2008.

  9. Gwilliam B, Keeley V, Todd C, Roberts C, Gittins M, Kelly L, et al. Prognosticating in patients with advanced cancer-observational study comparing the accuracy of clinicians’ and patients’ estimates of survival. Ann Oncol. 2013;24:482–8. https://doi.org/10.1093/annonc/mds341.

    Article  CAS  PubMed  Google Scholar 

  10. White N, Kupeli N, Vickerstaff V, Stone P. How accurate is the “Surprise Question” at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med. 2017;15:139. https://doi.org/10.1186/s12916-017-0907-4.

  11. Glare P, Virik K, Jones M, Hudson M, Eychmuller S, Simes J, et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ. 2003;327(7408):195–8. https://doi.org/10.1136/bmj.327.7408.195.

  12. Coventry PA, Grande GE, Richards DA, Todd CJ. Prediction of appropriate timing of palliative care for older adults with non-malignant life-threatening disease: a systematic review. Age Ageing. 2005;34:218–27. https://doi.org/10.1093/ageing/afi054.

    Article  PubMed  Google Scholar 

  13. Urahama N, Sono J, Yoshinaga K. Comparison of the accuracy and characteristics of the prognostic prediction of survival of identical terminally ill cancer patients by oncologists and palliative care physicians. Jpn J Clin Oncol. 2018;48:695–8. https://doi.org/10.1093/jjco/hyy080.

    Article  PubMed  Google Scholar 

  14. Stone P, Kalpakidou A, Todd C, Griffiths J, Keeley V, Spencer K, et al. Prognostic models of survival in patients with advanced incurable cancer: The pips2 observational study. Health Technol Assess (Rockv). 2021;25(28):1–118. https://doi.org/10.3310/HTA25280.

  15. Stone P, White N, Oostendorp LJM, Llewellyn H, Vickerstaff V. Comparing the performance of the palliative prognostic (PaP) score with clinical predictions of survival: A systematic review. Eur J Cancer. 2021;158:27–35. https://doi.org/10.1016/j.ejca.2021.08.049.

    Article  PubMed  Google Scholar 

  16. Stone P, Vickerstaff V, Kalpakidou A, Todd C, Griffiths J, Keeley V, et al. Prognostic tools or clinical predictions: which are better in palliative care? PLoS One. 2021;16(4):e0249763. https://doi.org/10.1371/journal.pone.0249763.

  17. Cherny NI, Fallon M, Kaasa S. Oxford textbook of palliative medicine. 5th edition. Oxford: Oxford University Press; 2015. https://doi.org/10.1093/med/9780199656097.001.0001. Accessed 18 Apr 2023.

  18. White N, Reid F, Harris A, Harries P, Stone P. A systematic review of predictions of survival in palliative care: How accurate are clinicians and who are the experts? PLoS One. 2016;11(8):e0161407. https://doi.org/10.1371/journal.pone.0161407.

  19. Downar J, Goldman R, Pinto R, Englesakis M, Adhikari NKJ. The “surprise question” for predicting death in seriously ill patients: a systematic review and meta-analysis. CMAJ. 2017;189:484–93. https://doi.org/10.1503/cmaj.160775/-/DC1.

    Article  Google Scholar 

  20. Murray SA, Kendall M, Boyd K, Sheikh A. Illness trajectories and palliative care. BMJ. 2005;330:1007–11. https://doi.org/10.1136/bmj.330.7498.1007.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Leniz J, Weil A, Higginson IJ, Sleeman KE. Electronic palliative care coordination systems (EPaCCS): A systematic review. BMJ Support Palliat Care. 2020;10:68–78. https://doi.org/10.1136/bmjspcare-2018-001689.

    Article  PubMed  Google Scholar 

  22. Petrova M, Riley J, Abel J, Barclay S. Crash course in EPaCCS (Electronic Palliative Care Coordination Systems): 8 years of successes and failures in patient data sharing to learn from. BMJ Support Palliat Care. 2018;8:447–55. https://doi.org/10.1136/bmjspcare-2015-001059.

    Article  PubMed  Google Scholar 

  23. Stone PC, Kalpakidou A, Todd C, Griffiths J, Keeley V, Spencer K, et al. The prognosis in palliative care study II (PiPS2): a prospective observational validation study of a prognostic tool with an embedded qualitative evaluation. PLoS One. 2021;16(4):e0249297. https://doi.org/10.1371/journal.pone.0249297.

  24. Gwilliam B, Keeley V, Todd C, Gittins M, Roberts C, Kelly L, et al. Development of Prognosis in Palliative care Study (PiPS) predictor models to improve prognostication in advanced cancer: prospective cohort study. BMJ Support Palliat Care. 2015;5:390–8. https://doi.org/10.1136/bmjspcare-2012-d4020rep.

    Article  PubMed  Google Scholar 

  25. Landis JRKG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    Article  CAS  PubMed  Google Scholar 

  26. Hiratsuka Y, Suh S-Y, Hui D, Morita T, Mori M, Oyamada S, et al. Are Prognostic Scores Better Than Clinician Judgment? A Prospective Study Using Three Models. J Pain Symptom Manage. 2022;64:391–9. https://doi.org/10.1016/j.jpainsymman.2022.06.008.

    Article  PubMed  Google Scholar 

  27. Morita T, Tsunoda J, Inoue S, Chihara S. The Palliative Prognostic Index: a scoring system for survival prediction of terminally ill cancer patients. Support Care Cancer. 1999;7:128–33. https://doi.org/10.1007/s005200050242.

    Article  CAS  PubMed  Google Scholar 

  28. Feliu J, Jiménez-Gordo AM, Madero R, Rodríguez-Aizcorbe JR, Espinosa E, Castro J, et al. Development and Validation of a Prognostic Nomogram for Terminally Ill Cancer Patients. JNCI: J Nat Cancer Instit. 2011;103:1613–20. https://doi.org/10.1093/jnci/djr388.

    Article  Google Scholar 

  29. Yourman LC, Lee SJ, Schonberg MA, Widera EW, Smith AK. Prognostic Indices for Older Adults. JAMA. 2012;307:182. https://doi.org/10.1001/jama.2011.1966.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Christakis NA, Iwashyna TJ. Attitude and Self-reported Practice Regarding Prognostication in a National Sample of Internists. Arch Intern Med. 1998;158:2389. https://doi.org/10.1001/archinte.158.21.2389.

    Article  CAS  PubMed  Google Scholar 

  31. Button E, Chan RJ, Chambers S, Butler J, Yates P. A systematic review of prognostic factors at the end of life for people with a hematological malignancy. BMC Cancer. 2017;17:213. https://doi.org/10.1186/s12885-017-3207-7.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Hauser CA, Stockler MR, Tattersall MHN. Prognostic factors in patients with recently diagnosed incurable cancer: a systematic review. Support Care Cancer. 2006;14:999–1011. https://doi.org/10.1007/s00520-006-0079-9.

    Article  PubMed  Google Scholar 

  33. White N, Oostendorp LJ, Tomlinson C, Yardley S, Ricciardi F, Gökalp H, et al. Online training improves medical students’ ability to recognise when a person is dying: The ORaClES randomised controlled trial. Palliat Med. 2020;34:134–44. https://doi.org/10.1177/0269216319880767.

    Article  PubMed  Google Scholar 

  34. Mackillop WJ, Quirt CF. Measuring the accuracy of prognostic judgments in oncology. J Clin Epidemiol. 1997;50:21–9. https://doi.org/10.1016/S0895-4356(96)00316-2.

    Article  CAS  PubMed  Google Scholar 

  35. White N, Reid F, Vickerstaff V, Harries P, Stone P. Specialist palliative medicine physicians and nurses accuracy at predicting imminent death (within 72 hours): a short report. BMJ Support Palliat Care. 2020;10:209–12. https://doi.org/10.1136/bmjspcare-2020-002224.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Etkind SN, Koffman J. Approaches to managing uncertainty in people with life-limiting conditions: role of communication and palliative care. Postgrad Med J. 2016;92:412–7. https://doi.org/10.1136/postgradmedj-2015-133371.

    Article  CAS  PubMed  Google Scholar 

  37. Koffman J, Yorganci E, Yi D, Gao W, Murtagh F, Pickles A, et al. Managing uncertain recovery for patients nearing the end of life in hospital: a mixed-methods feasibility cluster randomised controlled trial of the AMBER care bundle. Trials. 2019;20:506. https://doi.org/10.1186/s13063-019-3612-0.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Simpkin AL, Schwartzstein RM. Tolerating Uncertainty — The Next Medical Revolution? N Engl J Med. 2016;375:1713–5. https://doi.org/10.1056/NEJMp1606402.

    Article  PubMed  Google Scholar 

  39. Koffman J, Gross J, Etkind SN, Selman L. Uncertainty and COVID-19: how are we to respond? J R Soc Med. 2020;113:211–6. https://doi.org/10.1177/0141076820930665.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Rietjens JAC, Sudore RL, Connolly M, van Delden JJ, Drickamer MA, Droger M, et al. Definition and recommendations for advance care planning: an international consensus supported by the European Association for Palliative Care. Lancet Oncol. 2017;18:e543–51. https://doi.org/10.1016/S1470-2045(17)30582-X.

    Article  PubMed  Google Scholar 

  41. Zimmermann C, Swami N, Krzyzanowska M, Hannon B, Leighl N, Oza A, et al. Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial. The Lancet. 2014;383:1721–30. https://doi.org/10.1016/S0140-6736(13)62416-2.

    Article  Google Scholar 

  42. Vanbutsele G, Pardon K, van Belle S, Surmont V, de Laat M, Colman R, et al. Effect of early and systematic integration of palliative care in patients with advanced cancer: a randomised controlled trial. Lancet Oncol. 2018;19:394–404. https://doi.org/10.1016/S1470-2045(18)30060-3.

    Article  PubMed  Google Scholar 

  43. Browne B, Kupeli N, Moore KJ, Sampson EL, Davies N. Defining end of life in dementia: A systematic review. Palliat Med. 2021;35:1733–46. https://doi.org/10.1177/02692163211025457.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Livingston G, Huntley J, Sommerlad A, Ames D, Ballard C, Banerjee S, et al. Dementia prevention, intervention, and care: 2020 report of the Lancet Commission. The Lancet. 2020;396:413–46. https://doi.org/10.1016/S0140-6736(20)30367-6.

    Article  Google Scholar 

  45. Sim J, Wright C. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–68.

    Article  PubMed  Google Scholar 

  46. Warrens MJ. Cohen’s linearly weighted kappa is a weighted average. Adv Data Anal Classif. 2012;6:67–79. https://doi.org/10.1007/s11634-011-0094-7.

    Article  Google Scholar 

  47. Bruun A, Oostendorp L, Bloch S, White N, Mitchinson L, Sisk AR, et al. Prognostic decision-making about imminent death within multidisciplinary teams: a scoping review. BMJ Open 2022;12(4):e057194. https://doi.org/10.1136/bmjopen-2021-057194.

  48. Innes S, Payne S. Advanced cancer patients’ prognostic information preferences: a review. Palliat Med. 2009;23:29–39. https://doi.org/10.1177/0269216308098799.

    Article  CAS  PubMed  Google Scholar 

  49. NHS Health Research Authority. Does my project require review by a Research Ethics Committee? ‘Does My Project Require Review by a Research Ethics Committee?’ V 20 n.d. https://hra-decisiontools.org.uk/ethics/

Download references

Acknowledgements

We would like to thank Lisa Scerri and the Royal Marsden Biomedical Research Centre for their support of this study and the use of the Biomedical Research Informatics Digital Environment, a Trusted Research Environment.

This project represents independent research supported by the National Institute for Health and Care Research (NIHR) Biomedical Research Centre at The Royal Marsden NHS Foundation Trust and the Institute of Cancer Research, London. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

Funding

This study was sponsored by the Royal Marsden NHS Foundation Trust with support from the Royal Marsden National Institute for Health Research Biomedical Research Centre (NIHR BRC). Patient and Public Involvement was funded through an NIHR BRC grant for PPI in research (Grant Reference B112). The researcher’s salary was funded through the Coordinate My Care Charity Research Fund. PS is supported by Marie Curie (Grant reference MCCC-FCH-18-U). The funders had no role in the design of the study or the analysis and interpretation of the data.

Author information

Authors and Affiliations

  1. Royal Marsden NHS Foundation Trust, London, SW3 6JJ, United Kingdom

    M. Orlovic, J. Droney, J. Rosling, A. Bearne, M. Powell, J. Riley & P. McFarlane

  2. Imperial College London, London, United Kingdom

    M. Orlovic, J. Droney & J. Riley

  3. Marie Curie Palliative Care Research Department, Division of Psychiatry, University College London, London, United Kingdom

    V. Vickerstaff & P. Stone

  4. Hull York Medical School, Wolfson Palliative Care Research Centre, University of York, York, United Kingdom

    J. Koffman

Authors
  1. M. Orlovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Droney
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. Vickerstaff
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. Rosling
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. Bearne
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. J. Riley
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. P. McFarlane
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. J. Koffman
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. P. Stone
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MO, JD, JRi, and PS developed the study concept and designed the study protocol. JD and MO drafted and submitted the research governance approvals with input from PS, VV and JRi. MO led the statistical analyses with critical input from JD, PS and VV. JD, PS, VV, JRi, JR, AB, MP, JK and PM provided critical input into the data interpretation. JD led the PPI engagement work for this study, working closely with JR, AB and MP, the PPI co-investigators. JD, JR, AB and MP drafted and submitted the PPI grant application. MO drafted the paper and all authors read, refined, and provided significant input into the manuscript revisions and approved the final manuscript for submission for publication.

Corresponding author

Correspondence to J. Droney.

Ethics declarations

Ethics approval and consent to participate

This study was co-produced with input from patients and members of the public and supported through a Biomedical Research Centre at the Royal Marsden and Institute for Cancer Research grant for Patient and Public Involvement. Three members of the Royal Marsden BRC research review panel were involved in the study design, review of the results and drafting of the manuscript.

In accordance with guidance from the National Health Service Health Research Authority /Medical Research Council UK, this study of the anonymised use of data collected as part of routine care was approved as a service evaluation by the Royal Marsden Committee for Clinical Research and was exempt from National Health Service research ethics committee review [49]. Patients provided informed consent to the use of their anonymised data for research when creating a Coordinate My Care record. The study was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice.

Consent for publication

Not applicable.

Competing interests

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Supplementary Table 1. The RECORD statement – checklist of items,extended from the STROBE statement, that should be reported in observationalstudies using routinely collected health data. *Reference: Benchimol EI, SmeethL, Guttmann A, Harron K, Moher D, Petersen I, SørensenHT, von Elm E, Langan SM, the RECORD Working Committee.  The REporting of studies Conducted usingObservational Routinely-collected health Data (RECORD) Statement.  PLoSMedicine 2015. SupplementaryTable 2. Patients without arecorded prognostic estimate. Supplementary Table 3. Prognostic accuracyexcluding records created after start of COVID19 pandemic (March 2020). SupplementaryTable 4. Patients with over and underestimated prognosis.

Additional file 2.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Orlovic, M., Droney, J., Vickerstaff, V. et al. Accuracy of clinical predictions of prognosis at the end-of-life: evidence from routinely collected data in urgent care records. BMC Palliat Care 22, 51 (2023). https://doi.org/10.1186/s12904-023-01155-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12904-023-01155-y

Keywords

BMC Palliative Care

ISSN: 1472-684X

Contact us