Skip to main content
Download PDF

An online international comparison of thresholds for triggering a negative response to the “Surprise Question”: a study protocol

BMC Palliative Care volume 18, Article number: 36 (2019) Cite this article

A Correction to this article was published on 27 January 2020

This article has been updated

Abstract

Background

The Surprise Question (SQ) “would I be surprised if this patient were to die in the next 12 months?” has been suggested to help clinicians, and especially General Practitioners (GPs), identify people who might benefit from palliative care. The prognostic accuracy of this approach is unclear and little is known about how GPs use this tool in practice. Are GPs consistent, individually and as a group? Are there international differences in the use of the tool? Does including the alternative Surprise Question (“Would I be surprised if the patient were still alive after 12 months?”) alter the response? What is the impact on the treatment plan in response to the SQ? This study aims to address these questions.

Methods

An online study will be completed by 600 (100 per country) registered GPs. They will be asked to review 20 hypothetical patient vignettes. For each vignette they will be asked to provide a response to the following four questions: (1) the SQ [Yes/No]; (2) the alternative SQ [Yes/No]; (3) the percentage probability of dying [0% no chance – 100% certain death]; and (4) the proposed treatment plan [multiple choice]. A “surprise threshold” for each participant will be calculated by comparing the responses to the SQ with the probability estimates of death. We will use linear regression to explore any differences in thresholds between countries and other clinician-related factors, such as years of experience. We will describe the actions taken by the clinicians and explore the differences between groups. We will also investigate the relationship between the alternative SQ and the other responses. Participants will receive a certificate of completion and the option to receive feedback on their performance.

Discussion

This study explores the extent to which the SQ is consistently used at an individual, group, and national level. The findings of this study will help to understand the clinical value of using the SQ in routine practice.

Trial registration

Clinicaltrials.gov NCT03697213 (05/10/2018). Prospectively registered.

Background

Predicting how long a patient has left to live is an inaccurate and complicated clinical skill [1,2,3,4,5]. Yet it is a clinical skill expected of all medical doctors [6]. In the UK, early identification of people approaching the last year of life can facilitate referral to specialist services and funding, which in turn improves the patient’s quality of life [7,8,9,10,11,12]. The task of early identification of these patients often falls on General Practitioners (GPs) in the community. The timely identification of patients in the last 12 months of life has been described by GPs as one of the main challenges they face [13]. It has been reported that GPs often wait until very close to death before discussing end of life issues [14] which can often prevent the individual experiencing a “good death” [15, 16]. The “Surprise Question” (SQ) is one such tool available to identify those who might benefit from palliative care. The SQ describes the process whereby clinicians are encouraged to ask themselves whether or not they would “be surprised” if a patient were to die within a specific time (e.g. 12 months, 6 months, 1 month, 7 days). It is suggested that if a clinician “would not be surprised” if a patient were to die within, for example, the next 12 months then this should act as a trigger to adopting a more palliative care approach to their care (e.g. placing the patient on an end-of-life communication register or referring them to specialist palliative care services). Although the original aim of the SQ was not simply to predict survival but to identify people with palliative care needs [17, 18], it is often used as a means to identify people who might be in the last 12 months of life. The prognostic capability of the SQ has been reported to be variable [19,20,21].

One problem with the SQ is that it is not clear to what extent a death needs to be expected before a clinician would be “surprised” if it did or did not occur. It has previously been highlighted by GPs how subjective and difficult the SQ can be in clinical practice [22]. If there are large variations in these “trigger” values, or “threshold levels”, between clinicians then this would have important implications for how responses to the SQ should be interpreted. The variation between individuals’ threshold levels, and the consistency of the responses to the SQ, have not previously been evaluated. It is also not known which other factors may affect trigger values (e.g. nationality, age, gender, palliative care experience, or years of seniority). A better understanding about when clinicians would, or would not, be “surprised” if a patient were to die would help to standardise and calibrate the use of the SQ. Being aware of the threshold probabilities which trigger “surprise” in different groups may also lead to strategies to improve the accuracy and the consistency with which the SQ is used in clinical practice.

Research by Carel Veldhoven and colleagues [23, 24] reported that the use of an additional alternative Surprise Question “would I be surprised if this patient is still alive after 12 months?”, following a negative response (“No”) to the original Surprise Question, improves the predictive value of the question. Incorporating this question, alongside the original SQ could help improve the prognostic capability of the tool in addition to focusing care where it is needed most. Further research is needed to validate this finding.

In addition to this, there is little evidence about the relationship between the SQ response and the course of action decided upon. What evidence is available suggests that the treatment decisions made at the end of life vary considerable depending on clinician and patient factors [25]. Do GPs consistently pursue the same course of action for patients with similar symptoms, or for those with the same SQ response?

Objectives

The study has the following objectives:

  1. 1.

    To determine how consistent General Practitioners are in their response to the SQ, and what their “threshold” level is in order to provide a negative response (“No, I would not be surprised”) to the Surprise Question

  2. 2.

    To determine the relationship between the original Surprise Question, the alternative Surprise Question (“Would I be surprised if this person were still alive after 12 months”), and the estimated probability of death occurring in the next 12 months.

  3. 3.

    To understand the course of action decided upon in relation to the prognostic estimates provided.

Methods

Methods: participants, interventions, and outcomes

This protocol follows the SPIRIT reporting guidelines for research protocols [26]. The checklist can be found in Additional file 1.

Study design

The Surprise Study is a non-randomised multicentre online study with a single group of GPs who will complete 20 hypothetical patient summaries, or “vignettes”.

Study setting

The online study will be distributed to GPs in the following participating countries: Belgium, Germany, Italy, The Netherlands, Switzerland, and the United Kingdom.

Eligibility criteria

A convenience sample of GPs with a variety of years of experience and seniority across six European countries will be approached to participate.

Inclusion criteria

  • Registered GP in one of the six participating countries

  • Able to read and understand the language in which the study is presented to them

Exclusion criteria

  • Decline to participate

Procedure

Participants will receive a Participant Information Sheet (PIS) if they receive an email to participate and/or they will be able to download a PIS directly from the study website. Informed consent will be embedded in the study website and will be obtained via four checkboxes before starting the study. The first will be to consent to participate. The second will be to consent to the results being used in future publications, research and educational packages. The third box will be consent to provide an email address in order to enable participants to log out and return, and for the research team to send reminder emails. The fourth consent box is optional and will ask participants if they wish to receive feedback on their performance. The participant will be reminded that they are free to withdraw at any time. On completion, a debrief page will be provided to remind the participant what the results will be used for. The contact details for the study team will also be displayed should they have any concerns or issues they wish to raise.

Once consented, participants will be asked to read through 20 vignettes of patients with advanced diseases but who have not yet been referred to specialist palliative care services.

Vignette development

The vignettes were constructed by the collaborators in the group and were designed to represent typical patient cases some of whom would be expected to die within 12 months, some of whom would be expected to survive for 12 months and some with an uncertain prognosis. The actual survival of the patients depicted in the vignettes will not be known (since the cases will not represent “real” patients). However the purpose of the exercise is not to evaluate the accuracy of clinicians’ predictions but rather to investigate the level of probability at which respondents would not be surprised if the patient were to die within the next 12 months. The vignettes contain non-specialist information (e.g. they will not contain specialty specific blood test results or diagnostic test results) so that generalist doctors without any specialty-specific training will be able to understand the information provided. See Fig. 1 for an example of a patient vignette.

Fig. 1
figure 1

Example hypothetical vignette

Full size image

To ensure an equal distribution of symptoms and disease groups, the vignettes were developed follow a guidance template, informed from the Gold Standards Framework Proactive Indicator Guidance [27], as well as disease specific grading measures (Additional file 2). Each country participating developed a 5 vignettes following this template, from which the final set of vignettes were selected.

Translation process

We will adhere to the European Organisation for Research and Treatment of Cancer (EORTC) guidelines [28] for the translation process as much as possible within the confines of the resources available for this study. All study documents (the website content, the PIS, the certificate of completion, and the vignettes) will firstly be written in English, verified by the UCL study team. The forward translation into the languages of all countries participating in this study (English, German, Italian, and Dutch) will be completed independently by two translators per language. The translators will compare and form one translated set of documents. The documents will then be back translated to English by a third translator who was not involved in the forward translation. The UCL study team will then review the original version with the back translated version and any differences will be discussed between the translators until the differences have been resolved.

Outcomes

The primary outcomes will be the continuous estimates of probability of dying within the next 12 months for the SQ (0–100%).

The secondary outcomes will be:

  1. 1)

    Dichotomous response to the SQ (Yes/No)

  2. 2)

    Dichotomous response to the alternative SQ (Yes/No)

  3. 3)

    The options for the course of action selected by the participants in each vignette

Participant timeline

It is expected that participation should take no longer than 30 min. However, due to the busy work schedules of the participants, they will have the option to “log out” of the study and return at their convenience. This methodology has been used previously and shown to help recruitment as well as minimise attrition [29, 30] Should they not complete the study on the first assessment, they will be sent a reminder email 7 days after they started the study and a final one 3 weeks after starting the study (see Table 1 for the participant timeline). Once recruitment has started, the website will remain open until the end of the study period or until the recruitment target has been met.

Table 1 Participant Timeline
Full size table

Sample size

Twenty vignettes have been developed. This number of vignettes was estimated in order to keep the task burden to a minimum, to reduce attrition, and to collect enough data points to establish the threshold score for each participant.

We aim to recruit the target sample size of 600 doctors (100 per country) within 8 months of the study opening. This timeframe should be sufficient to allow GPs, with busy working schedules, to complete the study. The sample size is based on calculating the trigger probability to an acceptable level of precision. As there is no current evidence about the trigger probability level for individuals, we assumed a trigger probability of 50%. Using this conservative estimate, aiming at a 4% margin of error (equivalently a precision of 8%), with a level of confidence of 95%, we will need to recruit 600 participants. From previous research that approached a membership body of doctors to participate [31], we anticipate a response rate of approximately 10–15% and therefore will aim to approach 1000–1500 doctors per country.

Recruitment

The method of recruitment will vary slightly by country in order to accommodate national variations in the required approvals. Participants will be contacted via one of the following methods:

  1. 1)

    An email or receipt of study information within a newsletter from a membership affiliation email

  2. 2)

    An email or word of mouth from a colleague who has participated/heard of the study (snow-balling)

In the UK for example, we could use Local Medical Committees (LMCs), Clinical Commissioning Groups (CCGs) administrators, and the newsletters of the British Medical Association (BMA) and the Royal College of General Practitioners (RCGP) to disseminate a link to the website to participants. In addition to this, we will ask any GP who participates to share the study link with eligible colleagues.

Methods: data collection, management, and analysis

Outcome data collection

Demographic and biographical information about the participants will be collected (location, job title, specialty, setting, age, gender, years of experience since qualifying), and information about their use of the SQ (frequency of use, confidence).

After each vignette, the participant will be asked the following four questions in order to collect data for the primary and secondary outcomes:

  1. 1.

    Would you be surprised if this patient were to die in the next 12 months? (Y/N) (The Surprise Question)

  2. 2.

    Would you be surprised if this patient were to remain alive after 12 months? (Y/N) (The alternative surprise question)

  3. 3.

    What do you think the probability is of this patient dying within the next 12 months? 0% (Certain survival) - 100% (Certain death)

  4. 4.

    What do you think this patient needs? (select more than one if appropriate)

This final question (Q4) will contain a list of potential treatment or care options. The precise wording of these questions will be developed and agreed with the European collaborators to make sure that the questions are culturally and linguistically meaningful.

Participant retention

At the completion of the study, participants will be able to download a certificate of participation. Once the study has closed, they will receive feedback on their performance (if they requested this on the consent page). It is hoped that these options, as well as the option to log out of the website and return at a later date, will promote participant recruitment and retention.

Data management

The database will be designed by a database specialist (CT). The database will be rigorously tested by the UCL study team (NW, LO, VV, & PS) prior to the study starting.

Data collection tools

The study is accessed by the participants online and as such, the data will be collected directly from participants.

Each participant will be able to complete a practice vignette prior to completing the main set of vignettes in order to familiarise themselves with the online environment. There will be no time limit for completion so that each participant will be able to take their own time. Participants will not be able to move on to the next page if required information is missing or if responses have been submitted in an incorrect format (e.g. incorrect email format). The UCL study team and the statistician will review the data being collected before, during, and at the start of the recruitment phase to ensure data integrity.

Statistical methods

A detailed statistical analysis plan will be drawn up in a separate document prior to analysis.

Primary outcome analysis

This is an exploratory study to determine how the SQ is utilised by different GPs in different countries. For this reason, we will employ a per-protocol analysis where those participants who do not complete more than 15 vignettes or violate the protocol (e.g. putting the same answer for every vignette) will be removed from the analysis.

We will examine each participant’s responses to the 20 vignettes to calculate the ‘trigger probability’ (or ‘threshold’) for each individual. To calculate this threshold we will examine the participant’s responses to the SQ and the responses to the probability of dying in the next 12 months. We define the threshold as the probability level after which all responses to the SQ are ‘no’. We will also describe the level of uncertainty about this decision. This uncertainty will be calculated by looking at the difference between the lowest probability level with which the participant responded ‘no’ to the SQ and the threshold level.

Three example GP responses are described below to explain this further.

GP 1 responded to 20 vignettes. For each vignette the GP stated the probability of dying and a corresponding answer to the SQ (yes/no). The probability results ranged from 0 to 100%. With each probability the GP also stated ‘yes/no’ to the questions (for ease of display, the individual probability estimates have not been provided but the ‘Y’/‘N’ responses have been placed over the corresponding % to show the point at which the responses changed from Y to N):

In Fig. 2, the probability threshold would be 60%. This GP consistently said ‘no’ to the SQ if the probability of dying was greater than 60%. Consequently the threshold would be 60% and the uncertainty would be 0%.

Fig. 2
figure 2

GP 1’s responses

Full size image

GP 2 also responded to 20 vignettes.

In Fig. 3, the lowest probability level with which the GP responded to ‘no’ on the SQ was 30%. After 60% the GP consistently said ‘no’ on the SQ. Consequently the threshold is 60% but their uncertainty is 30% (= 60–30%).

Fig. 3
figure 3

GP 2’s responses

Full size image

GP 3 also responded to 20 vignettes.

In Fig. 4, the lowest probability for which the GP answered “no” to the SQ was 10%. After 95% the GP consistently said “no”. Consequently the threshold was 95% and the level of uncertainty was 85% (= 95–10%).

Fig. 4
figure 4

GP 3’s responses

Full size image

Once the thresholds are calculated for all participants, we will use linear regression to explore the differences, if any, in thresholds between countries and other demographic detail collected.

Secondary outcome analysis

We will explore the relationship between the alternative SQ and the other responses. We will also describe the actions taken by the clinicians and explore the differences between groups.

Methods: monitoring

Throughout the study, the research team will review recruitment figures and the study will close once each country has recruited 100 participants. Researchers at UCL will check the responses given by the participants to assess for compliance with the protocol. Participants may be excluded from the analysis if their response record strongly suggests that they did not comply with the study protocol (e.g. all items answered with the same response). The UCL Study Management Group (PS, LO, NW, CT, VV) will be responsible for overseeing the trial and will meet regularly (at least four times per year) to review recruitment figures.

This is a very low risk study. There are no expected side effects of our intervention and this study will not have a Data Monitoring Committee.

Data management & dissemination

Confidentiality

All data will be handled in accordance with the General Data Protection Regulation (GDPR) 2018 [32].

The study will ask the participants for their name and email address as a personal identifier as well as being assigned a participant ID. The purpose of this is to:

  1. 1.

    Enable the participant to log out and back in to the series of vignettes at the same place

  2. 2.

    To contact participants who do not complete the study with a reminder email at the one week and 3 week time points after the last website visit.

The participants will not be asked for any other personal identifiable information. During the study, all data will be kept securely on a web-based database, which is encrypted and password protected. The database will be accessible to approved members of the UCL research team only. Once all data have been reviewed, the names and email addresses will be retained for 12 months after publication, at which point all personal information will be destroyed and only the participant ID will be referenced. This will be explained clearly in the PIS and consent will be sought.

Access to the data

In the study, demographic data and outcome data will be collected from participants in accordance with the PIS and this protocol.

The final study database will be downloaded from the website by the research team (Marie Curie Palliative Care Research Department, Division of Psychiatry, UCL) for statistical analysis and UCL will act as the data controller of such data for the study.

The research team (Marie Curie Palliative Care Research Department, Division of Psychiatry, and UCL) will process, store and dispose of the final study database in accordance with all applicable legal and regulatory requirements, including the General Data Protection Regulation (GDPR) and any amendments thereto. Data will be stored electronically, on an encrypted hard drive, with restricted access. The hard drive is maintained by UCL and routinely backed up.

The anonymous dataset will be shared with the other research groups participating in the study.

Record keeping and archiving

At the end of the study, all essential documentation will be archived securely by the CI for a minimum of 20 years from the declaration of end of study.

Essential documents are those which enable both the conduct of the study and the quality of the data produced to be evaluated and show whether the site complied with all applicable regulatory requirements.

The sponsor will notify the site when study documentation can be archived. All archived documents will be continued to be available for inspection by appropriate authorities upon request.

Dissemination policy

Study results will be published in peer-reviewed, indexed, journals using an open access format, and the results will be presented at national and international conferences. Authorship eligibility will be in accordance with The International Committee of Medical Journal Editors. All proposed publications will adhere to UCL publication policy.

Marie Curie supports this study and we will work with them to disseminate findings through blogs, newsletters and social media.

Discussion

Four members of the Marie Curie Expert Research Voices Group have reviewed a summary of this study to ensure its aims, objectives, study design and outputs are in line with patient and family needs. All respondents felt it was important to identify patients who may require palliative care early, and they felt the Surprise Question could be a useful way to get doctors to think about the sorts of patients who may or may not require palliative care. Other comments included respondents’ own experiences when a loved one died. These comments were taken in to consideration when developing the study documents.

It is expected that the recruitment of GPs will be challenging due to the nature of their work. We further anticipate that attrition may be an issue due to time constraints on the participants. We have added a function for the doctor to “log out” of the study and return at a more convenient time. We have also endeavoured to minimise the number of vignettes. In addition to this, we will keep the study open as long as possible to try and maximise recruitment. We have adopted recruitment strategies that have proven to be successful in another study which also recruited GPs from the UK [33].

Change history

  • 27 January 2020

    Following publication of the original article [1], an error in reference 18 was reported.

Abbreviations

BMA:

British Medical Association

CCG:

Clinical Commissioning Groups

GDPR:

General Data Protection Regulation

GP:

General Practitioner

LMC:

Local Medical Committees

PIS:

Participant Information Sheet

RCGP:

Royal College of General Practitioners

SQ:

Surprise Question

UCL:

University College London

UK:

United Kingdom

References

  1. Glare P, Virik K, Jones M, Hudson M, Eychmuller S, Simes J, Christakis N. A systematic review ofphysicians' survival predictions in terminally ill cancer patients. BMJ. 2003;327:195.

  2. Glare PA, Sinclair CT. Palliative medicine review: prognostication. J Palliat Med. 2008;11(1):84–103.

    Article  Google Scholar 

  3. Maltoni M, Caraceni A, Brunelli C, Broeckaert B, Christakis N, Eychmueller S, Glare P, Nabal M, Viganò A, Larkin P, et al. Prognostic factors in advanced Cancer patients: evidence-based clinical recommendations—a study by the steering Committee of the European Association for palliative care. J Clin Oncol. 2005;23(25):6240–8.

    Article  Google Scholar 

  4. White N, Reid F, Harris A, Harries P, Stone P. A systematic review of predictions of survival in palliative care: how accurate are clinicians and who are the experts? PLoS One. 2016;11(8):e0161407.

    Article  Google Scholar 

  5. Zhou M, Holden L, Lao N, Lam H, Zeng L, Chow E. Accuracy of clinicians’ prediction of survival and prognostic factors indicative of survival: a systematic review. Hong Kong Journal of Radiology. 2013;16(3):168–82.

    Article  Google Scholar 

  6. Gibbins J, McCoubrie R, Maher J, Wee B, Forbes K. Recognizing that it is part and parcel of what they do: teaching palliative care to medical students in the UK. Palliat Med. 2010;24(3):299–305.

    Article  CAS  Google Scholar 

  7. Henson L, Gao W, Higginson I, Smith M, Davies J, Ellis-Smith C, Daveson B. Emergency department attendance by patients with cancer in the last month of life: a systematic review and meta-analysis. Lancet. 2015;385:S41.

    Article  Google Scholar 

  8. Seow H, Brazil K, Sussman J, Pereira J, Marshall D, Austin PC, Husain A, Rangrej J, Barbera L. Impact of community based, specialist palliative care teams on hospitalisations and emergency department visits late in life and hospital deaths: a pooled analysis. BMJ. 2014;348:3496.

  9. El-Jawahri A, Greer JA, Temel JS. Does palliative care improve outcomes for patients with incurable illness? A review of the evidence. J Support Oncol. 2011;9(3):87–94.

    Article  Google Scholar 

  10. Higginson IJ, Evans CJ. What is the evidence that palliative care teams improve outcomes for Cancer patients and their families? The Cancer Journal. 2010;16(5):423–35.

    Article  Google Scholar 

  11. Higginson IJ, Finlay IG, Goodwin DM, Hood K, Edwards AGK, Cook A, Douglas H-R, Normand CE. Is there evidence that palliative care teams Alter end-of-life experiences of patients and their caregivers? J Pain Symptom Manag. 2003;25(2):150–68.

    Article  Google Scholar 

  12. Hately J, Laurence V, Scott A, Baker R, Thomas P. Breathlessness clinics within specialist palliative care settings can improve the quality of life and functional capacity of patients with lung cancer. Palliat Med. 2003;17(5):410–7.

    Article  Google Scholar 

  13. Mitchell S, Loew J, Millington-Sanders C, Dale J: Providing end-of-life care in general practice: findings of a national GP questionnaire survey. Br J Gen Pract 2016:bjgpsep-2016-2066-2650-mitchell-fl-p.

  14. Abarshi E, Echteld M, Donker G, Van den Block L, Onwuteaka-Philipsen B, Deliens L. Discussing end-of-life issues in the last months of life: a nationwide study among general practitioners. J Palliat Med. 2011;14(3):323–30.

    Article  Google Scholar 

  15. The End of Life Care Strategy. In. Edited by Department of Health; 2008.

  16. Meier EA, Gallegos JV, Thomas LPM, Depp CA, Irwin SA, Jeste DV. Defining a good death (successful dying): literature review and a call for research and public dialogue. Am J Geriatr Psychiatry. 2016;24(4):261–71.

    Article  Google Scholar 

  17. Lynn J. Living long in fragile health: the new demographics shape end of life care. Hastings Cent Rep. 2005;35(7):s14–8.

    Article  Google Scholar 

  18. Lynn J. The utility and value of the “surprise question” for patients with serious illness. Can Med Assoc J. 2017;189(33):E1072–3.

    Article  Google Scholar 

  19. Downar J, Goldman R, Pinto R, Englesakis M, Adhikari NK. The “surprise question” for predicting death in seriously ill patients: a systematic review and meta-analysis. Can Med Assoc J. 2017;189(13):E484–93.

    Article  Google Scholar 

  20. White N, Kupeli N, Vickerstaff V, Stone P. How accurate is theSurprise question’ at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med. 2017;15(1):139.

    Article  Google Scholar 

  21. Mitchell GK, Senior HE, Rhee JJ, Ware RS, Young S, Teo PC, Murray S, Boyd K, Clayton JM. Using intuition or a formal palliative care needs assessment screening process in general practice to predict death within 12 months: a randomised controlled trial. Palliat Med. 2018;32(2):384–94.

    Article  Google Scholar 

  22. Elliott M, Nicholson C. A qualitative study exploring use of the surprise question in the care of older people: perceptions of general practitioners and challenges for practice. BMJ Support Palliat Care. 2014:bmjspcare–2014-000679.

  23. Veldhoven C, Weijers F, Verhagen S, Vissers KYE. Combining the original and alternative surprise question for the early identification of palliative patients: a Dutch pilot study. In: European Association of Palliative Care. Madrid. 2017:189. http://www.eapc-2017.org/files/EAPC17/dl/EJPC-Abstract-Book-2017.pdf.

  24. Weijers F, Veldhoven C, Verhagen C, Vissers K, Engels Y. Adding a second surprise question triggers general practitioners to increase the thoroughness of palliative care planning: results of a pilot RCT with cage vignettes. BMC palliative care. 2018;17(1):64.

    Article  CAS  Google Scholar 

  25. Hinkka H, Kosunen E, Lammi U-K, Metsänoja R, Puustelli A, Kellokumpu-Lehtinen P. Decision making in terminal care: a survey of Finnish doctors' treatment decisions in end-of-life scenarios involving a terminal cancer and a terminal dementia patient. Palliat Med. 2002;16(3):195–204.

    Article  CAS  Google Scholar 

  26. Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT). http://www.spirit-statement.org/. Accessed 1 Apr 2019.

  27. The Gold Standards Framework Proactive Identification Guidance (PIG). http://www.goldstandardsframework.org.uk. Accessed 1 Apr 2019.

  28. European Organisation for Research and Treatment of Cancer (EORTC): EORTC quality of life group translation procedure. In., 4th edn; 2017.

  29. Oostendorp L, White N, Harries P, Yardley S, Tomlinson C, Ricciardi F, Gokalp H, Stone P: 35 online randomised controlled trial to improve clinical estimates of survival (ORACLES): study design. BMJ Support Palliat Care 2018, 8(3):373–373.

  30. White N, Harries P, Harris AJL, Vickerstaff V, Lodge P, McGowan C, Minton O, Tomlinson C, Tookman A, Reid F, et al. How do palliative care doctors recognise imminently dying patients? A judgement analysis. BMJ Open. (in press).

  31. White N, Harries P, Harris AJ, Vickerstaff V, Lodge P, McGowan C, Minton O, Tomlinson C, Tookman A, Reid F et al: 44 an evidenced-based heuristics model (or rule of thumb) to improve doctors’ intuition about when patients are imminently dying. BMJ Support Palliat Care 2018, 8:376–376.

  32. General Data Protection Regulation (GDPR). https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/. Accessed 1 Apr 2019.

  33. Curtis P, Taylor G, Harris M. How preferred learning approaches change with time: a survey of GPs and GP specialist trainees. Education for Primary Care. 2018:1–6.

Download references

Acknowledgements

We would like to thank the members of the PPI group at Marie Curie Expert Voices for their invaluable feedback on the study documents.

Funding

This study is partially funded by the Marie Curie I-CAN-CARE Program grant (MCCC-FPO-18-U). Nicola White, Linda Oostendorp, Victoria Vickerstaff, and Patrick Stone are partly supported by the UCLH NIHR Biomedical Research Centre. There are no financial interests by the CI, PI, or study management members. University College London (UCL) is the sponsor. The sponsor will have no role in the design, collection, analysis, interpretation, or writing of this data.

Availability of data and materials

Not applicable.

Protocol version

3.0 2019.01.21

Author information

Authors and Affiliations

  1. University College London, London, UK

    Nicola White, Linda Oostendorp, Victoria Vickerstaff, Christina Avgerinou & Patrick Stone

  2. University of Mainz, Mainz, Germany

    Christina Gerlach

  3. Radboud University Medical Centre, Nijmegen, The Netherlands

    Yvonne Engels & Carel Veldhoven

  4. University Hospital of Bern, Bern, Switzerland

    Maud Maessen, Sofia Zambrano & Steffen Eychmüller

  5. Imperial College London, London, UK

    Christopher Tomlinson

  6. University of Antwerp, Antwerp, Belgium

    Johan Wens & Bert Leysen

  7. University of Bologna & Academy of the Sciences of Palliative Medicine, Bologna, Italy

    Guido Biasco

  8. University of Bologna, Bologna, Italy

    Rabih Chattat & Giovanni Ottoboni

Authors
  1. Nicola White
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linda Oostendorp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Victoria Vickerstaff
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christina Gerlach
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yvonne Engels
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maud Maessen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Christopher Tomlinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Johan Wens
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Bert Leysen
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Guido Biasco
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Sofia Zambrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Steffen Eychmüller
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Christina Avgerinou
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Rabih Chattat
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Giovanni Ottoboni
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Carel Veldhoven
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Patrick Stone
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The study will be led by the group based in the UK. There will be a group of lead researchers in each country who will manage the project within each country. United Kingdom: NW will create the vignettes and the online study, recruitment, analyse the data, and write up the results. LO will assist in the creation of the online study, recruitment, analyse the data, and write up the results. PS will oversee all actions of the study team. PS will oversee the analysis and will contribute to the final report. VV will provide support regarding the statistical analysis of the data and will review the statistical content within the final report. CT will build the online website and database which will hold the data gathered from the participants. CA will assist in the study design and will assist in the write up of the results. The researchers in all participating countries (CG, YE, MM, JW, BL, GB, SZ, SE, RC, GO, CV) helped to create the vignettes and the online study, the translation of study materials into each of the applicable languages, support with recruitment, and interpretation, write up and dissemination of the results. All authors have read and approved the manuscript.

Corresponding author

Correspondence to Nicola White.

Ethics declarations

Ethics approval and consent to participate

This study gained ethics approval from University College London REC on the 9th August 2018 (ref 8675/003), University of Antwerp REC on the 7th January 2019 (ref 18/50/589), University of Mainz REC on the 26th November 2018, Radboud University Medical Center REC on the 20th December 2018 (ref 2018–4949), Bern Ethics Committee on the 29th August 2018 (ref 2018–00710) and University of Bologna REC on the 25th January 2019 (ref 12,590). In case any protocol amendments are required, these will be reviewed by the study team before submission to the relevant committees, and the trial registry will be updated where necessary. Consent is obtained electronically through a series of check boxes, see section “Methods, Methods: Participants, interventions, and outcomes” for further detail.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

SPIRIT checklist. (PDF 172 kb)

Additional file 2:

Template for vignettes. (PDF 604 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

White, N., Oostendorp, L., Vickerstaff, V. et al. An online international comparison of thresholds for triggering a negative response to the “Surprise Question”: a study protocol. BMC Palliat Care 18, 36 (2019). https://doi.org/10.1186/s12904-019-0413-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12904-019-0413-x

Keywords

BMC Palliative Care

ISSN: 1472-684X

Contact us