Skip to main content
Download PDF

State-wide implementation of patient-reported outcome measures (PROMs) in specialized outpatient palliative care teams (ELSAH): A mixed-methods evaluation and implications for their sustainable use

BMC Palliative Care volume 21, Article number: 216 (2022) Cite this article

Abstract

Background

Such patient-reported outcome measures (PROMs) and patient-centered outcome measures as the Integrated Palliative Care Outcome Scale (IPOS), Phase of Illness, and IPOS Views on Care (IPOS VoC), facilitate patient-centered care and help improve quality. To ensure sustainability, implementation and usage should be adapted according to setting. When settings involve several distinct teams that differ in terms of views and working practices, it is more difficult to integrate outcome measures into daily care. The ELSAH study aimed to learn how health professionals working in specialized outpatient palliative care (SOPC) viewed the use of these outcome measures in daily care, and what they express is needed for successful sustainable, state-wide application.

Methods

We used a parallel mixed-methods design involving three focus groups (n = 14) and an online-survey based on normalization process theory (n = 76). Most participants were nurses and physicians from 19 SOPC-teams in Hesse, Germany. We used a triangulation protocol including convergence coding matrices to triangulate findings.

Results

The majority of health professionals were able to integrate the outcome measures into their working lives and said that it had become a normal part of their day-to-day work. To ensure their sustainable integration into daily care, the motivation and concerns of health professionals should be taken into consideration. Health professionals must clearly recognize how the measures help improve daily care and quality evaluation.

Conclusions

To implement the outcome measures in a number of teams, it will be necessary to take individual team characteristics into account, because they influence motivation and concerncs. Further, it will be necessary to offer opportunities for them to engage in peer support and share information with other teams. The sustainable use of outcome measures in SOPC will require continuous support within each team as well as across teams. When several distinct teams are working in the same setting, a cross-team coordination unit can help to coordinate their work efficiently.

Trial registration

German Clinical Trials Register DRKS-ID: DRKS00012421; www.germanctr.de/DRKS00012421

Peer Review reports

Background

In palliative care, patient-reported outcome measures (PROMs), patient-centered outcome measures and caregiver-reported outcome measures enable patients’ and families’ needs to be recognized and addressed [1]. Evidence that their use improves patient-relevant outcomes is growing [2]. Further, their use allows quality of care on a provider level to be improved through case-by-case evaluation and by monitoring care on a policy level [3].

In palliative care, a variety of outcome measures exist. Some focus on specific domains e.g. on physical, social, spiritual or cultural domains [4]. Some consider special target groups like patient, relatives, health professionals or the healthcare system [5]. The European Association for Palliative Care (EAPC) recommends to use validated outcome measures, that include patients and relatives, and cover all relevant aspects of care [1]. Initiatives like the Australian Palliative Care Outcomes Collaboration (PCOC) and the Outcome Assessment and Complexity Collaborative (OACC) developed comprehensive approaches to evaluate palliative care [6, 7].

The implementation of outcome measures is complex and needs to be adapted to the setting in which it is used [8]. The Consolidated Framework for Implementation Research (CFIR) implies that successful implementation must take characteristics of the intervention, the implementation process, the individuals, as well as the inner and outer setting into account [9]. Implementation in a broad setting involving different healthcare professionals presents an additional challenge because of differing backgrounds, views, management and working practices. Nevertheless, a degree of standardization and guidance helps promote fidelity and improve data quality. This condition must be fulfilled if benchmarking and national and international care comparisons are to be possible [1].

Health professionals’ views must be taken into consideration if usage is to be sustainable. In a similar study, a negative attitude of health professionals to quality measurement reduced the chance of successful implementation [10]. Previous research has also shown that participants may refuse to use PROMs if they feel not to have enough time and training for use [11, 12]. Lind et al. found a general feeling of fatigue followed changes to routines in various palliative-care settings [13]. Implementation research has shown that if staff do not feel that a task is appropriate to their job responsibilities, rejection and reduced fidelity may result [14]. Implementation frameworks can help organize tasks and improve implementation [15]. Normalization Process Theory (NPT) focuses on the work that needs to be done to implement, embed, and sustain practices in different contexts. For this purpose, the theory considers the four core constructs coherence (how health professionals make sense of a new practice), cognitive participation (relational work to redesign the work), collective action (to sustain use), and reflexive monitoring (how the health professionals appraise the impact on work) [16]. This theory is therefore particularly suitable for studying the normalization of change in the work of teams, and considering the perspectives of involved health professionals. Therefore, NPT has been widely used in research to evaluate and understand implementation processes [17, 18].

Specialized outpatient palliative care (SOPC) provides comprehensive care to patients with life-limiting diseases and complex needs in their familiar surroundings [19]. The aim of our ELSAH study (‘Evaluation of Specialized Outpatient Palliative Care by taking the example of Hesse’) was to implement outcome measures in all SOPC-teams in Hesse, Germany. Before our study patient records were based on the German National Hospice and Palliative Care Registry and, although including data on structure and process quality, symptoms, treatment and support needs, they barely considered outcomes from the perspective of patients and relatives [20]. To find out more about day-to-day care and include a quality assessment from their perspective, we added patient-reported and caregiver-reported outcome measures [21].

In a first step, we examined key features of successful care from the perspective of patients, relatives and health professionals. We found that ‘treatment of complex symptoms, comprehensive care and a sense of security, as well as a focus on the quality of relationships, respect for individuality and the facilitation of self-determination’ [22] are relevant topics that need to be covered. On this basis, we assembled a set of validated outcome measures, which is based on the Outcome Assessment and Complexity Collaborative (OACC) suite of measures [6]. The set of outcome measures includes the Integrated Palliative care Outcome Scale (IPOS) [23], Phase of Illness [24], and IPOS Views on Care (IPOS VoC) [25]. In the next step, we examined the feasibility, acceptability and appropriateness of these outcome measures in a sample of five SOPC-teams. We found that they should be used considerately in order to minimize the burden on patients and relatives, and ensure their administration was manageable. It was also important that participants understood that the measures were useful [26].

However, it is not yet clear how the sustainable implementation of these outcome measures can take place in a broad SOPC setting involving different healthcare professionals, backgrounds, views, management and working practices. Because the health professionals are the ones who use the outcome measures, and their attitudes influence sustainable use, their perspective is decisive. Therefore, we evaluated the state-wide use of these outcome measures in daily care from the perspective of health professionals in this study.

Methods

Aim

We aimed to understand how health professionals working in SOPC viewed the use of outcome measures in daily care, and to determine what they express is needed for successful sustainable, state-wide application. We assumed that the implementation and application of outcome measures in daily care is a complex, dynnamic, and intersubjective process [27]. Therefore, we chose to use normalization process theory (NPT) as an explanatory framework in order to investigate embedding into routine care and to gain an insight into participants' experience of implementation [16].

Design

We used a parallel mixed-method design. Althought we first conducted the qualitative focus groups and then a semi-structured online-survey, we analyzed them in parallel due to organisational reasons [28]. We considered mixed-methods appropriate and used an online survey to gather information on the views of professionals, and focus group discussions to understand participants’ views in more in depth [29]. We followed Mixed Methods Article Reporting Standards (MMARS) (supplemental material A) [30].

Setting

We gradually implemented the outcome measures in daily care from November 2018 to June 2019 via the electronic documentation systems (EDS) of 19 SOPC-teams caring for adults in Hesse, Germany. In a further three teams, implementation could only take place after data collection due to delayed software adaptation. Hesse is a federal state in Germany containing rural and urban areas and about 6.3 million inhabitants [31]. In total, 22 SOPC-teams provide state-wide care for adults. Teams mainly include nurses and physicians, but in some cases also involve psychologists and social workers. All SOPC-teams belong to the Professional Association of Specialized Palliative Homecare in Hesse, which collects standardized data in order to improve quality of care [32].

We conducted a training meeting with every team on their premises before implementation. Topics were reasons for implementation, the content and practical use of the outcome measures, and organizational integration into daily care routines. After the training meeting the SOPC-team members applied the outcome measures in every case. Table 1 describes the outcome measures used. For the self-report, the SOPC-team members included the items into the conversations. Because we found in a previous study that it needs tact to address sensitive topics, we did not arrange predefined time points for self-report, but made the default to assess every topic when the information was collectible [26]. If a self-report was not possible, e.g. due to deteriorating health, the health professionals asked relatives or completed proxy-reports themselves.

Table 1 Outcome measures used
Full size table

Data collection

Focus groups

Two experienced qualitative researchers (HS, KK) conducted focus groups to investigate SOPC-team members’ experiences and suggested changes. We developed the focus group topic guide by including the NPT core constructs (supplemental material B). We brought members from different teams together in order to stimulate a discussion [34]. We sent an email invitation to our contact partner in every team and asked them to forward it to team members and to choose one to three team members, independent of profession, to participate in a focus group. We did not sample for specific characteristics, but included the people who had interest to participate (convenience sampling) [35]. Participants gave their written informed consent to participate, permitted a video recording to be made, and provided demographic data in a questionnaire. We wrote field notes on the researchers’ thoughts directly afterwards.

Online survey

We used lime-survey [36] to conduct an anonymous semi-standardized online survey, based on the NoMAD (Normalization MeAsure Development) instrument [37, 38]. Together with other authors, the authors of the NPT, May and Finch, developed the NoMAD tool, which aims to understand participants’ implementation experiences. This validated generic quantitative measure contains 43 items and has been used in numerous previous studies [39,40,41]. It includes statements related to the core constructs of NPT and asks participants to rate their agreement on a 5-point Likert scale (strongly agree to strongly disagree plus the option ‘not relevant to my role’ and the possibility to omit questions) [37].

In line with the developer’s recommendations, we adapted NoMAD to suit our research interest by adding both an introduction and questions on participants’ characteristics [38]. We further added a question on how relevant they considered each outcome measure and gave them the opportunity to add comments in free text (supplemental material C). We emailed a participation link to all team leaders, and asked them to forward it to all team members involved in using the outcome measures (convenience sampling).

Analysis

Focus groups

We transcribed and pseudonymized the audio recordings using MAXQDA [42]. May et al. presented several possibilities to analyze qualitative data in line with NPT [18]. We (HS, KK) used qualitative content analysis, and combined inductive and deductive coding using MAXQDA [43]. By setting the four core constructs of NPT (coherence, cognitive participation, collective action, reflexive monitoring) and their respective outcome measure as a-priori codes, we were able to link the content to the online-survey and identify issues associated with a specific measure. We discussed and added inductive codes and assigned them to the four core constructs of NPT after initial coding.

Online survey

As suggested by the authors of the NoMAD, we (HS, KK) used Microsoft Excel, version 2016, to analyze the online survey descriptively by looking at the response distribution and percentages [38]. We (HS, KK) used MAXQDA to analyze the free-text answers using qualitative content analysis [42]. We used the same coding tree, that we have elaborated for analysing the focus groups.

Mixed-Methods integration

We triangulated the findings of the focus groups and the survey on and interpretation level by using a triangulation protocol [44]. We created a convergence coding matrix for NPT core constructs and another convergence coding matrix for specific feedback on outcome measures. During the process, we examined the key quantitative and qualitative findings for convergences, dissonances, complementary information and silences, and developed an overarching conclusion for each component [44].

Deduction of topics requiring particular attention

In the analysis we identified topics, that reoccured in all NPT core constructs. We assumed that, from the participants' point of view, these issues need to be addressed if further use is to be successful. From this, we have deduced that these topics require particular attention for sustainable use of the outcome measures. Therefore, we present the deduction and the topics contents separately in this article.

Results

Study sample

We conducted three focus groups with n = 14 SOPC-team members. Five participants cancelled at short notice due to time constraints. Discussions took place in September and October 2019 in the rooms of an independent palliative association in Fulda and at the University of Frankfurt, and lasted about 105 min. The time between implementation and data collection was 3–10 months, depending on team.

The anonymous online-survey was conducted from November 20, to December 31, 2019. At this time the outcome measures had been used for six to twelve months. In the end, n = 76 complete data sets were available, and 104 hits were registered in total. Detailed results of the online-survey are presented in supplemental material C.

Participants’ characteristics are shown in Table 2.

Table 2 Characteristics of participants
Full size table

In the following, we present the general appraisal and results according to NPT. Subsequently, we show key themes requiring particular attention, including specific findings on outcome measures.

Appraisal of use in daily care

In the overall appraisal, about 69% of participants in the online-survey said the use of the outcome measures currently belonged to their day-to-day work. Participants that disagreed generally believed this would not be the case in the future (supplemental material C).

We present our findings according to the four core NPT constructs in Table 3. For each subject in the online-survey, we present a synopsis of online-survey results and focus group findings, provide a statement on the agreement between findings, and draw a conclusion. In Table 3, we also present a summary for each core construct of NPT.

Table 3 Convergence coding matrix for NPT core constructs
Full size table

Topics requiring particular attention

We found that four key topics recurred across all NPT constructs (Fig. 1). From the perspective of health professionals in SOPC these topics require particular attention if the revised documentation is to be used sustainably in daily care: 1) Daily care, 2) quality evaluation, 3) motivation and engagement, and 4) fears and concerns. We illustrate our findings using translations of pseudonymized quotations from the focus groups.

Fig. 1
figure 1

NPT core constructs leading to four key themes

Full size image

Daily care

Most participants knew the difference between the old and new documentation, but several participants (42%) did not see that the new documentation provided additional value in their practical work. Correspondingly, many participants (46%) did not feel the time for documentation was well spent.

The patient has always been taken into account to a great degree, but just not in the documentation. Now the documentation is written as if the patient really said all this, but everything is actually the same as before. (1738E, female nurse)

Most participants (71%) assumed, that everyone in their team could use the documentation in day-to-day care, but we found they had difficulties integrating it. Participants that felt they did not have sufficient resources at their disposal thought they had too little time for documentation.

Many participants (50%) thought the use of the outcome measures promoted cooperation with their fellow employees. While some mentioned the results in their team meetings, others did not look at them after entering the data. It became clear that not all participants knew how to integrate the results into their work, which may explain a lack of appreciation of their usefulness.

Quality evaluation

Many participants (53%) doubted whether quality could be evaluated through the use of documentation. This reflected a general concern whether outcome measures accurately reflected the actual care situation:

What I want to make comprehensible is that I can’t use a table to illustrate the way a person is, the emotions a person has, the process of dying in a person, or how a person deals with an illness. (1738E, female nurse)

Some participants struggled with the idea of an objective assessment. They criticized proxy assessments as having no validity because the assessment of health professionals can differ. They also criticized patient self-assessments on the grounds that patients’ conditions might influence their assessment.

But isn't it again a bit of a subjective judgment that varies from one colleague to another, when we go through the symptoms and I assess something? [...] So I look-, and of course ask about pain, but of course I assess what he tells me in terms of the different categories. My colleague might see things quite differently. How coherent is the picture then? 1736E, female nurse)

It became obvious that most participants (57%) could identify with the topics covered by the measures. However, many participants (43%) had no knowledge of reports or experiences of the usefulness of such measures, which may explain why they were sceptical about their usefulness.

Motivation and engagement

Only 41% of participants were keen to use the outcome measures in practice. Beyond the overall burden of documentation, the assumed benefit also influenced their engagement. Those that assumed care would benefit showed more interest in using it than those that did not.

Uncertainty in using the documentation in practice may also have had a negative impact on motivation. Most participants had dealt with the topics theoretically (86%) and felt they knew how the outcome measures should ideally be used (83%), but the discussions showed that some details were unclear. For example, they were particularly unsure how to address IPOS questions on psychosocial issues.

Yes, because how do I interpret the question, right? ‘At peace with myself’. Hmm. I have cancer, shit, I'm going to die. How can you be at peace with yourself ... That's just an expression covering one, two questions, or three, four (laughs), that's actually something everybody has a hard time with, all of us. (1742E, female nurse)

We found that sharing information across teams and team-related feedback helped motivate them. Most online-survey participants agreed that sufficient support was provided during implementation (63%), and at the time of the online-survey (60%), and they agreed that the management team provided sufficient support (72%). Nevertheless, participants in the focus groups wished for more continuous training and regular support from experts in the use of outcome measures.

Fears and concerns

Some participants (43%) expressed concern about a negative impact on care. They were concerned they would have less time for patients because they would have to invest time in completing the documentation. They also feared data might be misinterpreted, and that their work would be poorly evaluated, even though they were doing a good job in practice:

The Phase of Illness assessment, that's where the ‘stable’ bothers me. [...] But the thing is always: is he stable BECAUSE he has SOPC? [...] Stable actually means: does he still have an indication for SOPC? [...] And that’s when I ask myself: will this be used against us at some point? If we use that in the evaluation for the [health] insurers, for the funding agencies, will that be used against us? [...] Is that stable at some point the time they say ‘stable, stable, and you’re providing SOPC!’? (1745E, female physician)

Several participants (48%) denied that the use increased the quality of relationships with patients and relatives. Participants reckoned the quality of relationships depended more on personal interactions than outcome measurements. They also feared that the quality of relationships might deteriorate because patients might find some of the items upsetting:

But I find it very difficult to ask such questions because I’m then rather afraid that they might close themselves off completely. They are at home; we are already intruding on their intimacy, and so I think we should show a great deal of tact. (1744E, female nurse)

Specific feedback on the outcome measures

Participants regarded IPOS, and Phase of Illness symptom documentation as most helpful in their day-to-day care. They struggled with the psychosocial questions asked in IPOS (anxiety, family anxiety, depression, feeling at peace, share feelings). Although most participants considered the content to be relevant to care, several participants found it difficult to integrate the questions into a conversation, and to classify the answers on the scale. Participants said it was often impossible to interpret answers to such questions at the beginning of care because trust first had to be built up before such topics could be addressed. These issues could therefore not be raised when care was only provided for a short period.

Some participants criticized the wording of the questions. They also said it was difficult for staff members to make proxy assessments and almost impossible with cognitively impaired patients. Most participants appreciated the usefulness of IPOS VoC in evaluating care, but not in their practical work. Detailed feedback on the outcome measures is presented in Table 4.

Table 4 Convergence coding matrix regarding specific feedback on the outcome measures
Full size table

Discussion

Main findings

Overall, participants said that they had accepted the use of the outcome measures in daily care and that it had become part of their day-to-day work. However, participants expressed that sustainable integration into daily care will require that special attention is drawn to their usefulness in daily care, in quality evaluations, in motivating and engagement and appreciating the concerns of health professionals.

Comparison of findings with those reported in the literature

Feedback from participants in our study was heterogeneous, suggesting that the effectiveness of the implementation varied. While implementation worked well in some teams, this was not the case in other teams, suggesting that it is more difficult to implement in different teams than in individual teams. Top-down implementation might reduce the sense of accountability and possibly even lead to resistance when compared to bottom-up implementation [15]. Although the implementation by our research team was carried out in consultation with the Professional Association of Specialized Palliative Homecare in Hesse and thus with all SOPC-teams, it can be assumed that individual health professionals did not feel they had been included in the decision-making process.

In addition to the initial implementation, we found that the way the measures are used by the teams in daily care influences sustainability. Other studies found that supportive leadership and peer support facilitated the implementation and encouraged its further use [45, 46]. Bradshaw et al. said that the benefits need to be demonstrated in order to increase motivation and engagement and to promote and sustain a collaborative effort, adding that measures may otherwise be dismissed as pointless [47]. Another study agreed that good training and guidance is required in the use of PROM in care [48]. It became clear that training before and during implementation is not enough, but that continuous support, which should be adapted to the needs of individual teams, should be provided to users.

Sustainable motivation requires that results are interpreted and fed back to health professionals [46]. Our participants distinguished between benefits in their practical work and benefits in terms of quality assurance, but for its sustainable use it is important to understand both aspects. Use in daily care, e.g. in team discussions, may clarify the direct benefits in practical work and for individual patients. Similar to our findings, another study showed that health professionals appreciated the opportunity to recognize unmet needs through the electronic use of PROMs, but were also concerned that it could disrupt care delivery processes, especially when intimate questions were asked of patients with whom the duration of contact had been only short [49]. Other research, however, showed that psychosocial support is also possible on first contact [50]. On specific outcome measures, further feedback from our participants was in line with the results of similar studies [47, 51]. This shows the need for ongoing training, particularly on how to integrate the psychosocial questions of IPOS into daily care. Peers with experience of using the measures could regularly address issues surrounding integration and other problems in the team. By ensuring they were accessible to colleagues as low-threshold contact persons, they could also help train new staff. Cross-team information sharing amongst peers could improve the promotion of long-term, sustainable use. Implementation in several teams thus also offers opportunities for mutual support.

To enhance a good quality of nursing documentation, Groot et al. recommended to use familiar terms [52]. Following this recommendation, we used the term documentation in conversations with SOPC-team members about the outcome measures. Looking back, we realized that the term has negative associations and often worries health professionals because they do not see documentation as a supportive component of care, but rather as something separate. In further implementations, therefore other terms like patient- and care-relevant outcomes should be used, in order to underline the relevance for care and the patients and relatives and to strenghten the motivation for use.

We found concerns about negative consequences to be fundamental but related more to the health care system than to the outcome measures. For example, although the participants of our study considered the content of the outcome measure ‘Phase of Illness’ important, they also saw it as a potential threat. They were afraid the health care system might force them to exclude patients from SOPC, if the measure ‘Phase of Illness’ was recorded as stable. This critical attitude may result from our participants having dealt with quality assessment for some time, as this may have given them the impression that control by a higher authority (the health system), as reflected in more and more assessments, is increasing. Krawczyk et al. stated that health professionals rather focus on the micro- and meso-level in daily care, so the macro perspective needs to be explained to them [53]. Although this might reduce scepticism, individual teams may not have the resources to do this. A community of several teams could concentrate resources on key events and address these issues together. Further, internal benchmarking of teams within their community could enable best practices and areas of improvement to be identified, without producing fears of control [54].

Strengths and limitations

The use of normalization process theory helped us focus on the relevant aspects of integration into daily care. The mixed-methods design offered the opportunity to gain a multi-perspective insight [38]. If we had used a sequential design with the online survey first, we could have considered the results in the focus groups, but this was not possible for organizational reasons. However, the online-survey enabled low-threshold participation, and anonymous participation may have made it easier for participants to make critical statements. The focus groups allowed a deeper understanding of the participants’ views.

For data protection purposes, we sent the invitations to participate to the team leaders and asked them to forward them to team members. Gatekeeping by the team leaders may therefore have influenced the results. Some participants could not participate because of their work obligations, or excessively long journeys to focus groups. In our focus group just two physiscians participated, but in total an adequate number of physicians could be recruited, so that the physician’s perspective could also be integrated. Furthermore, we did not include patients in this evaluation phase. However, it became apparent that team members were strongly patient-oriented and tried to call attention to their needs.

Implications

We identified certain factors that may help promote the sustainable integration of PROMs and caregiver reported outcome measures in daily care. In addition to findings relating to the use of specific outcome measures, we also present measures that collaborating teams should use collectively. These results are probably transferable to similar home-care settings that are responsible for caring for adults. The transfer of results to the care of children and adolescents, or to inpatient palliative care, may be limited due to the differing needs of such patients and the different structure of their daily care [55]. We could only derive implications for a sustainable application from the feedback of SOPC-team members after a short period of application. Further research should focus on applying and evaluating strategies for sustainability and integrate the perspectives of patients and relatives.

The use of PROM as a basis for funding and accreditation has raised fears among our participants. They feared that, for example, higher compensation could be paid if the outcome measures were rated well and deductions could be made if they were rated badly. This should be carried out with caution, as stakeholders have differing interests and there is no clear evidence for its appropriateness in this regard [53]. However, we recommend internal use for comparisons with other teams and for internal quality assurance. Further research is needed on how results should be fed back to health professionals.

It became obvious that not all participants felt that the use of the outcome measures had or should become part of their day-to-day work. Previous research has shown that the relationship between implementers and users influences the success of an implementation. It may be helpful for similar implementation projects to make peers responsible for implementation, as their expertise may be better accepted than that of external staff [56]. Furthermore, it became clear that the measures should only be implemented over the long term, as the process is a continuous one that requires constant support and further development, along with appropriate resources.

Conclusion

The sustainable integration of outcome measures into daily care will require that particular attention is paid to responding to the motivation and concerns of health professionals and making them aware of the value of the measures in daily care and quality evaluation. Combining the implementation of PROMs in a number of SOPC-teams is more complicated than implementation in a single team because teams’ specific characteristics must be taken into account and a comparable basis created. At the same time, combined implementation offers opportunities for mutual support and the pooling of resources. The provision of support by experienced peers within and across teams will enable concerns to be addressed and benefits in daily care to be explained on a low-threshold level. Centrally organized information sharing across teams is a suitable means of communicating the benefits of PROMs in quality evaluation, and, through the efficient use of resources, can help complex topics to be addressed. Both strategies can promote motivation and allay concerns. However, the task is ongoing and requires staff and time.

Data availability

All online-survey data generated or analyzed during this study are included in this published article and its supplementary information files. As the focus group transcripts may provide enough information to allow participants to be identified, no original data will be published, but online supplemental material on data collection is provided.

Abbreviations

CFIR:

Consolidated Framework for Implementation Research

DRKS:

German Clinical Trials Register (‚Deutsches Register Klinischer Studien‘)

EAPC:

European Association for Palliative Care

EDS:

Electronic documentation system

ELSAH:

Evaluation of Specialised Outpatient Palliative Care by taking the example of Hesse

HP:

Health professional

IPOS:

Integrated Palliative Outcome Scale

IPOS VoC:

IPOS Views on Care

NoMAD:

Normalization MeAsure Development

NPT:

Normalization Process Theory

OACC:

Outcome Assessment and Complexity Collaborative

PCOC:

Palliative Care Outcomes Collaboration

PROM:

Patient reported outcome measure

SOPC:

Specialized outpatient palliative care

MMARS:

Mixed Methods Article Reporting Standards

ICMJE:

International Committee of Medical Journal Editors

References

  1. Bausewein C, Daveson BA, Currow DC, Downing J, Deliens L, Radbruch L, et al. EAPC White Paper on outcome measurement in palliative care: Improving practice, attaining outcomes and delivering quality services - Recommendations from the European Association for Palliative Care (EAPC) Task Force on Outcome Measurement. Palliat Med. 2015;30:1–17. https://doi.org/10.1177/0269216315589898.

    Article  Google Scholar 

  2. Dudgeon D. The impact of measuring patient-reported outcome measures on quality of and access to palliative care. J Palliat Med. 2018;21:S76–80. https://doi.org/10.1089/jpm.2017.0447.

    Article  PubMed  Google Scholar 

  3. van Vliet LM, Harding R, Bausewein C, Payne S, Higginson IJ. How should we manage information needs, family anxiety, depression, and breathlessness for those affected by advanced disease: development of a clinical decision support tool using a delphi design. BMC Med. 2015;13:263. https://doi.org/10.1186/s12916-015-0449-6.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Aslakson R, Dy SM, Wilson RF, Waldfogel JM, Zhang A, Isenberg SR, et al. Assessment Tools for Palliative Care. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2017.

  5. Stiel S, Pastrana T, Balzer C, Elsner F, Ostgathe C, Radbruch L. Outcome assessment instruments in palliative and hospice care—a review of the literature. Support Care Cancer. 2012;20:2879–93.

    Article  PubMed  Google Scholar 

  6. Witt J, Murtagh FEM, de Wolf-Linder S, Higginson IJ, Daveson BA. Introducing the Outcome Assessment and Complexity Collaborative (OACC) Suite of Measures: A Brief Introduction. 2014. https://www.kcl.ac.uk/cicelysaunders/attachments/Studies-OACC-Brief-Introduction-Booklet.pdf. Accessed 12 Jul 2021.

  7. Eagar K, Watters P, Currow DC, Aoun SM, Yates P. The Australian Palliative Care Outcomes Collaboration (PCOC)–measuring the quality and outcomes of palliative care on a routine basis. Aust Health Rev. 2010;34:186–92. https://doi.org/10.1071/AH08718.

    Article  PubMed  Google Scholar 

  8. Antunes B, Harding R, Higginson IJ. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med. 2014;28:158–75. https://doi.org/10.1177/0269216313491619.

    Article  PubMed  Google Scholar 

  9. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sci. 2009. https://doi.org/10.1186/1748-5908-4-50.

    Article  Google Scholar 

  10. Leemans K, van den Block L, Vander Stichele R, Francke AL, Deliens L, Cohen J. How to implement quality indicators successfully in palliative care services: perceptions of team members about facilitators of and barriers to implementation. Support Care Cancer. 2015;23:3503–11. https://doi.org/10.1007/s00520-015-2687-8.

    Article  PubMed  Google Scholar 

  11. Daveson BA, Simon ST, Benalia H, Downing J, Higginson IJ, Harding R, Bausewein C. Are we heading in the same direction? European and African doctors’ and nurses’ views and experiences regarding outcome measurement in palliative care. Palliat Med. 2012;26:242–9. https://doi.org/10.1177/0269216311409614.

    Article  CAS  PubMed  Google Scholar 

  12. Nguyen H, Butow P, Dhillon H, Sundaresan P. A review of the barriers to using Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs) in routine cancer care. J Med Radiat Sci. 2021;68:186–95. https://doi.org/10.1002/jmrs.421.

    Article  PubMed  Google Scholar 

  13. Lind S, Sandberg J, Brytting T, Fürst CJ, Wallin L. Implementation of the integrated palliative care outcome scale in acute care settings - a feasibility study. Palliat Support Care. 2018;16:698–705. https://doi.org/10.1017/S1478951517001158.

    Article  PubMed  Google Scholar 

  14. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76. https://doi.org/10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  15. Ogden T, Fixsen DL. Implementation Science. Zeitschrift für Psychologie. 2014;222:4–11. https://doi.org/10.1027/2151-2604/a000160.

    Article  Google Scholar 

  16. May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43:535–54. https://doi.org/10.1177/0038038509103208.

    Article  Google Scholar 

  17. McEvoy R, Ballini L, Maltoni S, O’Donnell CA, Mair FS, MacFarlane A. A qualitative systematic review of studies using the normalization process theory to research implementation processes. Implement Sci. 2014;9:2. https://doi.org/10.1186/1748-5908-9-2.

    Article  PubMed  PubMed Central  Google Scholar 

  18. May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13:80. https://doi.org/10.1186/s13012-018-0758-1.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Gemeinsamer Bundesausschuss. Richtlinie des Gemeinsamen Bundesausschusses zur Verordnung von spezialisierter ambulanter Palliativversorgung: SAPV-RL; 25.06.2010.

  20. Deutsche Gesellschaft für Palliativmedizin. Nationales Hospiz- und Palliativregister: Informationen zum Nationalen Hospiz- und Palliativregister. 2019. https://www.dgpalliativmedizin.de/images/19_10_01_Informationsbrosch%C3%BCre_NHPREGISTER_V7.pdf. Accessed 5 Jul 2022.

  21. Kuss K, Seipp H, Becker D, Bösner S, Erler A, Gruber D, et al. Study protocol: evaluation of specialized outpatient palliative care in the German state of Hesse (ELSAH study) - work package I: assessing the quality of care. BMC Palliat Care. 2018;17:111–9. https://doi.org/10.1186/s12904-018-0363-8.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Seipp H, Haasenritter J, Hach M, Becker D, Ulrich L-R, Schütze D, et al. How can we ensure the success of specialised palliative home-care? A qualitative study (ELSAH) identifying key issues from the perspective of patients, relatives and health professionals. Palliat Med. 2021;35:1844–55. https://doi.org/10.1177/02692163211026516.

    Article  PubMed  Google Scholar 

  23. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Qual Saf Health Care. 1999;8:219–27. https://doi.org/10.1136/qshc.8.4.219.

    Article  CAS  Google Scholar 

  24. Eagar K, Gordon R, Green J, Smith M. An Australian casemix classification for palliative care: lessons and policy implications of a national study. Palliat Med. 2004;18:227–33. https://doi.org/10.1191/0269216304pm876oa.

    Article  PubMed  Google Scholar 

  25. Addington-Hall J, Hunt K, Rowsell A, Heal R, Hansford P, Monroe B, Sykes N. Development and initial validation of a new outcome measure for hospice and palliative care: the St Christopher’s Index of Patient Priorities (SKIPP). BMJ Support Palliat Care. 2014;4:175–81. https://doi.org/10.1136/bmjspcare-2012-000352.

    Article  PubMed  Google Scholar 

  26. Seipp H, Haasenritter J, Hach M, Becker D, Schütze D, Engler J, et al. Integrating patient- and caregiver-reported outcome measures into the daily care routines of specialised outpatient palliative care: a qualitative study (ELSAH) on feasibility, acceptability and appropriateness. BMC Palliat Care. 2022;21:60. https://doi.org/10.1186/s12904-022-00944-1.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Snape D, Spencer L. The foundations of qualitative research. In: Ritchie J, Lewis J, editors. Qualitative research practice: a guide for social science students and researchers. London: Sage Publications; 2003. p. 1–23.

    Google Scholar 

  28. Kuckartz U. Mixed methods: methodologie, Forschungsdesigns und Analyseverfahren. Wiesbaden: Springer VS; 2014.

    Google Scholar 

  29. Tariq S, Woodman J. Using mixed methods in health research. JRSM Short Rep. 2013;4:2042533313479197. https://doi.org/10.1177/2042533313479197.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suárez-Orozco C. Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. Am Psychol. 2018;73:26–46. https://doi.org/10.1037/amp0000151.

    Article  PubMed  Google Scholar 

  31. Statistisches Bundesamt. Bevölkerung Deutschlands im Jahr 2020 erstmals seit 2011 nicht gewachsen. 21.06.2021. https://www.destatis.de/DE/Presse/Pressemitteilungen/2021/06/PD21_287_12411.html. Accessed 5 May 2022.

  32. Fachverband SAPV Hessen, Hach M. Handbuch Qualitätsmanagement in der spezialisierten ambulanten Palliativversorgung (SAPV). Frankfurt am Main: Mabuse; 2015.

  33. Cicely Saunders Institute. IPOS Views on Care. https://pos-pal.org/maix/ipos-views-on-care.php. Accessed 5 May 2022.

  34. Pohontsch NJ, Müller V, Brandner S, Karlheim C, Jünger S, Klindtworth K, et al. Gruppendiskussionen in der Versorgungsforschung – Teil 1: Einführung und Überlegungen zur Methodenwahl und Planung. [Group Discussions in Health Services Research - Part 1: Introduction and Deliberations on Selection of Method and Planning]. Gesundheitswesen 2017. https://doi.org/10.1055/s-0043-104379.

  35. Etikan I. Comparison of convenience sampling and purposive sampling. AJTAS. 2016;5:1. https://doi.org/10.11648/j.ajtas.20160501.11.

    Article  Google Scholar 

  36. Limesurvey GmbH. LimeSurvey: An Open Source survey tool. Hamburg: LimeSurvey GmbH; 2019.

  37. Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, et al. Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:133. https://doi.org/10.1186/s12874-018-0590-y.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:135. https://doi.org/10.1186/s12874-018-0591-x.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Gillespie BM, Harbeck E, Lavin J, Gardiner T, Withers TK, Marshall AP. Using normalisation process theory to evaluate the implementation of a complex intervention to embed the surgical safety checklist. BMC Health Serv Res. 2018;18:170. https://doi.org/10.1186/s12913-018-2973-5.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hafid A, Howard M, Guenter D, Elston D, Fikree S, Gallagher E, et al. Advance care planning conversations in primary care: a quality improvement project using the Serious Illness Care Program. BMC Palliat Care. 2021;20:122. https://doi.org/10.1186/s12904-021-00817-z.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Goodridge D, Rana M, Harrison EL, Rotter T, Dobson R, Groot G, et al. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Serv Res. 2018;18:237. https://doi.org/10.1186/s12913-018-3045-6.

    Article  PubMed  PubMed Central  Google Scholar 

  42. VERBI Software. Consult. Sozialforschung GmbH. MAXQDA 2018. Berlin; 1989 – 2019.

  43. Kuckartz U. Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung. 4th ed. Weinheim, Basel: Beltz Juventa; 2018.

  44. O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ (Clinical Research Ed). 2010;341:c4587. https://doi.org/10.1136/bmj.c4587.

    Article  PubMed  Google Scholar 

  45. Howell D, Rosberger Z, Mayer C, Faria R, Hamel M, Snider A, et al. Personalized symptom management: a quality improvement collaborative for implementation of patient reported outcomes (PROs) in “real-world” oncology multisite practices. J Patient Rep Outcomes. 2020;4:47. https://doi.org/10.1186/s41687-020-00212-x.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Pinto C, Bristowe K, Witt J, Davies JM, de Wolf-Linder S, Dawkins M, et al. Perspectives of patients, family caregivers and health professionals on the use of outcome measures in palliative care and lessons for implementation: a multi-method qualitative study. Ann Palliat Med. 2018;7:S137–50. https://doi.org/10.21037/apm.2018.09.02.

    Article  PubMed  Google Scholar 

  47. Bradshaw A, Santarelli M, Mulderrig M, Khamis A, Sartain K, Boland JW, et al. Implementing person-centred outcome measures in palliative care: an exploratory qualitative study using Normalisation Process Theory to understand processes and context. Palliat Med. 2021;35:397–407. https://doi.org/10.1177/0269216320972049.

    Article  PubMed  Google Scholar 

  48. Bausewein C, Simon ST, Benalia H, Downing J, Mwangi-Powell FN, Daveson BA, et al. Implementing patient reported outcome measures (PROMs) in palliative care–users’ cry for help. Health Qual Life Outcomes. 2011;9:27. https://doi.org/10.1186/1477-7525-9-27.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Radionova N, Becker G, Mayer-Steinacker R, Gencer D, Rieger MA, Preiser C. The views of physicians and nurses on the potentials of an electronic assessment system for recognizing the needs of patients in palliative care. BMC Palliat Care. 2020;19:45. https://doi.org/10.1186/s12904-020-00554-9.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Hill HC, Paley J, Forbat L. Observations of professional-patient relationships: a mixed-methods study exploring whether familiarity is a condition for nurses’ provision of psychosocial support. Palliat Med. 2014;28:256–63. https://doi.org/10.1177/0269216313499960.

    Article  PubMed  Google Scholar 

  51. Bausewein C, Schildmann E, Rosenbruch J, Haberland B, Tänzler S, Ramsenthaler C. Starting from scratch: implementing outcome measurement in clinical practice. Ann Palliat Med. 2018;7:S253–61. https://doi.org/10.21037/apm.2018.06.08.

    Article  PubMed  Google Scholar 

  52. de Groot K, Triemstra M, Paans W, Francke AL. Quality criteria, instruments, and requirements for nursing documentation: a systematic review of systematic reviews. J Adv Nurs. 2019;75:1379–93. https://doi.org/10.1111/jan.13919.

    Article  PubMed  Google Scholar 

  53. Krawczyk M, Sawatzky R, Schick-Makaroff K, Stajduhar K, Öhlen J, Reimer-Kirkham S, et al. Micro-meso-macro practice tensions in using patient-reported outcome and experience measures in hospital palliative care. Qual Health Res. 2019;29:510–21. https://doi.org/10.1177/1049732318761366.

    Article  PubMed  Google Scholar 

  54. Bausewein C, Daveson B, Benalia H, Simon ST, Higginson IJ. Outcome measurement in palliative care: the essentials. 2011. https://www.kcl.ac.uk/cicelysaunders/attachments/Guidance-on-Outcome-Measurement-in-Palliative-Care.pdf. Accessed 5 May 2022.

  55. Schütze D, Engler F, Ploeger C, Ulrich L-R, Hach M, Seipp H, et al. Specialised outpatient paediatric palliative care team-parent collaboration: narrative interviews with parents. BMJ Support Palliat Care. 2021. https://doi.org/10.1136/bmjspcare-2020-002576.

    Article  PubMed  Google Scholar 

  56. Metz A, Albers B, Burke K, Bartley L, Louison L, Ward C, Farley A. Implementation practice in human service systems: understanding the principles and competencies of professionals who support implementation. Hum Serv Organ Manag Leadersh Gov. 2021;45:238–59. https://doi.org/10.1080/23303131.2021.1895401.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to express our gratitude to all members of the SOPC-teams in Hesse for their participation and engagement. We would further like to thank Phillip Elliott for the linguistic revision.

Funding

Open Access funding enabled and organized by Projekt DEAL. The authors disclose receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Innovation fund (‘Innovationsfonds’) of the Federal Joint Committee of Germany [grant number: 01VSF16006]. The funder had no role in study design, data collection and analysis, interpretation, decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Department of General Practice and Family Medicine, Philipps-University of Marburg, Karl-Von-Frisch-Straße 4, 35032, Marburg, Germany

    Hannah Seipp, Jörg Haasenritter, Stefan Bösner & Katrin Kuss

  2. Professional Association of Specialized, Palliative Homecare in Hesse, Weihergasse 15, 65203, Wiesbaden, Germany

    Michaela Hach & Dorothée Becker

  3. Institute of General Practice, Goethe-University Frankfurt, Theodor-Stern-Kai 7, 60590, Frankfurt am Main, Germany

    Dania Schütze & Jennifer Engler

Authors
  1. Hannah Seipp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jörg Haasenritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michaela Hach
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dorothée Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dania Schütze
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jennifer Engler
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stefan Bösner
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Katrin Kuss
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MH, SB, and JH developed the project and secured project funding. HS, KK, JH, and SB collected, analyzed and interpreted the data. HS wrote this draft of the paper. MH and DB supported the recruitment of participants as well as the implementation. JE, DS, MH and DB assisted in the analysis and interpretation of data. All authors revised the manuscript and approved the final version. Authorship follows ICMJE recommendations.

Corresponding author

Correspondence to Hannah Seipp.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was granted by the Ethics Committee of the Faculty of Medicine, Philipps University Marburg (05–04-2018; ref. 47/18). The study was conducted according to the Declaration of Helsinki. Focus group participants gave written informed consent before their participation. At the beginning of the online-survey, we provided information on the purpose, participation, contact details and data protection. Then participants had to give digital consent before taking part. No patients participated in this study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12904_2022_1109_MOESM1_ESM.pdf

Additional file 1.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Seipp, H., Haasenritter, J., Hach, M. et al. State-wide implementation of patient-reported outcome measures (PROMs) in specialized outpatient palliative care teams (ELSAH): A mixed-methods evaluation and implications for their sustainable use. BMC Palliat Care 21, 216 (2022). https://doi.org/10.1186/s12904-022-01109-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12904-022-01109-w

Keywords

  • Palliative care
  • Home care services
  • Quality of health care
  • Patient reported outcome measures
  • Routinely collected health data
  • Implementation science
  • Qualitative research
  • Surveys and questionnaires

BMC Palliative Care

ISSN: 1472-684X

Contact us