Table 3 Convergence coding matrix for NPT core constructs

Coherence:

 → How do participants evaluate the informative value of the outcome measures?

 → Is there a common understanding of the purpose behind them?

Differentiation between old and new documentation

82% agreed that they knew the difference, 3% disagreed (12% neither)

Differences are known, but sometimes confusion occurs as to which changes our study is responsible for

Convergence

Participants appear to be aware of the differences between the old and new documentation

Shared understanding of the aim/purpose in the team

43% considered a shared understanding to exist, 37% did not (12% neither)

Some participants said they use the documentation because they are required to. Others thought the items were of value in their daily work

Convergence

There does not seem to be a shared understanding of the aim/purpose

Potential value in practice

37% considered the outcome measures to provide added value in their work, 42% do not (20% neither)

Some participants valued changes such as new response options and new items. Others considered the previous documentation to be sufficient

Complementarity (which changes participants welcomed and comparison to previous documentation)

Several participants saw no added value in practice. This may be because they considered the previous documentation to have been sufficient

Identification with the topics

57% could identify with the topics, 30% could not (9% neither)

Most participants considered the topics to be relevant to care. There was more discussion about whether the outcome measures captured the required information

Complementarity (concerns related more to measures than issues)

Most participants identified with the topics. Those that said they could not seemed to struggle with the measures rather than the topics

Modified documentation revealed changes in quality

26% believed the quality of care could be evaluated this way, 53% did not (18% neither)

Some participants generally doubted that quality can be revealed in documentation, and said quality could only be seen in practical interaction. Most participants disagreed that outcome measures could depict reality but could imagine that outcome measures might provide an indication of care quality

Complementarity (doubt on principle, but not necessarily related to the measures used here)

The majority of participants doubted that quality could be revealed through the use of the documentation. Participants seemed to doubt whether outcome measures can provide a real representation of care, rather than want to criticize the employed measures

Concern about negative impact on specialized outpatient palliative care

43% were concerned, 24% were not (25% neither)

Some participants feared their data might be misinterpreted and lead to a poor assessment of their work, whether or not they did a good job in practice. They were also concerned that time taken for documentation took time away from patients

Complementarity (what concerns participants have)

Several participants feared such documentation might have a negative impact on specialized palliative home-care. They feared the documentation of poor health in their patients might have a negative impact on the evaluation of their work and result in less time for patient care

Time for documentation is invested wisely in quality of care/patient care

30% agreed that the time was invested wisely, 46% disagreed (21% neither)

Some participants thought time spent with the patient was particularly valuable and important for the relationship with him or her. For participants that saw no added value for practical care, documentation was simply extra work invested for external evaluation. Spending a lot of time on such assessments seemed to them to be counterproductive

Complementarity (explanation of why time is perceived as not being invested wisely)

Several participants did not feel that time taken for documentation was well spent in terms of quality of care. This seemed to be particularly true when they saw no added value for their practical work. For them, spending time on assessments seemed to be counterproductive, i.e. lead rather to a deterioration than an improvement in care

Summary on coherence:

It became apparent that not all participants shared a common understanding of the purpose of the measures. Although participants could identify with the topics, they doubted whether documentation would benefit their practical work and feared poor evaluations and a negative impact on practical work. Coherence is limited by general doubts about whether quality can be depicted through the use of outcome measures

Cognitive participation:

 → How do users engage in the use of the outcome measures?

 → How do users develop a collaborative approach to using them?

Key people in the team encouraged the continuous use of the outcome measures

62% agreed that some people in the team encouraged use of the measures, 8% disagreed (16% neither)

All participants said that at least one person in their team was responsible for documentation and encouraged its implementation and continuous use. They generally also made up the focus groups

Convergence

In most teams, at least one person encouraged continuous use of the modified documentation

Key people outside the team encouraged their use

42% agreed that people outside the team encouraged their use, 18% disagreed (16% neither)

All participants described the staff of the Professional Association of Specialized Palliative Homecare in Hesse as key people outside the team that encouraged its use

Some participants also named our study team as among the people encouraging the use of the modified documentation

Complementarity (who are the key people)

Some participants agreed that key people outside the team encouraged its use. The Professional Association of Specialized Palliative Homecare in Hesse and the study team were among those named

Knew how the documentation should ideally be used

83% agreed to knowing how the documentation should ideally be used 7% disagreed (5% neither)

Participants were familiar with the new documentation. However, it became obvious in discussions that details concerning its practical use and background were not completely clear. For example, they felt unsure how to address psychosocial questions

Complementarity (where insecurities prevail)

Although most participants felt they knew how the documentation should ideally be used, it became obvious that details were unclear

Theoretical examination of topics

86% agreed that they had dealt with the topics theoretically (5% neither, 4% disagreed)

Participants knew what had changed in the modified documentation, and most participants agreed that added topics played an important role in their work

Convergence

Most participants said they have examined the additional topics in the modified documentation

Motivation to use the documentation in practice

41% were motivated to use the documentation in practice, 38% were not (16% neither)

Participants explained that the overall workload linked to the documentation reduced their motivation. They felt that the modified documentation created additional work for health professionals because it contained more items. Participants also said team members became frustrated when it did not work as they had hoped in practice. The sharing of information among teams and team-related feedback were described as encouraging use

Complementarity (which factors influence motivation)

About the same number of participants agreed as disagreed to being motivated to use the documentation with patients. The overall burden of documentation as well as problems in its practical use may have reduced motivation. The sharing of information among teams and team-related feedback may increase motivation

Support for the modified documentation

29% agreed to support the documentation and 41% did not (25% neither)

Participants that expected use of the documentation to result in improvements were more likely to support and try to integrate it. Those that expected no improvements tended not to support its use and to want to undo the changes

Complementarity (influence of assumed benefit)

Participants were divided on support for the modified documentation. Those that expected benefits from it tended to support it and those that did not tended not to

Summary on cognitive participation:

Support for cognitive participation varied. Key persons that encouraged use of the revised documentation and expected benefits from it supported a collaborative approach to using it. The assumed benefit is influenced by participants’ theoretical understanding and perceived feasibility. Motivation for use was reduced by the overall burden of documentation and promoted by information sharing across teams and team-related feedback

Collective action:

 → Is collective action employed to promote the use of the outcome measures?

 → What factors hinder or promote the use of the outcome measures in day-to-day work?

Easy integration

into day-to-day work

While 42% said they could integrate the modified documentation into their day-to-day work, 32% said they could not (18% undecided)

Participants described symptom documentation as easy to use. They explained that the documentation on psychosocial issues and the assessment of self- and proxy-reports caused problems. Further, the organizational integration of paper-based IPOS VoC in day-to-day care was regarded as difficult. Some teams developed internal standards to facilitate use of the documentation, e.g. on how often to use the modified documentation in order to create a routine. Others tried to facilitate use of the documentation by sending out assessment documents by mail or conducting them on the telephone

Complementarity

(which measures caused problems, initiatives to improve these)

Several participants had problems integrating the modified documentation into their day-to-day work. They described problems with documentation relating to psychosocial issues, with self- and proxy assessments, and with the integration of the paper-based IPOS VoC. Some teams developed internal strategies to facilitate use of the modified documentation

Improvements in cooperation with team colleagues

18% agreed, but 50% disagreed that the modified documentation had led to improvement in cooperation with colleagues (26% neither agreed nor disagreed)

The participants described different ways of using the documentation in their teams. Some discussed the results in team meetings, and those that did particularly appreciated IPOS Phase of Illness and IPOS. Others did not discuss results with colleagues, but at least looked at the documentation before visiting patients. They explained that from their point of view the previous documentation had sufficed. Furthermore, they said information on patients was generally communicated orally rather than via documentation

Complementarity (the way the documentation was used in day-to-day care influenced whether advantages were seen)

Several participants disagreed that the modified documentation led to improvements in cooperation with colleagues. The use of the documentation in day-to-day care apparently differed, with those that used it appreciating IPOS Phase of Illness and IPOS for its impact on cooperation with colleagues. Those that did not already use the documentation in team meetings may have seen fewer advantages in terms of cooperation with colleagues. This may be why a minority recognized a positive impact on cooperation with colleagues

Improvements in the quality of relationships with patients and relatives

13% agreed, but 48% disagreed that the modified documentation improved the quality of relationships (28% neither agreed not disagreed)

Although most participants agreed that the topics are important, many emphasized that relationships are built during interactions with patients rather than via documentation. Some participants feared that the strict use of documentation might damage relationships and stressed that a sense of balance was necessary, especially with respect to IPOS psychosocial questions

Complementarity (Reasons why participants think the quality of relationships might suffer)

Only a few participants agreed that use of the modified documentation improves the quality of relationships. Reasons for this may have been that the participants considered relationship quality to be influenced more by interacting with patients than by documentation. Furthermore, they feared that the relationship could be damaged by strict application of documentation

Everyone in the team can use it according to instructions

71% agreed that everyone in the team could use the modified documentation, 18% disagreed (7% neither)

As use of the documentation is obligatory for all participants, they used it in practice and knew how to do so. Small misunderstandings and uncertainties became clear, e.g. on frequency of use

Convergence

Most participants agreed that everyone in the team could use the modified documentation in accordance with instructions

Sufficient support offered during implementation phase

63% agreed that support was sufficient during the implementation phase, 20% disagreed (9% were undecided)

Participants were pleased with the implementation and all teams commented favorably on the personal training and the opportunity to ask questions. Team leaders felt that explanations provided by the study team were better received than their own

Convergence

Most participants agreed that the support offered during the implementation phase was sufficient

Sufficient support is provided

60% agreed that sufficient support was provided at the time of the survey, 16% disagreed (17% were undecided)

Some participants wished for continuous support from the experts in the use of outcome measures, because they sometimes felt unsure about details or technical issues relating to the EDS

Dissonance

While most participants in the online-survey agreed that support was sufficient at the time of the survey, participants in the focus groups would have welcomed more continuous and more regular support from experts in the use of the outcome measures. This may be because several focus group participants were members of the management team and were contacted by colleagues when questions arose. They may therefore have perceived a greater need for support

Sufficient resources are available for use in everyday care

While 40% agreed, 30% disagreed that sufficient resources were available for use in everyday care (22% neither agreed nor disagreed)

Some participants said sufficient resources were available. Others felt that they did not have sufficient time. These participants emphasized the overall burden of documentation and that it had increased as a result of additional items in the modified documentation. Some participants described attempts to counter this by delegating organizational issues to one team member

Complementarity

(Resources mainly related to time, strategies for improvement)

Participants were divided over whether sufficient resources are available for use in everyday care. Those who felt they did not have sufficient resources said they lacked time for documentation

The management team provides sufficient support

72% agreed that the management team provided sufficient support for use of the modified documentation, 7% disagreed (11% neither)

Some management team attended the focus groups, so the adequacy of their support was not discussed. They themselves said it was difficult for them to motivate their teams over time

Silence

Most participants agreed that the management team provided sufficient support in the use of the modified documentation. Members of the management team said they had problems motivating their teams over time

Summary on collective action:

Obstacles to collective action were the difficulty of integrating the documentation into day-to-day work and concerns about lessening the quality of relationships. The way the documentation was employed in day-to-day care had an influence on whether it was seen to be beneficial. Participants were divided as to whether enough support was provided and enough resources were available, but they agreed that their management teams provided sufficient support

Reflexive monitoring:

 → How do the participants view the outcome measures?

 → What improvements did they suggest?

Awareness of reports / experiences of usefulness

26% agreed they knew of reports/experiences of usefulness, but 43% disagreed (24% neither)

Most participants said they did not know of any such reports, and were uncertain how the data was evaluated. Some were afraid that poor final values would be considered as signs of poor quality

Convergence

Most participants were not aware of reports/experiences of the usefulness of the modified documentation

Benefit for their own work

24% agreed, but 47% disagreed to seeing any benefit for their own work (24% neither)

With respect to the advantages of using the modified documentation, participants distinguished between their practical work and quality evaluation

In their practical work, participants valued the IPOS Phase of Illness, the symptom items of IPOS, the five-point Likert scale and the response option ‘not assessable’. Some participants did not see any improvement over the previous documentation. Some participants suggested that a clearer presentation in the electronic documentation system would help in their practical work

Some participants did not understand how the documentation could help them evaluate their own work. They doubted whether an assessment could really be objective and said the proxy assessment might not be valid because of differing opinions among staff members. They also criticized patient self-assessments on the grounds that their condition could influence their assessment

Complementarity (differentiation between advantages in their practical work and in its use for evaluation purposes)

Participants were divided over the advantages of using the documentation in their own work. With respect to any advantages, they distinguished between their practical work and quality evaluation. Participants basically saw how their day-to-day care benefited but thought some items were difficult to integrate. They said it was difficult for them to understand how outcome measures could be used in quality evaluation

Fellow employees’ agreement on usefulness

13% agreed, but 51% disagreed that their fellow employees thought the modified documentation was useful (22% neither)

Agreement among their fellow employees was not discussed in the focus groups

Silence

Most participants disagreed that their fellow employees thought the modified documentation was useful

Confidence that feedback will improve

59% said they were confident that feedback would improve the modified documentation, 14% disagreed (21% neither)

Participants emphasized that their motivation might deteriorate if the teams’ feedback was not taken seriously. They trusted that their feedback would lead to improvement

Convergence

Most participants were confident that their feedback would improve the usefulness of the modified documentation

Ability to adapt use of the modified documentation to suit their own way of working

61% could adapt the application of the documentation to suit their own way of working, 17% disagreed (13% neither)

All participants knew it was possible to include the items in conversation. They reported, however, that the practical implementation was sometimes difficult. Furthermore, IPOS VoC was described as difficult to adapt to suit their own work methods

Complementarity (aspects causing problems adapting)

Most participants agreed they could adapt use of the modified documentation to suit their own work methods

Integration into conversations and use of IPOS VoC were more difficult

Summary on reflexive monitoring

Participants were divided over the benefits of using the documentation. They reported that it was basically useful in day-to-day care, but said some items were difficult to integrate. It was difficult for them to understand how outcome measures can be used in quality evaluation