Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Development and validation of a French questionnaire concerning patients’ perspectives of the quality of palliative care: the QUALI-PALLI-Patient

BMC Palliative Care volume 18, Article number: 19 (2019) Cite this article

Abstract

Background

Indicators for the quality of palliative care are a priority of caregivers and managers to allow improvement of various care settings and their comparison. The involvement of patients and families is of paramount, although this is rarely achieved in practice. No validated assessment tools are available in French. Simple cultural adaption of existing questionnaires may be insufficient, due to the varying organization of care in different countries.

The purpose of this study was to develop and validate a new instrument to measure the quality of palliative care and satisfaction from the patient point of view.

Methods

Results from a qualitative study were used by a multi-professional workgroup to construct an initial set of 42 items exploring six domains. A cross-sectional survey was conducted in seven hospitals, encompassing three care settings: two palliative care units, one palliative care hospital, and four standard medical units with a mobile palliative care team. All items were assessed for acceptability. We conducted exploratory structural analysis using Principal Component Analysis (PCA), and evaluated external validity by comparison against global rating of satisfaction and the MD Anderson Symptom Inventory (MDASI) questionnaire.

Results

A total of 214 patients completed the questionnaire. After removing 7 items from the response distribution, PCA identified eight interpretable domains from the 35 final items: availability of caregivers, serenity, quality of information, pain management, caregivers’ listening skills, psychosocial and spiritual aspects, possibility to refuse (care or volunteers), and respect for the patient. Internal consistency was good or acceptable for all subscales (Cronbach’s α 0.5–0.84), except the last one (0.15). Factorial structure was found globally maintained across subgroups defined by age, sex, Palliative Performance Scale (PPS ≥ 60%, 40–50% and ≤ 30%), and care settings. General satisfaction was inversely correlated with the 2 scores of the MDASI questionnaire: symptoms’ severity and impact on life. Each subscale, except “possibility to refuse”, correlated with general satisfaction.

Conclusions

Quali-Palli-Pat appears to be a valid, reliable, and well-accepted French tool to explore the quality of care and the satisfaction of palliative care patients. It should be confirmed in a wider sample of care settings.

Trial registration

clinicaltrials.gov NCT02814682, registration date 28.6.2016.

Peer Review reports

Background

Improving palliative care requires an assessment of both patient needs and the quality of care they receive [1,2,3,4,5,6,7]. Although many tools focus on quality of life, notably on physical functioning or psychological symptoms as shown in a systematic review [1], fewer tools explore patient appraisal of their care [8,9,10]. Patients’ point of view is a major aspect of quality of care, [6, 11,12,13]. The variety of domains which should be considered by such an outcome measure has been explored (for example: physical, psychological, social, spiritual, cultural, ethical and legal aspects of care) [14, 15] and several tools assess the satisfaction or quality of palliative care, but they may be inadequate to explore the most distressing concerns of patients [16]. Moreover, none are available in French or tailored to French culture [16,17,18,19,20,21].

The organization of care (legislation, regulation, financing…) and the skills of caregivers (prescribing privileges for advanced practice nurses, for instance) vary between countries [22]. Simple cultural adaptation of existing assessment tools may be insufficient, thus requiring the development of tailored instruments. Quali-Palli is a research project aimed to develop multidimensional quality indicators of palliative care practices [23]. Instead of using predefined attributes determined elsewhere, we used a generative approach to obtain a thorough understanding of what is important concerning quality of care from three groups of stakeholders: patients, families, and healthcare professionals. We previously used a theoretical approach with a qualitative analysis method to develop a quality of care model from the patients’ perspective [23]. This approach highlighted the major dimensions of quality of palliative care for all stakeholders: comprehensive support for the patients themselves, clinical management, involvement of families, and care for the imminently dying person, as well as views of patients, families, and healthcare professionals on death [23]. The aim of the present study is the next step of the Quali-Palli project: to develop and validate a patient questionnaire (Quali-Palli-Pat) to measure the quality of palliative care and satisfaction, based on the assumption that patients are the experts of their own life. It had to be usable in all care settings where palliative patients are hospitalized, since the most of palliative care is provided in a hospital facility [24]. This questionnaire could complement the key elements concerning the quality of palliative care of all stakeholders.

Methods

Construction of the questionnaire

For the Qualli-Palli research project, a multi-professional workgroup (physicians, nurses, psychologists, social workers, and volunteers) conducted a literature review to identify research, indicators, and instruments that measure quality of life and quality of care for patients receiving palliative care. We selected a first set of 42 items exploring six domains, based on the major themes that emerged from the palliative literature, as well as from general purpose quality of care questionnaires, and from content analysis of our qualitative study [23, 25]. The six domains consisted of: the quality of information about health status and care (three items), availability of physicians and caregivers (10 items), attitudes of caregivers (six items), physical well-being and symptom management (10 items), functioning of the unit, including the handling of existential issues (nine items), and information and reception of relatives (four items) (Table 1). A four-point Likert scale was displayed after each question with the following ordered responses: no, not at all; not really; almost; yes, exactly, as well as a “does not apply to me” response (not applicable). The last (43rd) item assessed global satisfaction, according to four categories: not satisfied at all; partially satisfied; satisfied; very satisfied. This was followed by an open-ended question about any unmentioned aspects that seemed important to the patient.

Table 1 Quali-Palli questionnaire tested by the patients
Full size table

Pilot study for face validity

The questionnaire needed to be brief, understandable, and easy to complete for hospitalized palliative-care patients and was designed to be self-administered or completed with informal assistance. This questionnaire was presented to 12 patients from three different hospitals to verify comprehensibility and acceptability of the items and response patterns. The patients were asked to complete the first version and to rate each item for clarity in wording, understanding, and relevance to the quality of care [26].

Process of the clinical study

A cross-sectional study was conducted in the Paris metropolitan region in seven settings, encompassing three different care structures: two hospital-based palliative care units (Centre Hospitalier de Pontoise; Groupe Hospitalier Diaconesses Croix Saint-Simon, Paris), one palliative care hospital (Maison Médicale Jeanne Garnier; Paris), and four other standard medical units with mobile palliative care (Institut Curie, Paris, Hôpital Ambroise Paré, Boulogne, Hôpital Saint Antoine, Paris, Hôpital Saint Joseph, Paris). Eligible patients were consecutively recruited according to the following criteria: older than 18 years, hospitalized for palliative care for more than 72 h, a life expectancy under 3 months, and able to understand and communicate well in French and to respond to questionnaires.

One of two trained research assistants at each participating site communicated with the staff to identify potentially eligible patients and excluded those who were considered to be cognitively impaired. After assessing eligibility, the research assistant approached the patients, provided written and oral information about the study, invited them to participate, reassured them about confidentiality concerns and the independence of the study from the caregivers and the hospital, and received their consent. The research assistant gave the questionnaire to the patients informing them that they could take as much time as they needed to complete it or helped them to complete it if they were too tired. For validation purposes, patients completed the French version of the MDASI, which is a reliable assessment tool containing 13 items for the assessment of symptoms and six items to measure their global impact on their daily living and quality of life [27]. The completed questionnaires were directly transmitted to the coordinating investigator or addressed to him in a sealed envelope, without any interaction or supervision by professionals of the structure. Demographic (age, sex) and medical (pathology, stage) data were obtained from the caregivers who also informed us of changes in the health status of the patients (worsened, stable, improved) at the retest visit, relative to the beginning. The functional status of the patients was assessed using the French version 2 of the Palliative Performance Status (PPS) [28, 29].

Statistical analysis

Scoring

Data were re-scaled to range between 1 (no, not at all) and 4 (yes, exactly), as we considered that these items are discrete measurements about an underlying continuum [30]. The item scores were inverted for three negatively worded items so that higher scores indicated higher quality. Not applicable (NA) responses and missing data (MD) were regrouped as non-response (NR).

Acceptability was assessed by computing the proportion of missing responses per item, per subscale, and overall, as well as by measuring the time taken to complete the questionnaire.

Item selection

The first selection of items was made from the descriptive response distribution for each item. The criteria used to guide item selection/deletion were rates of missing data and ‘not applicable’ responses, ceiling and floor effects, and redundancy. Items for which the missing data rate was < 20% were estimated by multiple imputation based on four covariates: age, sex, structure, and Palliative Performance Scale (R software; package Mice - Multivariate Imputation by Chained Equations) [31, 32].

The impact of the following factors on the descriptive response distribution was studied: age, sex, and PPS score according to three categories (≤30, 40–50, ≥ 60) and the type of structure (palliative care structure (unit or hospital) or standard hospitalization). “Not applicable” responses were sure to be present, due to the any-structure purpose of the questionnaire, so an item that may have been discarded on the global analysis was retained if it showed good properties for one care structure.

The criteria used for removing items from the questionnaire were: (1) a high rate of NR (NA or MD) (≥ 20%), (2) a floor or a ceiling effect defined when a category had more than 90% of the same response; (3) and potential redundancy between items defined by a high Pearson correlation between two items (r ≥ 0.70). Clinical relevance considerations also tempered the selection: interest of the item in the literature, room for improvement or a special focus observed during the qualitative study.

Construct validity

The underlying structure among the remaining items was explored using principal component analysis (PCA). Before PCA, the suitability of the data for factor analysis was assessed using Bartlett’s sphericity test, which tests the overall significant differences in the correlation matrix, and the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy. An orthogonal varimax rotation on the correlation matrix was performed to explore the underlying construct because our purpose was to generate factor scores and we expected weak correlations between factors. The cut-off to determine the number of components resulted from a Cattell’s scree plot, completed by Horn’s parallel analysis using R paran package [33, 34]. Items with a low weight (< 0.35) were removed. The criteria to attribute each item to one factor resulted from predetermined rules: a substantial loading (> 0.35) on one principal component; if an item loaded across several factors, it was attributed to the factor for which it maximized the internal consistency, estimated by Cronbach’s α coefficient; at last, clinical judgment and substantive knowledge were used to make sense.

Indeed, internal consistency was estimated using Cronbach’s α for Quali-Palli-Pat total scores and any identified subscales. An α coefficient between 0.7 and 0.9 was considered to indicate good internal consistency without redundancy of items. Accordingly, we calculated the average inter-item correlation within subscales that should fall within the range of 0.25 to 0.50 and cluster narrowly around the mean value [35]. We also calculated the correlation of items in one subscale with the other subscale scores, considering that they should be lower than within their own subscale. An analysis of the inter-subscale correlation matrix was conducted to confirm independence of dimensions if the coefficient was under 0.40 [36] .

The stability of these factors was assessed by principal-component analysis i) using oblique rotations and ii) within different sub-groups of patients defined by sex, age, PPS (PPS ≥ 60%, 40–50% and ≤ 30%) or care settings. We compared factorial structures across related sub-groups (for instance, men and women) by summing how many items were attributed identically to a given factor between the two related structures. To take into account what may be due to chance, we performed a permutation test by comparing this observed statistic to the distribution of the same statistic across 10,000 sub-groups defined by a random equiprobable binomial variable. The reasoning is that if the null hypothesis of there being no influence of a given characteristic on the factorial structure is true, then permuting the values of this characteristic among individuals produces random noise and should be equally likely to produce a larger or a smaller proportion of identically attributed items; on the contrary if the characteristic has a strong influence, then the factorial structures should be different in the related sub-groups, giving a low proportion of identically attributed items. The significance of the observed proportion of identically attributed items for a characteristic is the proportion of such random permutations that lead to a lower proportion of identically attributed items.

Content validity

Experts in the field, i.e. six palliative care physicians, interpreted the results of the PCA, focusing on the clinical sense of the various components, henceforth called subscales. These subscales should reflect domains that are important to patients and other stakeholders and each domain should be represented in at least one subscale. For each subscale, a score was then generated by summing all responses.

Reproducibility

Reproducibility was assessed by test-retest procedures that should reflect the stability of the scores over time when no change has occurred in the patient. The time between two tests must be short enough to ensure that no change in the quality of care has occurred and long enough to prevent recall bias. A time of 1 week is generally considered to be appropriate, but feasibility may be a concern in the palliative context, as patients and their needs at the end of life are rarely stable [1].

External validity

There is currently no other French-validated or French-translated instrument for measuring quality of palliative care from the point of view of patients. Convergent/divergent validity was measured by comparing our instrument with another instrument that measures a theoretically related, but different, construct. We used i) the responses to the last question, which is a global assessment of satisfaction with care, and mainly ii) an assessment tool of symptoms and their impact, i.e. the French version of the MDASI [27]. However, although symptoms and satisfaction with care are related, no correlation could be hypothesized a priori: the aggravation of symptoms may precede the evolution of satisfaction related to care rather than occurring concurrently [37]. Spearman’s correlation between Quali-Palli-Pat global satisfaction and MDASI symptom scores and impact scores were calculated [38]. Specifically, we verified Spearman’s correlations between Quali-Palli-Pat availability of caregivers and MDASI symptom scores.

Results

Patient characteristics

The 43-item questionnaire was administered to 214 patients, from March 2012 to June 2013. Clinical and demographic details are provided in Table 2.

Table 2 Patient Characteristics
Full size table

Acceptability and delivery of the questionnaire

The 12 patients of the pilot study completed the questionnaire which was fully understood without difficulty in understanding the vocabulary or meaning of the questions. So the initial version was not modified.

Only 20% of the 214 patients completed the questionnaire themselves: the lower the PPS score, the more patients needed help to complete the questionnaire; only 6% of patients with a PPS ≤ 30 could do it on their own. One hundred and eighty patients (84.1%) provided complete questionnaires without missing data (MD) but most of them gave not applicable responses (NA); only 15 patients completed the entire questionnaire with no MD or NA. The remaining 34 patients did not complete from 1 to 30 items. Globally, 9.3% of the data was considered to be missing (860/9202); the median number of NR was three items (interquartile range IQR 2–5). Eight patients who completed less than 30% of the questionnaire and 14 who answered NR to the entire questionnaire were excluded. The final analysis included 192 patients. The time to fully complete the two questionnaires (Quali-Palli-Pat, MDASI) was between 20 and 25 min.

Item selection

Six items had a missing rate > 20%. Nine items obtained the highest level on the Likert scale with a ceiling effect of more than 90% of the responses. According to the rules for removing items, 12 (14, 22, 24, 25, 26, 33, 34, 35, 36, 38, 41, 42) should have been deleted. However, our qualitative study highlighted the importance of items 22 (“If necessary, my pain is taken into account before any nursing care”) and 35 (« I was able to refuse the presence of volunteers »), so we decided to retain both items [23]. Removing all items (24 to 26) related to dependent patients was considered unacceptable; item 24 and 25 were retained, but not 26, which combined two removal rules. Items 36 (” I can see a psychologist when I need to “») and 38 (“I can talk to someone about philosophical or religious issues if I wish”) were also retained because of the clinical importance of psychological or spiritual suffering.

Inter item correlations

According to the between items correlation matrix, items 24 (“I receive appropriate help when I eat”) and 25 (“I receive appropriate help to wash myself”) were highly correlated (r = 0.73; p < 0.0001); item 24, which had the higher NA rate (59%) was eliminated.

After removing seven items (14, 24, 26, 33, 34, 41, and 42), the questionnaire contained 35 items which were submitted to factorial analysis. Global KMO measure was .78 while items’ KMO were generally good (only one item <.4, median = .76).

Construct validity

After the imputation of missing data, PCA identified an eight factor-solution which accounted for 76% of the total variance, according to parallel analysis. One factor (#7) had only two items but with high loadings (> = .65). The matrix of factor loadings is shown in Table 3.

Table 3 Principal component factor analysis (varimax rotation) computed using the final 35-item version of the questionnaire
Full size table

Consistency

Table 4 shows the mean score for each factor with a high level of satisfaction (3.4 to 3.8), the normalized Cronbach’s alpha, and inter-item correlation. Internal consistency was very good for factors 1, 2, and 4, acceptable for factors 3, 5, 6, and 7, and low for factor 8. The inter-item correlations within factors were generally close to 0.5 within a narrow band, except for factor 6, and mainly 8 (Table 5). Between-subscale inter-item correlations were consistently lower, except for items 4, 16, 21, and 23. All inter-subscale correlations were lower than 0.4, except those between factors 1 (availability of caregivers) and 2 (serenity) and factors 4 (pain) and 5 (attention to the patient), which were close to 0.5 (Table 6).

Table 4 Validity of convergence and reliability
Full size table
Table 5 Validity of divergence
Full size table
Table 6 Inter-subscale correlation matrix a coefficient of under 0.4 confirms the independence of dimensions
Full size table

Reproducibility

It was not possible to test reproducibility as only 10 patients were able to perform the test/retest procedure, due to the unstable health of this group of terminally ill patients.

Stability

We confirmed stability using the Promax and Oblimin rotation in PCA with similar results (not presented) of those of the Varimax rotation (Table 3). The factorial structure was found globally maintained across the different sub-groups: the proportion of identically attributed items were respectively for .46 for age (permutation test p = .92), .37 for sex (p = .52), .43 for PPS ≤ 30% (p = .84) and .37 for PPS ≥ 60% (p = .52), .4 for settings (p = .7). Among most stable items were those related to availability of caregivers, information, pain and serenity. Then, the mean scores of factors were close across sub-groups, except for factor 1 (availability) and 2 (serenity) which were significantly better in the palliative care unit, but this statistical difference between such high scores may not be clinically relevant (Table 7).

Table 7 Comparison of the means of the scores (sd) according to age, sex, PPS score, and care structure
Full size table

External validation

Global satisfaction with care was inversely correlated with the global symptoms score of the MDASI and the interference of symptoms on the quality of life score (Table 8). Each factor of the questionnaire correlated with the global satisfaction with care, except factor 7. Factor 1 (availability) was clearly negatively correlated with both the symptoms score and the interference with quality of life score.

Table 8 Correlation between global satisfaction and the symptoms score (MDASI) and interference of the symptoms score with each factor of the Quali-Palli-Pat
Full size table

Discussion

Quali-Palli-pat: A validated tool to explore eight domains

In this multicenter prospective study, we created and validated a French questionnaire – Quali-Palli-Pat – that evaluates the quality of palliative care and the satisfaction with care from the patients’ perspective. We started with an initial questionnaire of 43 items. After analysis, the final questionnaire includes 35 items that explore the eight following domains: availability of caregivers, serenity, quality of information, pain management, willingness to listen to patients, psychosocial and spiritual aspects, right of refusal, and respect of the patients. The questionnaire concludes with a 36th question concerning general satisfaction. These eight dimensions, obtained after PCA, are in accordance with the six initially expected, but also include the theme of pain management and separate items which initially belonged to the relationship with doctors and caregivers. These results highlight major themes previously described in the palliative care literature and finally correspond to patient priorities, such as the availability of caregivers or the need for serenity [3, 14, 39,40,41,42]. By exploring these eight dimensions, Quali-Palli-Pat differs from unidimensional satisfaction questionnaires, such as Famcare Pat [19, 43]. The stability of these domains was maintained across different sub-groups of patients (sex, age, PPS) and healthcare settings.

External validation and relation with satisfaction and quality of life

Our results indicate that global satisfaction is linked primarily to the global score of the MDASI symptom assessment scale and secondly to the interference of symptoms on the quality of life score. An important factor, such as the availability of caregivers, was linked both to the symptom score and interference score. However, Quali-Palli-Pat is neither a questionnaire of symptom assessment nor of quality of life. Satisfaction with care and quality of care are not interchangeable measures [44]. Our results indicate that all but one (right to refuse) of the explored dimensions positively correlated with the score of general satisfaction and probably directly contributed to it, as previously suggested [39, 41, 44, 45]. As quality of care may be some externally assessed measure, satisfaction with care deals not only with quality from a technical perspective but also with discrepancies between patients’ attempts and provision of care. [45]. Health status, quality of care, satisfaction with care, and quality of life appear to be in a dynamic loop [17]. We observed that a relative well-being (MDASI score) is associated to the satisfaction, related to obvious efforts of the caregivers mediated through interpersonal communication, as postulated by Stewart et al. [45]. However, according to the integrative response shift model proposed by Sprangers and Schwartz, a worsening of health status may modify the tolerance of the patient, and thus paradoxically maintain the level of the quality of life [46]. A similar process may have affected our measures of quality of care and satisfaction, found to be stable irrespective of the PPS, such as the recently reported independence between quality of life and ECOG performance status [37].

Strengths of the method

The strength of the method used to create and validate Quali-Palli-Pat makes it a theoretically sound and valid instrument, based on scoring of measurement properties as formalized in the COSMIN checklist (www.cosmin.nl/), which includes internal consistency, reliability, measurement error, content validity, construct validity, criterion validity, and responsiveness [47, 48]. Quali-Palli-Pat meets most of these criteria. Except for reliability, measurement error, and responsiveness whose evaluation was hampered by the rapid deterioration of patients, all other criteria were excellent or good. Sample size and the methods appeared to be good both for dimensionality and internal consistency analysis, while also managing missing data. Concerning sample size, we had an initial sample-to-variable ratio of 5:1 and final variable-to-factor ratio of 4:1. Our sample size thus appeared to be adequate, considering the mean variable-to factor ratio and the relatively high value of the loadings (mean = 0.55) [49]. Content validity was built on a qualitative study among patients, and was confirmed by the easy interpretability of the final subscales. Construct validity was based on exploratory factor analysis and we verified the stability of the structure. Criterion validity mainly confirmed a priori hypotheses, facing global satisfaction or relation with the impact of the disease. Finally, the questionnaire was administered in various settings, to patients of various ages and physical status, thus enhancing the generalization of the study findings, although future validation should include home hospitalized patients.

Quali-Palli-pat: A useful tool for public health policy makers

We conceived Quali-Palli-patient as a diagnostic tool that provides a quality profile at the service level, highlighting domains which need improvement, rather than a unique and absolute score, which is difficult to interpret. Evaluation at the patient level is not appropriate, as evaluating and comparing individuals within a setting requires very high internal consistency, high precision and adjustment on patient status while there is the problem of the lack of independence of the patient from the caregiver. Our instrument may be used as a pre- and post-measurement tool in quality improvement interventions, as recommended by the e-satis program of the French National Authority for Health with periodic audits of satisfaction of hospitalized patients [50]. Periodically using this tool to compare various settings of palliative care could provide the means to better adapt public health policies and funding.

Communication between patients and health providers

Quali-Palli-Pat largely explores the communication between patients and health providers. This is an extremely important domain, especially when deciding to stop specific treatments, such as chemotherapy, and at the beginning of exclusive palliative care. Teno et al. previously highlighted this domain and developed a specific tool focusing on information needs, especially at the time of diagnosis or making decisions concerning treatment [41]. In our study, this domain showed the lowest values, highlighting the necessity of further improvement, especially in palliative care units. A single item concerned spirituality (“I can talk to someone about philosophical or religious issues if I wish”) but with a rate of 27% of NA, despite its harmless formulation, this item would have been deleted according to our rules. However, it was retained by the working group because spirituality is an essential component of palliative care [51, 52]. The NA response can be explained by the specificity of French secularism, which considers religious or spiritual domains as private; French patients cannot understand the relationship between care, quality of care, and spirituality [53].

Limits of the study

Our study had limitations. First, we used a sample of inpatients hospitalized in palliative care units or academic hospitals involved in quality of palliative care. This choice was probably not representative of the general population of palliative care inpatients, nor of the numerous settings tagged with palliative care services, ours being volunteers. This could explain the ceiling effect frequently observed and justifies retaining some items, despite this effect. The choice of retaining some items, despite their ceiling effect, should be confirmed in a future confirmatory study including a more diverse population of structures and patients.

Second, reproducibility could not be assessed due to the rapid evolution of the health status of the patients. Testing reliability requires a test/retest in which patients remain stable for a reasonable period, perhaps a week [1]. This short timeframe is far too long for hospitalized palliative care patients, because the median overall survival in palliative care units is approximately 12 days, during which patients quickly and inevitably get worse. Thus, it was not possible to test reliability and sensitivity to change, as in other studies [1]. Quali-Palli-Pat, like all instruments of this kind, can only be used by fully communicative patients, which does not reflect the palliative care population, especially at the end of life [54]. Thus, we plan to create and validate the Quali-Palli-Fam questionnaire, intended for relatives. We did not estimate the relative weight of each of the eight domains, as tested in other settings [42]. We believe that it would be too difficult for terminally ill tired patients to disentangle the two related questions of perceived quality and subjective importance, item by item [43]. This could eventually be done in a subsequent confirmatory survey, which could also verify the existence of these eight domains in other contexts.

Although the questionnaire was created to be as brief, understandable, and easy as possible, 80% of patients required assistance to complete it. Response bias was reduced by the fact that the interviewer was a clinical research associate, independent from the caregivers of the hospitals [55]. This raises the question of who will assist in the completion of the questionnaire for periodic assessments. In our recent experience, it is possible to involve volunteers. However, some patients with delirium or vigilance disorders will never complete it, explaining the lack of patients with brain tumors in our data. The Quali-Palli-Fam, which explores the family perspective, should at least partially compensate for this.

Conclusion

Quali-Palli-Pat appears a valid, reliable, and well-accepted French tool that explores the quality of palliative care and the satisfaction in patients with advanced disease, hospitalized in various settings, and is ready for use in the French language (Additional file 1: French Quali-Palli-Pat). Providing a voice for the most vulnerable patients is the right way to assess the gap between their needs and provided care. This questionnaire reflects the quality of a structure and is not considered at the individual level. Periodic patient audits would help care teams to improve the quality of palliative care. It must be complemented in the future by a relatives’ questionnaire.

Abbreviations

EFA:

Exploratory factor analysis

KMO:

Kaiser–Meyer–Olkin

MD:

Missing data

MDASI:

MD Anderson Symptom Inventory

NA:

Not applicable

NR:

Non response

PCA:

Principal component analysis

PPS:

Palliative performance scale

References

  1. Albers G, Echteld MA, de Vet HCW, Onwuteaka-Philipsen BD, van der Linden MHM, Deliens L. Evaluation of quality-of-life measures for use in palliative care: a systematic review. Palliat Med. 2010;24:17–37.

    Article  Google Scholar 

  2. Bausewein C, Le Grice C, Simon S, Higginson I, On behalf of PRISMA. The use of two common palliative outcome measures in clinical care and research: a systematic review of POS and STAS. Palliat Med. 2011;25:304–13.

    Article  CAS  Google Scholar 

  3. Dy SM, Shugarman LR, Lorenz KA, Mularski RA, Lynn J. RAND-Southern California evidence-based practice center. A systematic review of satisfaction with care at the end of life. J Am Geriatr Soc. 2008;56:124–9.

    Article  Google Scholar 

  4. Hales S, Zimmermann C, Rodin G. Review: the quality of dying and death: a systematic review of measures. Palliat Med. 2010;24:127–44.

    Article  Google Scholar 

  5. Lorenz KA, Lynn J, Dy SM, Shugarman LR, Wilkinson A, Mularski RA, et al. Evidence for improving palliative care at the end of life: a systematic review. Ann Intern Med. 2008;148:147–59.

    Article  Google Scholar 

  6. Mularski RA, Dy SM, Shugarman LR, Wilkinson AM, Lynn J, Shekelle PG, et al. A systematic review of measures of end-of-life care and its outcomes: measures of end-of-life care. Health Serv Res. 2007;42:1848–70.

    Article  Google Scholar 

  7. Parker D, Hodgkinson B. A comparison of palliative care outcome measures used to assess the quality of palliative care provided in long-term care facilities: a systematic review. Palliat Med. 2011;25:5–20.

    Article  Google Scholar 

  8. De Roo ML, Leemans K, Claessen SJJ, Cohen J, Pasman HRW, Deliens L, et al. Quality indicators for palliative care: update of a systematic review. J Pain Symptom Manag. 2013;46:556–72.

    Article  Google Scholar 

  9. Lendon JP, Ahluwalia SC, Walling AM, Lorenz KA, Oluwatola OA, Anhang Price R, et al. Measuring Experience With End-of-Life Care: A Systematic Literature Review. J Pain Symptom Manage. 2015;49:904–15 e1–3.

    Article  Google Scholar 

  10. Kearns T, Cornally N, Molloy W. Patient reported outcome measures of quality of end-of-life care: a systematic review. Maturitas. 2017;96:16–25.

    Article  Google Scholar 

  11. Weltgesundheitsorganisation, editor. Quality of care: a process for making strategic choices in health systems. Geneva: WHO; 2006.

    Google Scholar 

  12. Grande G. Palliative care in hospice and hospital: time to put the spotlight on neglected areas of research. Palliat Med. 2009;23:187–9.

    Article  CAS  Google Scholar 

  13. Unroe KT, Meier DE. Research priorities in geriatric palliative care: policy initiatives. J Palliat Med. 2013;16:1503–8.

    Article  Google Scholar 

  14. Ferrell BR. Overview of the domains of variables relevant to end-of-life care. J Palliat Med. 2005;8:s–22.

    Article  Google Scholar 

  15. Fowler FJ, Coppola KM, Teno JM. Methodological challenges for measuring quality of care at the end of life. J Pain Symptom Manag. 1999;17:114–9.

    Article  Google Scholar 

  16. Tishelman C, Lövgren M, Broberger E, Hamberg K, Sprangers MAG. Are the Most distressing concerns of patients with inoperable lung Cancer adequately assessed? A Mixed-Methods Analysis J Clin Oncol. 2010;28:1942–9.

    PubMed  Google Scholar 

  17. Heyland DK, Cook DJ, Rocker GM, Dodek PM, Kutsogiannis DJ, Skrobik Y, et al. The development and validation of a novel questionnaire to measure patient and family satisfaction with end-of-life care: the Canadian health care evaluation project (CANHELP) questionnaire. Palliat Med. 2010;24:682–95.

    Article  Google Scholar 

  18. Heyland DK, Jiang X, Day AG, Cohen SR. The development and validation of a shorter version of the Canadian health care evaluation project questionnaire (CANHELP lite): a novel tool to measure patient and family satisfaction with end-of-life care. J Pain Symptom Manag. 2013;46:289–97.

    Article  Google Scholar 

  19. Ringdal GI, Jordhøy MS, Kaasa S. Measuring quality of palliative care: psychometric properties of the FAMCARE scale. Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2003;12:167–76.

    Article  Google Scholar 

  20. Miyashita M, Wada M, Morita T, Ishida M, Onishi H, Tsuneto S, et al. Care evaluation scale—patient version: measuring the quality of the structure and process of palliative care from the Patient’s perspective. J Pain Symptom Manag. 2014;48:110–8.

    Article  Google Scholar 

  21. Poirier A, Kwiatkowski F, Commer J, Swaine-Verdier A, Montel S, Charpy J, et al. Report on the first stages in the translation of measures of health-related quality of life at the end of life. J Palliat Care Med. 2014;4:178.

  22. Klinger CA, Howell D, Zakus D, Deber RB. Barriers and facilitators to care for the terminally ill: a cross-country case comparison study of Canada, England, Germany, and the United States. Palliat Med. 2014;28:111–20.

    Article  Google Scholar 

  23. Vedel I, Ghadi V, Lapointe L, Routelous C, Aegerter P, Guirimand F. Patients’, family caregivers’, and professionals’ perspectives on quality of palliative care: a qualitative study. Palliat Med. 2014;28:1128–38.

    Article  Google Scholar 

  24. Centeno C, Lynch T, Garralda E, Carrasco JM, Guillen-Grima F, Clark D. Coverage and development of specialist palliative care services across the World Health Organization European region (2005-2012): results from a European Association for Palliative Care Task Force survey of 53 countries. Palliat Med. 2016;30:351–62.

    Article  Google Scholar 

  25. Salomon L, Gasquet I, Mesbah M, Ravaud P. Construction of a scale measuring inpatients’ opinion on quality of care. Int J Qual Health Care J Int Soc Qual Health Care. 1999;11:507–16.

    Article  CAS  Google Scholar 

  26. McColl E, Jacoby A, Thomas L, Soutter J, Bamford C, Steen N, et al. Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients. Health Technol Assess Winch Engl. 2001;5:1–256.

    CAS  Google Scholar 

  27. Guirimand F, Buyck J-F, Lauwers-Allot E, Revnik J, Kerguen T, Aegerter P, et al. Cancer-related symptom assessment in France: validation of the French M. D. Anderson symptom inventory. J Pain Symptom Manag. 2010;39:721–33.

    Article  Google Scholar 

  28. Anderson F, Downing GM, Hill J, Casorso L, Lerch N. Palliative performance scale (PPS): a new tool. J Palliat Care. 1996;12:5–11.

    CAS  PubMed  Google Scholar 

  29. Ho F, Lau F, Downing MG, Lesperance M. A reliability and validity study of the Palliative Performance Scale. BMC Palliat Care [Internet]. 2008 [cited 2018 Mar 7];7. Available from: http://bmcpalliatcare.biomedcentral.com/articles/10.1186/1472-684X-7-10

  30. Norman G. Likert scales, levels of measurement and the “laws” of statistics. Adv Health Sci Educ Theory Pract. 2010;15:625–32.

    Article  Google Scholar 

  31. Resseguier N, Verdoux H, Giorgi R, Clavel-Chapelon F, Paoletti X. Dealing with missing data in the Center for Epidemiologic Studies Depression self-report scale: a study based on the French E3N cohort. BMC Med Res Methodol. 2013;13:28.

    Article  Google Scholar 

  32. Legault S, Laurin C, Bouvette M. French version of Palliative Performance Scale v2 [Internet]. PPS. 2001 [cited 2018 Mar 7]. Available from: https://www.victoriahospice.org/sites/default/files/ppsv2-french_0.pdf

  33. Dinno A. Exploring the sensitivity of Horn’s parallel analysis to the distributional form of random data. Multivar Behav Res. 2009;44:362–88.

    Article  Google Scholar 

  34. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179–85.

    Article  CAS  Google Scholar 

  35. Briggs SR, Cheek JM. The role of factor analysis in the development and evaluation of personality scales. J Pers. 1986;54:106–48.

    Article  Google Scholar 

  36. Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1994.

  37. Miyashita M, Wada M, Morita T, Ishida M, Onishi H, Tsuneto S, et al. Development and validation of the comprehensive quality of life outcome (CoQoLo) inventory for patients with advanced cancer. BMJ Support Palliat Care. 2015;0:1-9. https://doi.org/10.1136/bmjspcare-2014-000725.

  38. Cleeland CS. The M. D. Anderson Symptom Inventory - User Guide [Internet]. [cited 2018 Feb 20]. Available from: https://www.mdanderson.org/documents/Departments-and-Divisions/Symptom-Research/MDASI_userguide.pdf

  39. van Soest-Poortvliet MC, van der Steen JT, Zimmerman S, Cohen LW, Munn J, Achterberg WP, et al. Measuring the quality of dying and quality of care when dying in long-term care settings: a qualitative content analysis of available instruments. J Pain Symptom Manag. 2011;42:852–63.

    Article  Google Scholar 

  40. Aspinal F, Hughes R, Dunckley M, Addington-Hall J. What is important to measure in the last months and weeks of life?: a modified nominal group study. Int J Nurs Stud. 2006;43:393–403.

    Article  Google Scholar 

  41. Teno JM, Lima JC, Lyons KD. Cancer patient assessment and reports of excellence: reliability and validity of advanced cancer patient perceptions of the quality of care. J Clin Oncol Off J Am Soc Clin Oncol. 2009;27:1621–6.

    Article  Google Scholar 

  42. Sandsdalen T, Hov R, Høye S, Rystedt I, Wilde-Larsson B. Patients’ preferences in palliative care: a systematic mixed studies review. Palliat Med. 2015;29:399–419.

    Article  Google Scholar 

  43. Lo C, Burman D, Hales S, Swami N, Rodin G, Zimmermann C. The FAMCARE-patient scale: measuring satisfaction with care of outpatients with advanced cancer. Eur J Cancer. 2009;45:3182–8.

    Article  Google Scholar 

  44. Aspinal F, Addington-Hall J, Hughes R, Higginson IJ. Using satisfaction to measure the quality of palliative care: a review of the literature. J Adv Nurs. 2003;42:324–39.

    Article  Google Scholar 

  45. Stewart AL, Teno J, Patrick DL, Lynn J. The concept of quality of life of dying persons in the context of health care. J Pain Symptom Manag. 1999;17:93–108.

    Article  CAS  Google Scholar 

  46. Sprangers MA, Schwartz CE. Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med 1982. 1999;48:1507–15.

    CAS  Google Scholar 

  47. van Campen C, Sixma H, Friele RD, Kerssens JJ, Peters L. Quality of care and patient satisfaction: a review of measuring instruments. Med Care Res Rev MCRR. 1995;52:109–33.

    Article  Google Scholar 

  48. Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2012;21:651–7.

    Article  Google Scholar 

  49. MacCallum RC, Widaman KF, Preacher KJ, Hong S. Sample size in factor analysis: the role of model error. Multivar Behav Res. 2001;36:611–37.

    Article  CAS  Google Scholar 

  50. Haute Autorité de Santé. e-satis : mesure de la satisfaction et de l’expérience des patients hospitalisés plus de 48h [Internet]. [cited 2018 Apr 24]. Available from: https://www.has-sante.fr/portail/jcms/c_2030354/fr/iqss-2018-e-satis-mesure-de-la-satisfaction-des-patients-hospitalises

  51. Speck P. Culture and spirituality: essential components of palliative care. Postgrad Med J. 2016;postgradmedj–2015.

  52. Daaleman TP, Reed D, Cohen LW, Zimmerman S. Development and preliminary testing of the quality of spiritual care scale. J Pain Symptom Manag. 2014;47:793–800.

    Article  Google Scholar 

  53. Lucette A, Brédart A, Vivat B, Young T. Pilot-testing the French version of a provisional European organisation for research and treatment of cancer (EORTC) measure of spiritual well-being for people receiving palliative care for cancer. Eur J Cancer Care (Engl). 2014;23:221–7.

    Article  CAS  Google Scholar 

  54. McPherson CJ, Addington-Hall JM. Judging the quality of care at the end of life: can proxies provide reliable information? Soc Sci Med 1982. 2003;56:95–109.

    CAS  Google Scholar 

  55. Ciccarelli SK, White JN. Psychology. 4th ed . Harlow: Pearson; 2014.

Download references

Acknowledgements

The investigators thank the members of the Quali-Palli Group: Carole Bouleuc, Louise Geoffroy, Laurence Gineston, Hélène Godard, Xavier Grenet, Karima Nfissi, Jean-François Richard, Sylvie Rostaing, Yael Tibi-Lévy, Isabelle Vedel, Kate Vincent, and the Clinical Research Unit team that worked on Quali-Palli: Layde Meaude, Sylvie Laot-Cabon, Sarah Gaston-Dreyfus, Audrey Angelard, Yasmine Saidji, Karima Mesbahi, Wen Teng, and Pr Bruno Falissard for insightful comments.

Funding

This study was supported by a grant from the French Ministry of Health (PreQHos 2009–04) and sponsored by the Assistance Publique-Hôpitaux de Paris (NI10052).

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

  1. Pôle recherche SPES “Soins Palliatifs en Société”, Maison Médicale Jeanne Garnier, 106 avenue Emile Zola, 75015, Paris, France

    Frédéric Guirimand

  2. Unité de Recherche Clinique URC HU PIFO, AP-HP, Hôpital Ambroise Paré, Boulogne, France

    Patricia Martel-Samb

  3. Équipe Mobile de Soins Palliatifs, AH-HP, Hôpital Saint Antoine, Paris, France

    Christian Guy-Coichard

  4. Unité de Soins Palliatifs, Groupe Hospitalier Diaconesses Croix Saint-Simon, Paris, France

    Stéphane Picard & Laure Copel

  5. Centre de Recherche Interprofessionnel Bientraitance et Fin de Vie, Service de Médecine Palliative, Hôpital de Pontoise, Pontoise, France

    Bernard Devalois

  6. Équipe Mobile de Soins Palliatifs, AP-HP, Hôpital Ambroise Paré, Boulogne, France

    Anne Abel

  7. Haute Autorité de Santé, Saint Denis, France

    Véronique Ghadi

Authors
  1. Frédéric Guirimand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patricia Martel-Samb
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Guy-Coichard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stéphane Picard
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bernard Devalois
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Laure Copel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Anne Abel
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Véronique Ghadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

QUALI-PALLI group and Philippe Aegerter

Contributions

FG and PA were responsible for the study concept and design, obtained funding and supervised the study. PMS and PA carried out the statistical analysis. Quali Palli Group conducted a literature review and selected a first set of items. All authors analyzed and interpreted the data. FG, PS and PA drafted the manuscript. All authors critically revised the manuscript, read and approved the final version of the manuscript.

Corresponding author

Correspondence to Frédéric Guirimand.

Ethics declarations

Ethics approval and consent to participate

This study was approved by a regional ethics committee (#12059, Comité de Protection des Personnes “CPP île de France XI”, Saint Germain-en-Laye, France) and registered on clinicaltrials.gov (NCT02814682). All the participants were duly informed and oral consent was obtained prior to the research. According to French health regulations no written informed consent is required as the research is an observational study on registered data.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Quali-Palli-Pat : Questionnaire sur la qualité de la prise en charge. Original French version of the questionnaire concerning patients’ perspectives of the quality of palliative care: Quali-Palli-Pat. (DOCX 44 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guirimand, F., Martel-Samb, P., Guy-Coichard, C. et al. Development and validation of a French questionnaire concerning patients’ perspectives of the quality of palliative care: the QUALI-PALLI-Patient. BMC Palliat Care 18, 19 (2019). https://doi.org/10.1186/s12904-019-0403-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12904-019-0403-z

Keywords

BMC Palliative Care

ISSN: 1472-684X

Contact us