Skip to main content

An instrument to assess the education needs of nursing assistants within a palliative approach in residential aged care facilities



Providing quality palliative care in residential aged care facilities (RACFs) (aged care homes) is a high priority for ageing populations worldwide. Older people admitted to these facilities have palliative care needs. Nursing assistants (however termed) are the least qualified staff and provide most of the direct care. They have an important role at the frontline of care spending more time with residents than any other care provider but have been found to lack the necessary knowledge and skills to provide palliative care. The level of competence of this workforce to provide palliative care requires evaluation using a valid and reliable instrument designed for nursing assistants’ level of education and the responsibilities and practices of their role.


The overall study purpose was to develop and test an instrument capable of evaluating the knowledge, skills and attitudes of nursing assistants within a palliative approach in RACFs. Development consisted of a four-phase mixed-methods sequential design. In this paper, the results and key findings following psychometric testing of the instrument in Phase 4 is reported using data collected from a random sample of 17 RACFs and 348 nursing assistants in the Greater Sydney region. Study hypotheses were tested to confirm discriminative validity and establish the utility of the instrument in both research and training assessment.


Individual item properties were analysed for difficulty, discrimination and item-total correlations. Discriminative and structural validity, and internal consistency and test-retest reliability were demonstrated. Three separate questionnaires comprising 40 items were finalised: The Palliative Approach for Nursing Assistants (PANA)_Knowledge Questionnaire (17 items), the PANA_Skills Questionnaire (13 items) and the PANA_Attitudes Questionnaire (10 items).


This study provides preliminary evidence for the validity and reliability of three new questionnaires that demonstrate sensitivity for nursing assistants’ level of education and required knowledge, skills and attitudes for providing a palliative approach. Implications for practice include the development of palliative care competencies through structured education and training across this workforce, and ongoing professional development opportunities for nursing assistants, especially for those with the longest tenure.

Peer Review reports


Older people with palliative care needs related to chronic, life-limiting diseases admitted to nursing homes or residential aged care facilities (RACF) as termed in Australia, are highly dependent on skilled, compassionate care. Nursing care is an integral component of residential aged care and is provided by the nursing team comprising registered nurses (RNs), enrolled nurses (ENs) (similar to licensed practical nurses) and nursing assistants (however termed). RACFs were identified over a decade ago as the ‘hospices of the future’ [1] and are now a major provider of aged palliative care as evidenced by the number of older people being admitted to these services for end-of-life care [2,3,4]. In 2015, 34% of all Australian deaths occurred within RACFs with more than 60% of residents dying within six months of admission [5]. The majority of permanent residents assessed as requiring palliative care were in the 85 years and older age group and over half of all residents in RACFs have a diagnosis of dementia [5, 6].

Whether palliative care is required on admission to the RACF or at some later stage, nursing assistants are involved in providing comfort care and in the resident’s transition to end of life. A palliative approach is considered best practice for this population with its focus on the needs of the person, not the disease [7, 8] and can be delivered by a range of non-specialists at point of admission to the service [9]. Nursing assistants who provide routine personal care such as bathing, dressing or grooming, and assisting elderly, convalescent or disabled people with eating, mobility and communication provide most direct care to residents and spend more time with them than any other care providers [10]. Nursing assistants are required to work under the supervision of RNs or ENs who are responsible for the overall care of residents. However, recruitment has shifted to nursing assistants as the mainstay of the RACF workforce (70%) [11], and there are fewer RNs occupying a supervisory role [12].

Although nursing assistants are best placed to identify and support residents’ physical, psychological, emotional and spiritual needs [13], and provide support to family carers reflecting core elements of palliative care, they require palliative care knowledge, essential skills and positive attitudes necessary for adopting core values of palliative care, such as an open attitude to dying and death [7, 14, 15]. Of the nursing assistants (n = 108,126) surveyed in the Australian National Aged Care Workforce Census and Survey 2016, only 7.4% reported a specialised qualification in palliative care and 72% identified this as a priority area for training [11]. Nursing assistants are less likely than RNs or ENs to attend non-mandatory professional development education [16] and this may not be optimally aligned to the demands of the workplace [17]. Furthermore, the education curricula of nursing assistants, in general, has been found to vary in content [17] and there is no specific prior learning or mandatory requirement in Australia to participate in pre-service education nor is there currently any licensing, legislative, or regulatory requirements attached to the entry-level industry qualification [18]. With respect to palliative care, the only educational unit that explicitly addressed a palliative approach to care [19] was changed in 2015 from a core unit to an elective within this Australian qualification [18]. This means that nursing assistants, as a broad-based group, may not have any pre-service education in palliative care.

It is important to be able to evaluate nursing assistants’ palliative care knowledge, skills and attitudes with a validated instrument. When nursing assistants have been evaluated with existing instruments, they have been found to have a low knowledge of palliative care [20,21,22,23,24,25] and deficits in skills and attitudes [21, 26]. However, only one instrument out of seven identified and critically examined was developed for the nursing assistants’ role limiting the validity of study findings related to this group [27].

This study has developed an instrument designed to evaluate the knowledge, skills and attitudes of nursing assistants within a palliative approach in order to elicit the educational needs relative to their role and responsibilities in providing care with a palliative approach. This paper reports the results of Phase 4 of a four-phase mixed methods study to validate three questionnaires entitled PANA (Palliative Approach for Nursing Assistants) and discusses the main findings of this phase of the study (Additional file 2).

To test the validity of the instrument for nursing assistants, it was hypothesised that an instrument designed specifically for nursing assistants’ level of education and scope of practice would:

  1. I.

    Perform better than the Palliative Care Quiz for Nursing (PCQN) [28] in discriminating knowledge of a palliative approach between groups of nursing assistants.

  2. II.

    Demonstrate that experience measured as length of time in the nursing assistants’ role will be a better predictor than education in discriminating knowledge of a palliative approach between groups of nursing assistants.

  3. III.

    Detect differences between groups of nursing assistants for knowledge of a palliative approach based on experience in the role.

  4. IV.

    Detect differences between groups of nursing assistants in self-perceived skills for a palliative approach based on experience in the role.

  5. V.

    Detect differences between groups of nursing assistants for attitudes towards a palliative approach based on experience in the role.

The relevance of these hypotheses to the development of the instrument is, first, to confirm that the instrument can be used to evaluate differences in scores between groups with varying levels of experience and education; and, second, to provide insight into the effectiveness of educational processes.

The broader study was conducted over four sequential phases according to recommended psychometric processes for instrument development [29, 30]. The four phases were item generation, content validation with experts, pilot testing and field testing and are shown in summary in Table 1. Human Research Ethics approval was obtained from Western Sydney University (H9963).

Table 1 Summary of the four sequential phases of the study

Instrument development process

Phase 1 commenced with the generation of items. The Guidelines for a palliative approach in residential aged care [7] (Additional file 1) provided the conceptual framework for the inclusion of content in the instrument (included as supplementary material). The educational unit Deliver care services using a palliative approach [19], offered in the nationally recognised aged care qualifications [18, 31], was used to delineate content specific to the performance criteria or ‘scope of practice’ of nursing assistants.

Phase 2 involved content validation by experts. Clarity and relevance were evaluated using the Content Validity Index (CVI) on 135 items [32]. Of the 89 items retained following the content validation process, content validation of these total items in the new instrument was assessed. First, the average of all items rated either 4 or 3 were summed to produce an overall score for content validity of 0.96, and then the S-CVI universal method, which requires agreement by all experts, was applied. Eighty-five items were endorsed, giving an overall score for the instrument’s content validity of 0.99.

In Phase 3, a final pool of 85 items was established for pilot testing, consisting of 28 knowledge, 38 skills, and 19 attitude items. The prefix PANA (Palliative Approach for Nursing Assistants) was adopted for each of the three questionnaires. The procedure and results of this phase have been reported elsewhere [33]. As a result of pilot testing in Phase 3, amendments were made to two knowledge questions and the response options for the skills statements. The result was an 85-item instrument ready for field testing and psychometric evaluation.


Phase 4 Field testing procedure

Validity analysis

Three types of measurement-related validity were examined. Known groups (discriminative validity), convergent and divergent validity, and structural (construct) validity using an exploratory factor analysis was assessed. For the known groups, nursing assistants were allocated into one of three groups at time of completion of the questionnaires based on information provided in the demographic questions. The three groups were: group 1: staff with less than or equal to two years’ experience in role; group 2: between two and five years’ experience in role; and, group 3: staff with more than five years’ experience in role.

Reliability analysis

Internal consistency was assessed with Cronbach’s alpha coefficient [34] and stability (test-retest reliability) was assessed with an intra-class correlation (ICC), which is sensitive both to agreement of scores and to association between scores [35].


A manual method of random allocation was used to recruit facilities for Phase 4 field testing. The Aged Care Service List New South Wales (NSW) (June 2014), which provides information on aged care services subsidised by the Australian Government, was used to identify facilities within the Greater Sydney area with more than 50 operational places or ‘beds’ (n = 116). These facilities were allocated a number corresponding to the List. One person randomly selected 20 numbers representing these eligible facilities from a concealed container in front of the study team. Twenty facilities were sought for recruitment on the basis that if each facility only had a minimum of 50 places or ‘beds’, the mean ratio (worker: resident) of 0.7 direct care workers, with the majority of these being nursing assistants (70%) [11], there would be a sample of 490 (20x50x0.7x0.7) nursing assistants. A minimum sample of 300 participants was sought based on statistical advice and published literature on sample size required for exploratory factor analysis [36,37,38]. Of these twenty facilities, six initially declined to participate: two of these were later granted organisational permission and a randomly selected back-up site using the same procedure that determined the initial sites agreed to participate. Altogether, 17 sites were included, with 1888 operational places yielding a potential sample of 1321 nursing assistants. Approval was obtained from each participating facility or from the organisation’s research department if the facility was part of a larger organisation.


The number (or approximate number) of nursing assistants working in each facility was provided by the Facility Care Manager (FCM). All nursing assistants providing direct care to residents from the participating RACFs were eligible to participate. The FCM or delegated other was responsible for distributing the instrument booklet to interested staff. A subsample of thirty nursing assistants was also sought to complete the instruments a second time for the purpose of test-retest reliability and responsiveness.

Instrument booklet

A 16-page A4 instrument booklet was compiled in English containing the three new questionnaires developed and refined in Phases 1, 2 and 3, and the Palliative Care Quiz for Nurses (PCQN) [28]. The PCQN was designed to test nurses’ knowledge of palliative care and has been used in a number of Australian studies to test nursing assistants’ knowledge of palliative care [20, 23, 39]. Participant information, agreement to participate and demographic items were presented at the beginning of the booklet. Demographic data related to gender, age, whether born in Australia, years of experience in role and highest level of education were collected. For each instrument, instructions and generic information, contact details of the researcher, preceded its set of items. To ensure participant confidentiality, separate survey administration instructions and an individual envelope stamped ‘Confidential’ were provided for each paper copy. A link on the paper-based administration instructions provided the address to the online version of the questionnaires. Consent was implicit when participants indicated their agreement to proceed following the participant information. Whether participants completed the questionnaires in their own time or in work time was left to the discretion of the individual facilities and no assistance was given by RACF or research staff for completion.

PANA_Knowledge questionnaire

This questionnaire consisted of 28 items testing nursing assistants’ knowledge of a palliative approach. Response options to each knowledge item were True/False/Don’t Know with each correct item scored one and incorrect or DK responses scored zero with a possible total correct score of 28.

PANA_Skills questionnaire

The second questionnaire comprised 38 skill items, the purpose of which was to identify nursing assistants’ self-perceived skills when providing care with a palliative approach. The response options were: I know how to do this; I am unsure how to do this; I do not know how to do this. Each know how response was scored one and unsure/do not know how responses were scored zero with a total possible response of 38.

PANA_Attitudes questionnaire

The third questionnaire comprised 19 attitudinal statements, the purpose of which was to understand how nursing assistants feel about providing care with a palliative approach. A five-point Likert scale was used to collect responses, a number shown to be both reliable and low enough for participants to answer quickly [40]. Responses options were: Strongly Agree (SA); Agree (A); Unsure (U); Disagree (D); Strongly Disagree (SD). Items scored SA/A reflecting more positive responses were scored one point, items scored U, D, SD were scored zero with a total positive score of 19.

Palliative care quiz for nurses (PCQN)

The fourth and last instrument was the PCQN which is a 20-item quiz. Response option in the quiz were True/False/Don’t Know with each correct item scored one and incorrect or DK responses scored zero with a possible total correct score of 20.


Analyses of demographic data was conducted in Excel and all other analyses using the Statistical Package for the Social Sciences (SPSS) version 24 [41]. Analysis evaluated individual item properties for item difficulty, item discrimination, item-total correlation above 0.20 [42] and mean group scores.

Individual item analysis

Item difficulty and item discrimination provide important information about the performance capabilities of items to differentiate levels of the attributes being tested across the target population [43, 44]. Item discrimination and item difficulty indices are used to measure the reliability of scores and are independent of one another: an item can be easy or difficult whether it is discriminating or non-discriminating [43]. Difficulty and discrimination indices provide the criteria for item inclusion in the definitive version of the new knowledge instrument. Item difficulty analysis was not applied to the attitudinal items on the basis that attitudes are neither correct nor incorrect; nor was it applied to the self-reported skills items.

A random half of the sample (n = 174) was used for item discrimination and item difficulty, using the top 25% of scorers (n = 43) and the bottom 25% of scorers (n = 43). The number of correct items for each set was calculated, and for each item, the percentage of participants in each group who answered correctly was obtained. The two percentages (high scorers minus low scorers) were then subtracted from each other to arrive at the item’s discrimination index. The decision to retain an item was based on a positive discrimination index above .20 [28, 45, 46]. The item difficulty index provided the percentage of participants in the half-sample who scored each item correctly. Items that are answered either correctly or incorrectly by a high percentage of participants (e.g. 95% or more do not allow any discrimination among the users of the instrument and are candidates for deletion [36]. The values used in this study were 0.10 and 0.90.



Nursing assistants (n = 353) across 17 sites completed the demographic questions (Table 2). Of these, 348 also completed the knowledge questions; 343 completed demographic, knowledge, skill and attitude items, and 326 completed all questions including the PCQN. The majority was between 25 and 34 years of age and held the Certificate III in Aged Care. Forty-one per cent (n = 145) had been employed in their role for more than five years in the role. For those with the least experience in group 1, only 11.3% were born in Australia compared to 53% of staff in group 3. Role descriptions included Assistant in Nursing (AIN), care companion, care staff, aged care worker, care services employee (CSE), personal carer/care assistant. Five participants did not proceed to the first questionnaire. The reasons for staff not progressing beyond the demographic questions are not known.

Table 2 Demographic characteristics of sample Phase 4 (N = 353)

PANA_Knowledge questionnaire

The results of the Item Discrimination Index and the Item Difficulty Index are presented in Table 3.

Table 3 Item characteristics PANA_knowledge questionnaire

For the PANA_Knowledge Questionnaire, high scorers (top 25%) answered between 24 and 27 items correctly (M = 25, SD = 0.67); low scorers (bottom 25%) answered between 8 and 20 items correctly (M = 18.2, SD = 2.77). Twelve items were candidates for deletion, based on values below 0.10 and above 0.90 for difficulty and a negative discrimination index below 0.20 and had item-total correlations less than 0.20. One item (22) (Spiritual care identifies what is important to a person) scored above 0.10 and less than 0.90 for difficulty, the low item-total correlation made it too a candidate for deletion.

For the 16 knowledge items retained following the individual item analyses, Cronbach’s alpha was 0.69 (reduced from 0.79 on the original 28 items). To ensure alpha did not drop below the acceptable lower limit of α = 0.70 [44, 47], two items falling below 0.20 (0.16, 0.19) in discrimination were retained as both these items had scores below 0.90 for difficulty among the three groups of nursing assistants. Retaining these items increased alpha to 0.72 and enhanced content coverage. The removal of item 22 further improved Cronbach’s alpha from α = 0.72 to α = 0.74. A total of 17 items were retained as the final set for the PANA_Knowledge Questionnaire, shown in Table 3.

Descriptive statistics for the final knowledge items are shown in Table 4.

Table 4 Descriptive statistics for final knowledge items

PANA_Skills questionnaire

The results of the item discrimination, item-total correlation and alpha values for skills items are presented in Table 5. Twenty-five items were candidates for deletion based on a discrimination index below .20. The thirteen retained items correlated above 0.30. Cronbach’s alpha for individual items if the item was removed from the scale was between 0.79 and 0.81, with the final alpha on the remaining 13 items equal to 0.81. A total of 13 items were retained as the final item set for the PANA_Skills Questionnaire.

Table 5 Item characteristics PANA_Skills Questionnaire

Descriptive statistics for the final skills items are shown in Table 6.

Table 6 Descriptive statistics for final skills items

PANA_Attitudes questionnaire

The results of item discrimination, item-total correlation and alpha values for attitudinal items are presented in Table 7. Twelve items were candidates for deletion based on a discrimination index below 0.20. Three items showed a value below 0.20 for the item-total correlation, two of these were reverse-scored items and showed negative values. With only seven items above 0.20 for discrimination, Cronbach’s alpha dropped from 0.77 on the original 19 items to 0.36 on the revised sample of seven items. To ensure an acceptable alpha (0.70–0.80), six items were retained using 0.10 as the cut-off. However, three of these items had low item-total correlations and these were excluded. Cronbach’s alpha for the ten retained items was 0.80 and all item-total correlations were above 0.10. These 10 items were the final item set for the PANA_Attitudes Questionnaire.

Table 7 Item characteristics PANA_Attitudes Questionnaire

Descriptive statistics for the final attitudes items are shown in Table 8.

Table 8 Descriptive statistics for final attitudes items


Discriminative validity

To evaluate the discriminative capabilities of the PANA_Knowledge Questionnaire compared to the existing PCQN, a between-groups analysis and test of significance using ANOVA was performed. A two-way ANOVA was performed on the final 17 items of the PANA_Knowledge Questionnaires with level of experience as the first factor and level of education as the second factor (Hypothesis I and II). The results are presented in Table 9. Results trend towards a correlation between higher knowledge scores and experience in role as measured by the PANA_Knowledge Questionnaire (F = 2,390, df = 2, ρ = 0.093); this is not the case for level of education or for the interaction of education and experience on total knowledge scores (F = .853, df = 10, ρ = 0.578).

Table 9 Two-way ANOVA for knowledge scores: experience in role and level of education

A two-way ANOVA was also performed on the PCQN to compare the instrument’s performance with the PANA_Knowledge Questionnaire. The results showed no significant difference between groups of nursing assistants based on years or experience or level of education or the interaction of experience and education on total knowledge scores using the PCQN (see Table 10).

Table 10 Two-way ANOVA for PCQN scores: experience in role and level of education

A one-way ANOVA was performed on the PANA_Skills questionnaire and the PANA_Attitudes Questionnaire (hypotheses IV and V); the results are presented in Tables 11 and 12 and show that there was no significant difference between groups based on experience for skills scores. However, there was a significant difference between groups for the attitudes scores, demonstrating that the PANA_Attitudes Questionnaire was able to discriminate between groups based on experience (F = 5.252, df = 2, ρ = 0.006).

Table 11 One-way ANOVA for skills scores and experience in role
Table 12 One-way ANOVA for attitude scores and experience in role

Divergent validity

The prediction that there would be a divergent relationship between the PANA_Knowledge Questionnaire and the PCQN was supported by the results presented in Table 13.

Table 13 Correlation between scores PANA_Knowledge Questionnaire and PCQN

A correlation coefficient using Kendall’s tau statistic specifying a one-tailed test (the hypothesis was directional) was performed on the PANA_Knowledge Questionnaire to establish whether there was convergence or divergence with the PCQN. A small correlation (r = .166) was observed between scores on the PANA_Knowledge Questionnaire and those on the PCQN, with a significance value of less than 0 .001 (ρ < 0.001). This value indicates that the probability of getting a correlation coefficient of this magnitude in the sample of 350 participants is very low and that there is little correlation between the two knowledge instruments based on this sample.

Structural validity: exploratory factor analysis

A principal component analysis was conducted on the full final item set of knowledge, skill and attitudes. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy verified the sample’s adequacy for the analysis with a value close to 1.0 (KMO .903). The Bartlett’s test of sphericity, Approximate Chi-Square was 2435.617 and was significant (ρ. > 001) indicating that correlations between individual variables (items) were sufficiently different from one another and suitable for factor analysis [37]. As well, the correlation matrix was examined to identify that there were correlations between variables > 0.3 and that the data set was factorable. Components with loadings above 0.3 were extracted. A pattern and structure matrix was generated for interpretation. The analysis reduces variables into clusters and provides conceptual representation of the data. This analysis showed that the unrotated first principal component accounted for only 14.2% of the variance. Following oblique rotation, 13 components had eigenvalues over Kaiser’s criterion of 1, and in combination explained 58.7% of the variance. The scree plot and a parallel analysis confirmed five components that accounted for only 36% of the variance, whereas a minimum variance of 50% is recommended [44]. This low variance as well as the minimal variance in the correlations of the knowledge items with all other variables, the inclusion of mixed variables and problems with equivalence between variables resulted in the retention of the knowledge and skills items as separate indices and not as separately scored subscales within a single instrument [48] .

The PANA_Attitudes Questionnaire with response options provided as ordinal variables was independently subjected to a principal components analysis with oblique rotation. Analysis was conducted on a sample of 343 following the exclusion of five cases with missing values. The Kaiser-Meyer-Olkin Measure of sampling adequacy was .861 and the Bartlett’s Test of Sphericity was significant (ρ < 0.001) indicating that the data was factorable. The variables in the anti-image correlation ranged from .782 to .927. Two components made up the subscales. The pattern matrix was used for interpretation as follows: Component 1 represented holistic care; Component 2 represented a palliative approach. Table 14 attached presents the rotated component loadings, communalities, eigenvalues, percentage of variance, and alpha values.

Table 14 Factor structure PANA_attitudes questionnaire


Internal reliability was assessed with Cronbach’s alpha coefficient and test-retest reliability was assessed with intra class correlations (ICC). This was performed on all original items (n = 85) as well as on each separate item set using a two-way random effects model with a single measure and consistency of scores. The results are presented in Table 15. Twenty participants completed the three questionnaires of the instrument at time 1; only 16 completed the instrument at time 2, which was a small sample. One participant did not proceed to the PANA_Attitudes Questionnaire at time 2.

Table 15 Intraclass correlation, Cronbach’s alpha, 95% confidence intervals and significance for PANA_KSAq, PANA_Knowledge Questionnaire, PANA_Skills Questionnaire, PANA_Attitudes Questionnaire


The psychometric properties of the new instrument were evaluated to establish a final set of items that meet recommended criteria for validity and reliability. An analysis of the PANA_Knowledge Questionnaire’s psychometric properties indicates it has good internal consistency (0.74) and retest reliability (0.709) performed better than the PCQN in discriminating knowledge of a palliative approach between more and less experienced staff trending towards a correlation with higher knowledge scores and experience in role.

The 13-item PANA_Skills Questionnaire is an index of overall self-perceived skill in the delivery of a palliative approach. The instrument indicates high internal consistency (0.81) and adequate stability (0.608). The 10-item PANA_Attitudes Questionnaire, scored on a 5-point ordinal scale demonstrates strong discriminative validity (p = 0.006) demonstrated by its ability to detect a statistical difference between groups of nursing assistants based on experience in the role, and high internal consistency (0.80). The ICC value for the instrument is 0.335. This low value may reflect factors such as workplace satisfaction/dissatisfaction or changes in the emotional state of the participant [49], as well as the small sample completing the retest (n = 15).

Other key findings that provide insight for future workforce development emerged from these results. Firstly, nursing assistants demonstrated higher scores on the new PANA_Knowledge Questionnaire compared to scores on the PCQN [28], consistent with other studies reporting knowledge results for nursing assistants using this instrument or elements of it [20, 21, 23, 25]. Similarly, nursing assistants, when tested with the Palliative Care Survey (PCS) [22], have demonstrated low scores compared to registered or licensed nurses [24, 50]. While it is expected that these staff will perform better than nursing assistants considering educational differences related to biomedical and pharmacological knowledge, and health literacy, the instruments were not developed for nursing assistants and include elements that are not usual practice for nursing assistants. By comparison, the PANA Knowledge items satisfactorily tap into the knowledge required for the nursing assistant’s role and demonstrate a range of knowledge scores from low to high (M = 13.72 out of a possible 17 correct responses in the final instrument). In contrast to previous studies that have shown that nursing assistants have low overall knowledge of palliative care [20,21,22,23,24], this study shows that across the spectrum of experience, nursing assistants perform well when measured with a tailored instrument developed using the target group in the development process. A desirable quality of a knowledge instrument is that it can discriminate between participants with specific characteristics pertaining to that knowledge [28] as the PANA_Knowledge Questionnaire is able to demonstrate according to experience.

The second major finding is that there was no significant difference in nursing assistants’ skills across level of experience when evaluated with the new PANA_Skills Questionnaire. These findings are within the context of a relatively stable Australian RACF workforce and relatively long tenure for many staff with 34% having stayed in the same job for between one and four years and 28% between four and nine years [11]. A major implication in the lack of difference in skills scores between less and more experienced staff is that the skills of nursing assistants, across years of experience, remain largely static when measured with a tailored instrument. To date, the scope of practice of nursing assistants is not clearly defined, and there are no standards or regulatory framework for this workforce. Usual activities delegated to nursing assistants relate to personal care and activities of daily living in support of registered and enrolled nurses. Uncertainty regarding the role and scope of nursing assistants and the delegation of additional activities beyond usual practices have been identified as a result of role ambiguity, staffing levels and staffing models in RACFs [51]. This raises questions about the level of training and the extent to which current training meets the needs of residents, as well as ongoing professional development opportunities and support for nursing assistants. It is also worth commenting that fifty-two countries were identified by study participants as country of birth for those not born in Australia which points to the need for cultural awareness training to be included.

At 13 items, the PANA Skills Questionnaire satisfactorily assesses the skill set of nursing assistants, the largest group of care providers in RACFs [11]. Nursing assistants have previously returned significantly lower total practice (skills) scores compared with other groups of RACF staff such as RNs, ENs, and social workers when bereavement, planning and intervention, and provider coordination were also evaluated using the Palliative Care Survey (PCS) [22]. However, the PCS [22] evaluates skill domains that are not usual practice for nursing assistants. In an effort to address this, items of the PANA_Skills Questionnaire are specific to the role of a nursing assistant. They include the ability to observe changes, communicate and report these, and liaise effectively with other members of the care team, and are essential skills that can directly impact on the quality of residents’ care [52,53,54]. The inclusion of items specifically related to pain observation and the use of reliable pain assessment tools in the skill set of nursing assistants, are an important implication for the delivery of quality palliative care, with nursing assistants identified in previous studies as an untapped and untrained resource in pain management [55,56,57,58].

However, the difficulty of designing an instrument that captures what staff actually do instead of what they think they do or are supposed to do, has been acknowledged [22]. Also, because inexperienced care providers may underestimate the physical and emotional demands and skills required in providing care with a palliative approach [21], they may potentially think they know, for example, how to attend to a dying individual’s care when in fact they do not. Nevertheless, previously described as ‘expert behaviours’ and practices [24, 59], a new language is taking shape around what nursing assistants do, articulated as ‘clinical practice skills’ [17, 60] and, inarguably, the development of nursing assistants’ skills is needed to deliver quality palliative care, and in order to balance the technical, professional and emotional aspects of care [61].

The third major finding is that, when evaluated with the new PANA_Attitudes Questionnaire, a significant difference appears in nursing assistants ‘attitudes relative to their experience in the role. Those participants who had been in the role for more than five years demonstrated significantly more positive attitudes than those with less experience. This result warrants a more focused analysis of attitudes, and of factors known to affect workplace attitudes such as affective state and workplace satisfaction, which may be independent of education and experience [14].

Out of the original 19 attitudinal items, ten are included in the final instrument. When compared with other instruments designed to capture views and attitudes about palliative care [26, 62, 63], the new instrument more specifically reflects the nursing assistants’ role. The items illuminate the concerns and challenges faced by nursing assistants, and identify their unmet needs in education, training and support. For example, feeling comfortable when an individual says they are ready to die was a highly discriminating item in the PANA_Attitudes Questionnaire, indicating a wide variation between participants who feel comfortable when their resident expresses a readiness to die and those who do not. This reflects the findings of Nochomovitz et al. (2010) in their use of the Comfort Scale. The difference is that the Comfort Scale focuses on symptoms, treatment and comfort in talking about death and being present at death, while the new instrument delineates both positive and negative attitudes about providing a palliative approach in the context of the nursing assistants’ role. Making a difference to a person’s day and having an important role in pain assessment and management emphasises and recognises the value of nursing assistants. Overall, the finalised items of the PANA_Attitudes Questionnaire reflect important attitudes required for nursing assistants that are necessary for the provision of psychosocial and spiritual care, acceptance of dying and death, pain management, and managing grief and loss.

The fourth major finding is that there were no significant differences in the interaction between level of education and experience for the attributes of knowledge and attitudes. These results (taken together with the finding that the PANA_Knowledge Questionnaire trended towards a correlation in correct knowledge scores based on experience) support the hypothesis that experience measured as length of time in the nursing assistant’s role is a better indicator of knowledge of a palliative approach than education. In effect, years on the job translate into higher knowledge scores than education. These results derive from a sample in which the majority of participants in Group 1 and Group 2 held the industry recognised qualification (53.9% and 34.1 respectively), and the majority in Group 3 held the higher level Certificate IV in Aged CareFootnote 1 (39.3%) with Certificate III in Aged Care held by 29.7%.

It is understood that education alone does not change palliative care outcomes [64, 65], or practice development, or organisational culture [61]. It does however, play an important part in enhancing knowledge, providing an evidence base for practice and fostering confidence, as well as skill development and competence to enhance clinical practices [60]. Evaluation is part of this process and an important aspect of quality improvement, hence the importance of these tools as a means to identifying the gaps and educational shortfalls in nursing assistants’ knowledge, skills and attitudes. Development of core competencies is important because what has been found to matter most to patients and families across palliative care settings relates to those providing care and how care is provided [66]. Key indicators most associated with patients’ quality of life reported by bereaved families were whether health professionals, including nursing assistants, provided the desired physical comfort and emotional support to the dying person, supported shared decision-making, treated the dying person with respect, attended to the emotional needs of the family, and provided coordinated care [66]. These are all areas identified in the PANA questionnaires.

Strengths and limitations

A key strength of this study is the inclusion of nursing assistants in developing items, providing role-incumbent knowledge, and enhancing the relevancy of items; and, where possible, in directing the use of terminology. The input of aged care experts from four professional and industry groups increased the validity of the instrument. Pre-testing the instrument was an invaluable component of the development process and allowed the researcher to evaluate the usability of the instrument in the practice setting. There are some limitations to the new instrument and the study sample. First, the sample development and testing took place within greater metropolitan Sydney, an area that does not necessarily reflect regional or rural nursing assistants’ perspectives. The sample is, however, representative of an increasingly culturally diverse and overseas-born workforce that remains predominately female [11, 67]. The questionnaires were developed for an Australian context, although the wider international literature was used to corroborate what nursing assistants know, do and how they feel when providing care with a palliative approach, suggesting uptake in the international setting may be possible. The PANA_Skills Questionnaire measured self-perceived skills and not actual competency against specific criteria. This area of workforce evaluation will be considered for further research. Finally, it is not possible to create an item pool within one manageable instrument that captures every aspect of the construct with subtlety and the complexity of providing palliative care adhering to a defined scope of practice and other factors, such as cultural context. The instrument development process is the first step in a process of ongoing evaluation, review and refinement. Overtime, new items may need to be included to reflect any enhancement to the nursing assistants’ role, and their required knowledge and skills.


The PANA instruments have demonstrated preliminary evidence for validity and reliability for nursing assistants’ level of education and role responsibility providing a palliative approach in RACFs and, as such, are valid tools by which to identify the educational needs in palliative care of the largest cohort of the aged care workforce.

Availability of data and materials

Data sets used in the study analysis are available upon reasonable request from the corresponding author.


  1. Superseded in 2015 by the Certificate in Ageing Support



Aged Care Funding Instrument


Australian Institute of Health and Welfare


Content Validity Index


Enrolled Nurse (similar to Licensed Practical Nurse in the USA)


Facility Care Manager


Palliative Approach for Nursing Assistants


Residential Aged Care Facility


Registered nurse


Registered Training Organisation


  1. Abbey J, et al. Palliative care in long-term care: a system in change. Int J Older People Nursing. 2006;1(1):56–63.

    Article  Google Scholar 

  2. Australia Institute of Health and Welfare. Palliative care in residential aged care Palliative care services in Australia 2018 [cited 2018 24 August]; Available from:

  3. Pivodic L, et al. Quality of dying and quality of end-of-life care of nursing home residents in six countries: an epidemiological study. Palliat Med. 2018;32(10):1584–95.

    Article  Google Scholar 

  4. Meier DE, Lim B, Carlson MDA. Raising the standard: palliative care in nursing homes. Health Aff. 2010;29(1):136–40.

    Article  Google Scholar 

  5. Australian Institute of Health and Welfare. Residential aged care and home care 2014-15; 2016. [cited 2017 6 September]; Available from:

  6. Australian Institute of Health and Welfare. GEN aged care data: People’s care needs in aged care; 2018. [cited 2019 18 January]; Available from:

  7. Commonwealth of Australia. Guidelines for a palliative approach in residential aged care - enhanced version. Canberra; 2006.

  8. Palliative Care Australia. National Palliative Care Standards; 2018. PCA

    Google Scholar 

  9. Reimer-Kirkham S, et al. ‘Close to’ a palliative approach: nurses’ and care aides’ descriptions of caring for people with advancing chronic life-limiting conditions. J Clin Nurs. 2016;25(15–16):2189–99.

    Article  Google Scholar 

  10. Department of Employment, A.G. Job outlook. Nursing support and personal care workers; 2016. Nursing Support and Personal Care Workers]. Available from:

    Google Scholar 

  11. Mavromaras K, et al. 2016 National aged care workforce census and survey – the aged care workforce, 2016; A.G. Deparment of Health, editor. 2017.

    Google Scholar 

  12. Willis E, et al. Meeting residents’ care needs: a study of the requirement for nursing and personal carestaff. National aged care staff and skills mix project report 2016; 2016. Australian Nursing and Midwifery Federation Australian Nursing and Midwifery Federation (ANMF)

    Google Scholar 

  13. Fryer S, et al. “Sometimes I’ve gone home feeling that my voice hasn’t been heard”: a focus group study exploring the views and experiences of health care assistants when caring for dying residents. BMC Palliat Care. 2016;15(1):1–9.

    Article  Google Scholar 

  14. Cohen L, O’Connor M, Blackmore AM. Nurses’ attitudes to palliative care in nursing homes in Western Australia. Int J Palliat Nurs. 2002;8(2):88–94 96-98.

    Article  Google Scholar 

  15. Gamondi C, Larkin P, Payne S. Core competencies in palliative care: an EAPC white paper on palliative care education - part 1. Eur J Palliat Care. 2013;20(2):86–91.

    Google Scholar 

  16. Frey R, et al. Necessary but not yet sufficient: a survey of aged residential care staff perceptions of palliative care communication, education and delivery. BMJ Support Palliat Care. 2016;6(4):465–73.

    Article  Google Scholar 

  17. Martyn J-A. Continuing education needs of personal care workers in residential aged care facilities. Hervey Bay: University of the Sunshine Coast; 2016.

    Google Scholar 

  18. Community Services and Health Industry Skills Council. CHC33015 certificate III in individual support; 2015. [cited 2016; Available from:

    Google Scholar 

  19. Community Services and Health Industry Skills Council, C.S.a.H.I.S. council. CHCPAL001 deliver care services using a palliative approach: Commonwealth of Australia; 2015.

  20. Ford R, McInerney F. An evaluation of aged-care workers’ knowledge of and attitudes toward the palliative approach. Res Gerontol Nurs. 2011;4(4):251–9.

    Article  Google Scholar 

  21. Pfister D, et al. German nursing home professionals’ knowledge and specific self-efficacy related to palliative care. J Palliat Med. 2013;16(7):794–8.

    Article  Google Scholar 

  22. Thompson S, et al. A measure of palliative care in nursing homes. J Pain Symptom Manag. 2011;41(1):57–67.

    Article  Google Scholar 

  23. Ronaldson S, et al. A study of nurses’ knowledge of a palliative approach in residential aged care facilities. Int J Older People Nursing. 2008;3(4):258–67.

    Article  Google Scholar 

  24. Unroe K, et al. Nursing home staff palliative care knowledge and practices: results of a large survey of frontline workers. J Pain Symptom Manag. 2015;50(5):622–9.

    Article  Google Scholar 

  25. Kada O, et al. Palliative care in nursing homes: results of a survey about knowledge and self-efficacy of nursing staffSchmerz; 2017. p. 1–8.

    Google Scholar 

  26. Leclerc B-S, et al. Attitudes toward death, dying, end-of-life palliative care, and interdisciplinary practice in long term care workers. J Am Med Dir Assoc. 2014;15(3):207–13.

    Article  Google Scholar 

  27. Karacsony S, et al. Measuring nursing assistants’ knowledge, skills and attitudes in a palliative approach: a literature review. Nurse Educ Today. 2015;35(12):1232–9.

    Article  Google Scholar 

  28. Ross MM, McDonald B, McGuinness J. The palliative care quiz for nursing (PCQN): the development of an instrument to measure nurses’ knowledge of palliative care. J Adv Nurs. 1996;23(1):126–37.

    Article  CAS  Google Scholar 

  29. DeVon H, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39(2):155–64.

    Article  Google Scholar 

  30. Polit D, Yang F. Measurement and the measurement of change. A primer for the health professions: Wolters Kluwer; 2016.

  31. (CSHISC), C.S.a.H.I.S.C. CHC43015 certificate IV in ageing support (release 2); 2015. [cited 2016; Available from:

    Google Scholar 

  32. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–6.

    Article  CAS  Google Scholar 

  33. Karacsony S, et al. Assessing nursing assistants’ competency in palliative care: an evaluation tool. Nurse Educ Pract. 2018;33:70–6.

    Article  Google Scholar 

  34. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–5.

    Article  Google Scholar 

  35. Streiner D. A guide for the statistically perplexed: selected readings for clinical researchers: University of Toronto Press; 2013.

  36. Clark LA, Watson D. Constructing validity: basic issues in objective scale development. Psychol Assess. 1995;7(3):309–19.

    Article  Google Scholar 

  37. Field A. Discovering statistics using SPSS. Third ed. London: Sage; 2009.

    Google Scholar 

  38. Rouquette AB. Sample size requirements for the internal validation of psychiatric scales. Int J Methods Psychiatr Res. 2011;20(4):235–49.

    Article  Google Scholar 

  39. Pitman S. Evaluating a self-directed palliative care learning package for rural aged care workers: a pilot study. Int J Palliat Nurs. 2013;19(6):290–4.

    Article  Google Scholar 

  40. Wakita T, Ueshima N, Noguchi H. Psychological distance between categories in the Likert scale: comparing different numbers of options. Educ Psychol Meas. 2012;72(4):533–46.

    Article  Google Scholar 

  41. IBM. IBM SPSS statistics. Chicago: IBM. Statistical computer progrm

  42. Pope G. Psychometrics 101: item total correlationin; 2009.

    Google Scholar 

  43. DeMars C. In: Beretvas N, Leavy P, editors. Item response theory. Understanding statistics: measurement. New York: Oxford University Press; 2010.

    Chapter  Google Scholar 

  44. DeVellis R. Scale development: theory and applications. 3rd ed. Thousand Oaks: Sage; 2012.

    Google Scholar 

  45. D’Sa JL, Visbal-Dionaldo ML. Analysis of multiple choice questions: item difficulty, discrimination index and distractor efficiency. Int J Nurs Educ. 2017;9(3):109–14.

    Article  Google Scholar 

  46. Rush BR, Rankin DC, White BJ. The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Med Educ. 2016;16(1):250.

    Article  Google Scholar 

  47. Nunnally J. Psychometric theory. 3rd ed. New York: McGraw Hill; 1994.

    Google Scholar 

  48. Ackerman RA, et al. What does the narcissistic personality inventory really measure? Assessment. 2011;18(1):67–87.

    Article  Google Scholar 

  49. Kim J, Lim J-S, Bhargava M. The role of affect in attitude formation: a classical conditioning approach. J Acad Mark Sci. 1998;26(2):143–52.

    Article  Google Scholar 

  50. Smets T, et al. The palliative care knowledge of nursing home staff: the EU FP7 PACE cross-sectional survey in 322 nursing homes in six European countries. Palliat Med. 2018;32(9):1487–97.

    Article  Google Scholar 

  51. Laging B, et al. Recognition and assessment of resident’ deterioration in the nursing home setting: a critical ethnography. J Clin Nurs. 2018;27(7–8):1452–63.

    Article  Google Scholar 

  52. Phillips JL, et al. Multi-faceted palliative care intervention: aged care nurses’ and care assistants’ perceptions and experiences. J Adv Nurs. 2008;62(2):216–27.

    Article  Google Scholar 

  53. Mitchell G, et al. Enhancing palliative care in rural Australia: the residential aged care setting. Aust J Prim Health. 2011;17(1):95–101.

    Article  Google Scholar 

  54. Page CG, Rowles GD. “It doesn’t require much effort once you get to know them” certified nursing assistants’ views of communication in long-term care. J Gerontol Nurs. 2016;42(4):42–51.

    Article  Google Scholar 

  55. Liu JYW. Exploring nursing assistants’ roles in the process of pain management for cognitively impaired nursing home residents: a qualitative study. J Adv Nurs. 2014;70(5):1065–77.

    Article  Google Scholar 

  56. Dobbs D, et al. Certified nursing assistants’ perspectives of nursing home residents’ pain experience: communication patterns, cultural context, and the role of empathy. Pain Manag Nurs. 2014;15(1):87–96.

    Article  Google Scholar 

  57. Holloway K, McConigley R. Understanding nursing assistants’ experiences of caring for older people in pain: the Australian experience. Pain Manag Nurs. 2009;10(2):99–106.

    Article  Google Scholar 

  58. De Witt Jansen B, et al. Exploring healthcare assistants’ role and experience in pain assessment and management for people with advanced dementia towards the end of life: a qualitative study. BMC Palliat Care. 2017;16(1).

  59. McClement S, Wowchuk S, Klaasen K. Caring as if it were my family: health care aides perspectives about expert care of the dying resident in a personal care home. Palliat Support Care. 2009;7(4):449–57.

    Article  Google Scholar 

  60. Williams J, et al. Raising the bar: a pilot evaluation of the effect of a residential aged care workforce development model on staff and residents. J Nurs Home Res. 2016;2:50–6.

    Google Scholar 

  61. Williams L, et al. Improving skills and care standards in the support workforce for older people: a realist synthesis of workforce development interventions. BMJ Open. 2016;6(8)1–11.

    Article  CAS  Google Scholar 

  62. Eager K, et al., The rural palliative evaluation tool kit: a compendium of tools to aid in the evaluation of palliative care projects. 2003, The Centre for Health Services Development, University of Wollongong.

  63. Phillips J, Salamonson Y, Davidson P. An instrument to assess nurses’ and care assistants’ self-efficacy to provide a palliative approach to older people in residential aged care: a validation study. Int J Nurs Stud. 2011;48(9):1096–100.

    Article  Google Scholar 

  64. Frey R, et al. Family experiences of the transition to palliative care in aged residential care (ARC): a qualitative study. Int J Palliat Nurs. 2017;23(5):238–47.

    Article  Google Scholar 

  65. Mitchell SL, Kiely DK, Hamel MB. Dying with advanced dementia in the nursing home. Arch Intern Med. 2004;164:321–6.

    Article  Google Scholar 

  66. Bainbridge D, Hsien S. Measuring the quality of palliative care at end of life: an overview of data sources. Healthly Aging Clin Care Elderly. 2016;8:9–15.

    Article  Google Scholar 

  67. Negin J, et al. Foreign-born health workers in Australia: an analysis of census data. Hum Resour Health. 2013;11(1):1–9.

    Article  Google Scholar 

Download references


We would like to thank all the nursing assistants, facility care managers, registered nurses and academics who participated in the four phases of this study. We are also grateful for statistical advice and support from the University’s statistical consultant and from the School of Social Sciences and Psychology.


The study was funded from PhD candidature funds for the principal author. No external funding was sought or obtained.

Author information

Authors and Affiliations



SK, EC, AJ, ME, AG contributed to the conceptualisation and design of the study; SK conducted all data collection and analysis. EC, AG participated in the data analysis. SK wrote the first draft of this paper and made all revisions. All authors read the paper, made suggestions and approved final manuscript.

Corresponding author

Correspondence to Sara Karacsony.

Ethics declarations

Ethics approval and consent to participate

Received from Western Sydney University’s Human Research Ethics Committee. The approval number is H9963. Consent sought from study participants using institutional consent forms. Written consent obtained for Phase 1. Consent for Phases 2–4 was obtained when participants entered the respective surveys.

Consent for publication

Sought from study participants using institutional consent forms for data in aggregated form only.

Permission to use the PCQN was sought by email from the person responsible for overseeing the PCQN on behalf of the authors.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Guidelines Framework (DOCX 16 kb)

Additional file 2:

The Palliative Approach for Nursing Assistants (PANA) questionnaires URI: (DOCX 28 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karacsony, S., Good, A., Chang, E. et al. An instrument to assess the education needs of nursing assistants within a palliative approach in residential aged care facilities. BMC Palliat Care 18, 61 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: