Table 1 Characteristics of included studies

Dunckley [19], 2005, UK, 0.45

POS

Palliative care Outcome Scale (POS)

The POS is a 10-item questionnaire covering physical and psychological symptoms; and spiritual, practical and emotional concerns.

X

X

Qualitative study

Practitioners working in nursing homes or clinical hospices

To identify facilitators and barriers to implementing outcome measures.

Barriers to implementing POS included, a top-down decision-making approach, time-consuming tools, limited resources for data analysis and a lack of practitioner knowledge of the importance of using tools. Facilitators to successful implementation included practitioners being involved in implementation decisions and using a tool that can be adapted to clinical practice and organisational needs.

Tavares [20], 2017, Brazil, 0.7

X

X

Observational study

Patients admitted to a specialist palliative care unit. Mean age 77.5 years.

To implement the POS in a specialist palliative care inpatient unit in daily practice.

POS is feasible to implement and improves quality of care. Pain was particularly improved between timepoints.

Kane [21], 2018, UK, 0.6

Kane [22], 2017, UK,0.9

Integrated Palliative care Outcome Scale (IPOS)

The IPOS has 10 questions with two open questions covering patients’ main concerns and symptoms, respectively, and a five-point Likert scale (0–4) accompanying common symptoms, patient and family distress, existential well-being, sharing feelings with family, information available and practical concerns.

X

X

X

A parallel, mixed methods embedded study

Advanced chronic heart failure patients in a nurse-led chronic heart failure disease management clinic. Mean age 75 years. Plus 4 nurses.

To examine the feasibility and acceptability of using a patient reported outcome measure and its potential to influence patient perceptions of care.

IPOS was feasible and acceptable to patients and practitioners for use in clinical care and research. IPOS also allowed patients to become more engaged in their clinical care and highlight their unmet needs.

Ellis-Smith [23], 2017, UK, 0.85

Integrated Palliative care Outcome Scale for Dementia (IPOS-Dem)

A 28-item questionnaire with all questions, apart from the first, rated on a 5-point scale.

X

X

X

A multi-method qualitative study

Care home residents with dementia, family members, care home practitioners, GPs and district nurses.

To examine the content validity, acceptability and comprehension of IPOS-Dem for routine use in long-term care settings for people with dementia and to refine the tool.

IPOS-Dem is a comprehensive and acceptable way to detect symptoms and problems for those with dementia.

It is also acceptable as a carer-reported measure. Refinements have been made to maximise caregiver expertise.

Ellis-Smith [24], 2018, UK, 0.9

A qualitative study with an embedded quantitative component

Care home residents with dementia, family members, care home practitioners, GPs and district nurses.

To explore the mechanisms of action, feasibility, acceptability and implementation requirements of a the IPOS-Dem

Key mechanisms of action were identified, and a theoretical model was developed.

IPOS-Dem was shown to be acceptable and feasible. Assessment and management of symptoms and concerns is supported by IPOS-Dem.

Salisbury [25], 2018, UK, 0.86

Mann [26], 2019, UK, 0.85

Thorn [27] 2020, UK, 0.85

3D Approach

3D Approach

Replaces disease specific reviews of each health condition with one 6-montly comprehensive multidisciplinary review, including medication review.

X

X

X

A pragmatic cluster-randomised controlled trial

Patients of participating GP surgeries with at least 3 chronic conditions. Mean age 71 years.

To implement, at scale, a new approach to managing patients with multimorbidity in primary care and to assess its effectiveness.

The 3D intervention did not improve patients’ quality of life.

Both implementation and intervention failure were cited as reasons for failure.

Cost effectiveness was equivocal.

Forbat [28], 2019, Australia, 0.73

Liu [29], 2020, Australia, 0.93

Palliative care needs rounds

Needs rounds are monthly clinical meetings that are conducted at the care facility that integrate a specialist palliative care perspective into nursing home care.

X

X

A prospective stepped-wedge cluster randomised control trial

Care home residents. Mean age 85 years. Care home practitioners interviewed.

To determine whether a model of care providing specialist palliative care in care homes, called Specialist Palliative Care Needs Rounds, could reduce length of stay in hospital.

The primary outcome was length of stay in acute care. Secondary outcomes included number and cost of hospitalisations.

Palliative care needs rounds reduced the number of hospitalisations and length of stay.

Forbat [30], 2018, Australia, 0.5

Development of checklist

A grounded theory ethnography

To describe the activities, thought processes and activities of practitioners that are generated within and from needs rounds. To develop a model that explains what occurs in needs rounds and distil checklist from this. To finalise the checklist.

The checklist was suitable to support the integration of specialist palliative care into residential care.

Waller [31], 2012, Canada, 0.91

Needs Assessment Tool

Needs Assessment Tool: Progressive Disease Cancer (NAT: PD-C)

One-page practitioner completed questionnaire

X

Interrupted time series trial

Advanced cancer patients recruited from medical oncology, radiation oncology, and haematology outpatient clinics with an average age of 67 years.

To assess the impact of the systematic and ongoing use of the Guidelines and NAT: PD-C on patient outcomes including level of need, quality of life, anxiety, and depression.

The NAT:PD-C reduces health system and information needs, and patient care and support needs.

Janssen [32], 2019, The Netherlands, 0.73

Needs Assessment Tool: Progressive Disease Heart Failure (NAT: PD-HF)

One-page practitioner completed questionnaire

X

Mixed methods

Outpatients with diagnosis of chronic heart failure. Average age 84.4 (SD: 7.7) years.

To translate and study the feasibility and acceptability of the NAT:PD-HF.

The NAT:PD-HF identified palliative care needs in all participants, and triggered action to address these in half. Palliative care communication skills training is required when implementing this tool.

Actcherberg [33], 2001, The Netherlands, 0.86

Resident Assessment Instrument

Resident Assessment Instrument (RAI)

The RAI consists of a structured screening questionnaire [the Minimum Data Set (MDS)], an algorithm that links the information from the MDS to certain important problem areas, and triggers protocols for these problem areas if required.

X

X

Non randomised controlled trial

Residents admitted for long term care in a somatic ward. Average age 78.6 years

Does the implementation of the RAI method improve the quality of the co-ordination of care in Dutch nursing homes?

Improvements in case history, care plan, end of shift reporting, communication, patient allocation and patient report in the RAI group. RAI has the potential to improve the quality of co-ordination of care in nursing homes

Gestsdottir [34], 2015, Iceland, 0.91

InterRAI Palliative Care

The InterRAI PC is divided into 16 domains: demographics, health conditions, oral and nutritional status, skin condition, cognition, communication, mood and behaviour, psychosocial wellbeing, physical functioning, urinary and bowel continence, medications, treatments and procedures, responsibility/directives, social relationships, discharge or death, and assessment information.

X

Longitudinal

Patients using the services of the palliative consultation team and hospital general and palliative care units

To assess the symptoms and functional status of patients at the point of admission to specialised palliative care in Iceland and to investigate whether symptoms and functional status change over time. Also, to examine the difference in symptoms and functional status between care settings. A secondary aim was to participate in the development of interRAI PC assessment tool

Symptom burden and functional loss were significantly experienced by patients from admission to discharge or death.

Symptoms indicating progressive deterioration also increased in frequency and severity. Physical and cognitive function decreased at all levels. Inpatients had more symptoms and experienced more functional decline than home-care patients. The interRAI PC version 8 supported capture of important clinical information and monitoring changes over time.

Hill [35], 2002, New Zealand, 0.6

The Missoula-VITAS Quality of Life Index (MVQOLI)

The Missoula-VITAS Quality of Life Index (MVQOLI)

The Missoula-VITAS Quality of Life Index (MVQOLI) is a 25-item patient-centred index that weights each of five QOL dimensions by its importance to the respondent.

X

A pre-test/post-test quasi-experimental design

72 hospice patients and 10 nursing practitioners. Ages ranging from 20 to 89 years old.

To examine the concept and measurement of quality of life (QOL) in terminally ill patients and how QOL can be improved within a hospice setting

Providing nurses with access to information on the patient’s QOL perspective better prepares them

to meet the patient’s QOL needs. This results in clinically significant improvements to patient QOL.

Schwartz [36], 2005, USA, 0.77

Missoula-VITAS Quality of Life Index - Revised (MVQOLI-R)

As above, without the weighting.

X

X

Psychometric evaluation and intervention study

End-stage renal disease patients and hospice, or long-term care facility, patients. Mean age 66.3 years.

To evaluate the MVQOLI-R from both psychometric and clinimetric perspectives.

The MVQOLI-R has clinical utility as a patient QOL assessment tool and may support communication between patients and clinicians.

Rockwood [37], 2000, Canada, 0.79

Comprehensive Geriatric Assessment

CGA and Goal Attainment Scale (GAS)

Unspecified group of tools used together in CGA.

GAS is used to record patient goals and the achievement of those goals.

X

X

Randomized, controlled, single-blinded trial

Frail patients living in a rural community. Mean ages 82.2 and 81.4 years.

Testing of the CGA in the common, but constrained, environment of frail older patients without nearby access to specialized care.

Intervention group more likely to achieve their goals. No change or difference in function, QOL, survival or time to institutionalisation.

Parlevliet [38], 2012,

The Netherlands

0.86

CGA:

Comprising of: Charlson’s comorbidity index; Activities of Daily Living; Instrumental Activities of Daily Living; MMSE; SDGS; SNAQ; VAS; EuroQol-6D; IQCOD-SF; NPI-q; CAM; EDIZ; De Jong-Gierveldschaal

X

Cross sectional comparative and feasibility study

Patients with end stage renal disease aged 65 years or above, either receiving peritoneal dialysis or haemodialysis in hospitals with dialysis facilities

To perform a systematic CGA to investigate somatic, psychological, functional and social function in a group of older dialysis patients. Secondly, we aimed to place our findings in a broader perspective by comparing our group to a population of elderly cancer patients who likewise suffered from an end-stage chronic progressive disease. Finally,

we asked the multidisciplinary team for their opinion on the feasibility of the systematic CGA and the relevance

of its outcome.

Geriatric conditions were highly prevalent among elderly dialysis patients and prevalence’s were comparable in both intervention and control populations.

The CGA was feasible for use of recognition of conditions and overburdened carers.

Basic [39], 2002, Australia

0.86

CGA

Comprising of: Activities of Daily Living; Instrumental Activities of Daily Living; MMSE; GDS; SSI; Waterlow Risk Assessment Scale

X

Observational study

Older people presenting to the emergency department who were considered at high risk of admission but who were not severely ill. Mean age 79.4 years

To evaluate the ability of the nurse to assess high risk elderly patients comprehensively. A secondary aim was to explore patient characteristics associated with referral to community aged care services from the emergency department.

A single nurse working in a busy emergency department can successfully identify patients with increased care needs, and direct high-risk patients to existing services.

Mariano [40], 2015, Canada

0.73

Geriatric assessment

Geriatric assessment (unspecified)

X

Pilot study

Cancer patients. Mean age 77 years. Hospitals

To evaluate the feasibility of GA in this frail, historically difficult-to-study population. Secondary objectives were to describe the level of deficits detected on GA, to assess whether hospital-based clinicians recognized and addressed these deficits, and to describe hospital-based outcomes including length of stay, discharge disposition, and 30-day readmission rates

GA was feasible in this population. Hospitalized older cancer patients experience more functional and psychosocial issues. Clinical recognition and management of these issues was poor.

GA tools can be used to inform guide referrals to appropriate services.

Jadczak [41], 2017, Australia

0.59

Geriatric Assessment Comprising of: FRAIL screen; CCI; SF-36; TMT; MNA-SF; RCS; Lawton IADL

X

Observational study

Patients from a Geriatric Evaluation and Management Unit (GEMU) screened pre-frail or frail on the FRAIL Screen. Mean age 85.37 years.

To determine the feasibility of standardised geriatric assessments and standard physical exercises in hospitalised pre-frail and frail older adults

The FRAIL Screen, MNA-SF, Rapid Cognitive Screen, Lawton iADL and the physical exercises were deemed to be feasible with only minor comprehension, execution and safety issues. The TMT was not considered to be feasible and the SF-36 should be replaced by its shorter form, the SF-12.

Pepersack [42], 2008, Belgium, 0.67

Minimum Geriatric

Screening Tools (MGST)

Battery of tools including: ADL; IADL; CSDD; Socios Scale; MUST; pain indicators; ISAR

X

Prospective observational survey

Patients attending an acute geriatric unit, mean age 83.3 years.

The aims of this project were: 1) to assess the feasibility of a MGST within the teams of Belgian geriatric units; 2) to assess the efficacy of a MGST on the detection rate of the geriatric problems; and 3) to analyse quality variables within the data collected.

MGST leads to better assessment of geriatric domains (functional, continence, cognition, depression, nutrition, pain, social), apart from falls.

Cheang [43], 2014, Australia, 0.72

Advance Care Planning

ACP screening interviews

Guided interview

X

Cross-sectional

Patients ages 80 years or over, who have been admitted for at least 48 h to an adult medical/surgical ward

To assess the prevalence of advanced care documents and documented medical orders regarding end-of-life care in the medical record of elderly inpatients and to explore the feasibility and acceptability of an advanced care planning screening interview.

Advance Care Directives and correct documentation of suitable decision-maker were uncommon in the medical records. The ACP screening interview appears feasible and acceptable and may be a useful tool for identifying suitable decision-maker and patients willingness to discuss ACP further.

Silvester [44] 2013, Australia

0.68

Advance Care Plan

Two-sided questionnaire asking about values and beliefs, unacceptable health condition, specific treatments wanted and unwanted.

X

Audit of pre-existing documentation and pilot study

No patients recruited.

The development of the aged care specific Advance Care Plan template, the pre-implementation quality of ACP documents and the performance of the newly developed Advance Care Plan template

Standardised procedures and documentation are needed to improve the quality of processes, documents and outcomes of ACP.

Miller [45], 2019, Australia

0.75

Advance Care Planning

GP completes referral to GPN including health and social information. GPN conducts ACP discussion using an Advance Care Planning workbook and Advance Care Directive template was used to guide discussions and to record the patient’s wishes if required.

X

Qualitative interviews

Patients of participating GP surgeries. Mean age 81 years.

To understand how patients experienced involvement in advanced care planning in the general practice setting when common barriers to uptake were addressed and what impact this has on patients and their families.

GPNs are able to hold ACP conversations with patients when provided with training and support. GPNs involvement in these conversations can benefit patients. Some patients may feel uncomfortable communicating results of ACP conversations with family.

Sudore [46], 2013, USA

0.9

Advance Care Planning Engagement survey

Survey with two sections containing 31 items in ‘process measures’ and 18 items in ‘action measures’.

X

Development and psychometric evaluation

Patients recruited from hospitals, outpatient clinics and nursing homes. Mean age 69.3 years.

To develop and validate a survey designed to quantify the process of behaviour change in the advance care planning process.

The Advance Care Planning Engagement Survey measuring behaviour change and multiple advance care planning actions demonstrated good reliability and validity.

Bristowe [47], 2015, UK

0.85

The AMBER Care Bundle

The AMBER Care Bundle

This intervention

has an algorithmic approach and is intended to encourage the clinical team to develop and document a clear medical plan and consider anticipated outcomes and resuscitation and escalation status; this is revisited daily.

X

X

Mixed methods observational study

Patients in the acute hospital setting who are deteriorating, clinically unstable, with limited reversibility and at risk of dying in the next 1–2 months. Mean age 77 years.

Aims to examine the experience of care supported by the AMBER care bundle compared to standard care in the context of clinical uncertainty, deterioration and limited reversibility

Patients in the intervention group appeared to have higher awareness of prognosis. This does not translate to better quality communication and information was judged less easy to understand.

Koffman [48], 2019, UK, 0.82

Randomised controlled trial

Hospital inpatients. 38.5% were aged 60–79 years old, 46.2% were aged over 80 years old.

To investigate the feasibility of a cluster RCT of the AMBER care bundle.

The cluster RCT was feasible. However, optimal recruitment was prevented by impracticalities in the fundamental issues in operationalising the intervention’s eligibility criteria.

McMillan [49], 2011, USA, 0.86

Tools used together as a package

Patient instruments

Palliative Performance Scale (PPS)

Memorial Symptom Assessment Scale-Revised (MSAS)

Hospice Quality of Life Index-14 (HQLI-14)

Instruments for Both Patients and Caregivers

Center for Epidemiological Study-Depression Scale (CES-D)

Spiritual Needs Inventory

Short Portable Mental Status Questionnaire

X

Clinical trial

Patients newly admitted to hospice care and their family caregivers. Patient mean age 72.66 years, caregiver mean age of 65.37 years.

To determine the efficacy of providing systematic feedback from standardized assessment tools for hospice patients and caregivers in improving hospice outcomes compared to the usual clinical practice

Depression scores were improved in the intervention group.

Standard care received was so good that the overall quality of life improved as a result. This prevented improvement in other variables.

Gilbert [50], 2012, Canada, 0.55

Edmonton Symptom Assessment Symptom (ESAS),

Palliative Performance Scale and Advance Care Plan

X

X

Mixed methods quality improvement

Cancer patients receiving community palliative care

The project involved 1) implementation of the ESAS for symptom screening, 2) use of “rapid-cycle change” quality improvement processes to improve screening and symptom management, and 3) improvements in integration and access to palliative care services.

The Provincial Palliative Care Integration Project demonstrated that by using rapid-cycle change and collaborative approaches, symptom screening and responses can be improved. Improvements can occur in the long and short term but require changes in system design and changes in clinical practice culture.

Mercandante [51], 2019, Italy, 0.82

Patient Dyspnea Goal, Patient Dyspnea Goal Response and Patient Global Impression

Patient Dyspnea Goal is an assessment tool to tailor symptom management, providing a therapeutic ‘target’. Patient Dyspnea Goal Response is the achievement of the goal. Global impression is global rating-of-change scale that assesses patients’ subjective response based on the individual feeling of improvement or deterioration.

X

X

Secondary analysis

Advanced cancer patients admitted to palliative care units. Mean age 68.2 years.

To characterize the Patient Dyspnea Goal and Patient Dyspnea Goal Response, and Patients Global Impression after 1 week of a comprehensive symptom management. The secondary aim was to find possible factors influencing the clinical responses assessed as Patient Dyspnea Goal Response and Patient Global Impression.

Patient Dyspnea Goal Response and Patient Global Impression seem to be relevant for evaluating the effects of a comprehensive management of symptoms, assisting decision making process.

Cox [52], 2011, UK, 0.5

Edmonton Symptom Assessment Scale (ESAS) and the Euro-QoL (EQ-5D)

Technologies HealthHUB (held by patients) and CareHUB (held by clinicians) used as prompts to complete ESAS and EQ-5D questionnaires to assess symptoms and QoL respectively.

X

X

Mixed methods

Hospice patients with a diagnosis of lung cancer

This study had two aims: [1] to test and evaluate the support provided to patients by the computerized assessment tool and [2] to determine the clinical acceptability of the technology in a palliative care setting.

Clinicians acknowledged patient and practice benefits of computerised patient assessment but highlighted the importance of clinical intuition over standardised assessment.

While clinicians were positive about palliative care patients participating in research, they did indicate concerns around age and potential for rapid deterioration. The contribution of e-technology needs to be prompted, particularly in its potential to improve patient outcomes and experience, to encourage acceptance of its use in palliative care.

Hockley [53], 2010, UK, 0.82

Liverpool Care Pathway and Gold standards framework

X

X

Evaluation

Nursing home residents aged 66–103 years. 51% of residents had 3 or more diagnoses.

Using tools to help improve end-of-life care in care homes

There was a highly statistically significant increase in use of Do Not Attempt Resuscitation (DNAR) documentation, advance care planning and use of the LCP. An apparent reduction in unnecessary hospital admissions and a reduction in hospital deaths post-study were also found.

Jennings [54], 2016, USA, 0.95

Physician Orders for Life-Sustaining Treatment (POLST)

Legal document indicating preferences for life sustaining treatment

X

Observational study

Residents in nursing facilities with a mean age of 78 years.

To evaluate the use of POLST among California nursing home residents, including variation by resident characteristics and by nursing home facility.

State-wide nursing home data show broad uptake of POLST in California without racial disparity. However, variation in POLST completion among nursing homes indicates potential areas for quality improvement.

Krumm [55], 2014, Germany, 0.8

Minimal Documentation system for Palliative Care (MIDOS)

One-page symptom assessment tool

X

X

Qualitative multiple-unit study

Nurses and care assistants from specialist dementia units

To describe health professionals’ experiences of assessing the symptoms of people with dementia using a cancer-patient-oriented symptom-assessment tool from a palliative care context

The MIDOS tool was perceived as a helpful and valuable. Practitioners expressed some concerns regarding the subjective nature of perceiving symptoms and clinical decision making. The use of tools such as this has the potential to enhance the quality of palliative care in dementia care.

Landi [56], 2001, Italy, 0.93

Minimum Data Set for Home Care (MDS-HC)

X

X

Single blind randomized controlled trial

Older people living in the community receiving home care services

To test the effectiveness—in standardized home care programmes with case management—of a new, internationally validated assessment instrument, the MDS-HC

The intervention group used more at home services, were sent to hospital later and less often following assessment using MDS-HC assessment, therefore, reducing costs. MDS-HC also indicated improvements in physical and cognitive function in the intervention group.

Ratner [57], 2001, USA, 0.55

The Kitchen Table Discussion

Formally structured social work visits at patients’ homes to discuss end-of-life issues, with communication of results to home health nurses and attending physicians.

X

Case series

Patients with a serious or life-threatening illness with a life expectancy of less than 2 years receiving home care. 75% aged 65 years and older.

To determine whether home health agency patients’ preferences to die at home can be honoured following a structured, professionally facilitated advance-care planning (ACP) process provided in the home.

Patients were willing to take part in ACP discussions at home. Most patients preferred to die at home. Facilitating ACP among such patients and their families was associated with end-of-life care at home. Use of hospice services was common following ACP in this population.

Schamp [58], 2006, USA, 0.68

Pathways Tool

A documentation tool that captures both present and advance directives in a framework of “pathways,” blending goals of care with typical procedure-oriented directives.

X

Pre and post observational study design

Elderly, frail and medically complex population with an average of 8 chronic medical conditions living in the community. More than 133, of the 160 patients, were over the age of 65 years.

To determine the effect of using the Pathways Tool upon the rates of completion of health care wishes and whether the distinction of “present” versus “advance” directives might be associated with differing qualitative choices expressed

The Pathways Tool was associated with increased completion of health care wishes, preferences toward less invasive levels of care at life’s end, and increased compliance with participants’ wishes and deaths at home.

Zafirau [59], 2012, USA, 0.64

Resident Change in Condition Assessment/Transfer Form

The form provides background information on patient’s health history and other information helpful and necessary for receiving hospitals. It also records the presence of advanced directives. If a DNR order exists, a copy is attached directly to the form.

X

Pre and Post test intervention evaluation

Patients in long term care facilities transferring to the emergency department, mean ages 72.8 and 76 years.

To test the efficacy of a standardized form used during transfers between long-term care facilities and the acute care setting

Communication between LTCFs of advanced directives was improved by use of the standardised transfer form. The form may also have increased admissions to the palliative care unit.

McGlinchey [60], 2019, UK, 0.8

Serious Illness Conversation Guide

Guide to support clinician’s communication with patients regarding current and future care and to promote shared decision making

X

X

Stage 1: Nominal Group Technique

Stage 2: Cognitive Interviews

Stage 3: Stakeholder review and consensus

Stage 1: Medical oncologists, palliative care and communication skills experts.

Stage 2: Lay representatives

Stage 3: Stakeholders made up of lay members and health service practitioners and researchers.

To explore the ‘face validity’, applicability and relevance of the clinical tool, the Serious Illness Conversation Guide, to explore whether adaptations were required for the UK before its use in the pilot.

Interviews indicate acceptance from practitioners with some considerations. Use of the guide has the potential to benefit patients, facilitating a ‘person-centred’ approach to these important conversations, and to provide a framework to promote shared decision making and care planning.

Mills [61] 2018, Australia, 0.55

Goals-of-Care form

A one-page document used to guide and record discussions between clinicians and patients around care preferences.

X

X

A prospective mixed methods study

108 forms were available from hospital inpatients. Median age 91 years. 16 doctors were interviewed.

To evaluate the utility to doctors of a form specifically designed to guide and document Goals of Care discussions at point of care. A secondary aim was to collect data on the length of GOC conversations and documentation.

Having a Goals-of-Care form in emergency medicine is supported. However, the ideal contents of the form were not determined.

Bouvette [62], 2002, Canada, 0.4

Pain and Symptom Assessment Record (PSAR)

Two-sided questionnaire.

X

X

Mixed methods

Palliative care patients in acute care institutions and community palliative care and oncology services, such as hospices and nursing agencies

To determine the feasibility of implementing the Pain and Symptom Assessment Record (PSAR) to assess the pain and symptoms of palliative care patients in a variety of settings

Based on the results from this study, the tool has been modified and is currently utilized in a variety of settings.