Development of the preliminary questionnaire
Developing the questionnaire item pool
An advisory committee of palliative care co-coordinators provided direction throughout the entire development process through a series of focus groups. Committee members were palliative medicine specialist experienced in palliative care with responsibilities for education and training. Their responsibilities entailed assessing the awareness and attitude of other medical specialties, identifying aspects of knowledge that were central to palliative care practice, determining the format to be used for the tool, specifying the appropriate level of difficulty for items, and generating the items [16, 18].
On the basis of the review of the tools available in the current literature and how they created by their developers either about attitude [13,14,15, 19, 20] or knowledge [13, 16, 17, 21] towards palliative care, it was decided to divide the questionnaire into two main sections; the first section was to assess the attitude of physicians and the second section was to assess self-knowledge (self-efficacy) and aspects of the basic knowledge that were central for palliative care practice by any non-palliative physician such as the principles of palliative care, symptoms assessment and management, and the use of painkillers.
Generation and pre-testing of items
Members of the advisory committee generated a pool of approximately 102 items covering all dimensions [16, 18]. For the attitude, 33 items were identified and measured through Likert scale [22,23,24]. It helped to quantify subjective preferential opinion, attitude, thinking and feeling in a scientifically accepted, validated and reliable manner [25,26,27]. We used Likert 5 points symmetrical scale. In many studies, five points scale is comprehensible, enabling respondents to accurately express their views [24, 27]. Participants were asked to show their level of agreement (from strongly disagree to strongly agree) with the given statement (items) on a metric scale. Here all the statements in combination reveal the specific dimension of the attitude towards the palliative care, hence, necessarily inter-linked with each other [26,27,28].
Attitude was defined as a system of beliefs and knowledge that everyone has got or has learned throughout his lifetime [29]. Health care providers’ attitude toward caring patients with life threatening disease may have an important influence on the quality of care they provide [20]. Dimension of self-knowledge included 7 items and measured by 5 points Likert scale ranging from excellent, very good, good, weak, to none. Self-efficacy was defined by Bandura, 1994 as ‘people beliefs about their abilities to make designated levels of performance that implement influence over events that affect their lives’ [30] while self-knowledge was defined as ‘competence to perform certain procedure’ [16].
Dimension of basic knowledge that where central to palliative care practice reflects the philosophy and principles of palliative care, the management and control of pain and other symptoms, and use of painkillers.
Sixty two items were identified with five response alternatives consisting of the correct response, three distracters and an ‘I don’t know’ alterative aiming to distinguish between lack of information and misinformation as well as to reduce guessing [31]. Items were generated from the literature with expert advice from experienced palliative medicine specialists based on their knowledge and real life experiences with palliative care patients. It was believed that this process served to maximize the content validity of the questionnaire to ensure that items selected were representative of the whole area of attitude and knowledge being measured.
Knowledge was defined as ‘knowing something with familiarity that acquired through experience such as understanding of a science or technique’. [32] While Ross et al. [16] defined palliative care knowledge as ‘understanding of death and dying, symptom management, medications and any intervention needed for those patients care’. We meant by basic knowledge minimal knowledge needed in physicians to deal effectively in different clinical situation commonly seen in those patients.
We omitted any items related to death or dying as our aim to shift the focus of palliative care from only caring of patients at the end of life to the delivery of highly specialized supportive care to any patients with life threatening-illness through the disease trajectory.
To ensure high face validity and the representation of a reasonably valid sample of items from the substantive areas of interest, items were then reviewed by all members of the advisory committee and other specialists in palliative care to select the best in terms of clarity and relevance of the questions, accuracy of the palliative knowledge being tapped, doubling or closeness of the items, and interpretability [16, 18]. This process reduced the number of items to 46 as a preliminary palliative care attitude and knowledge questionnaire with five response alternatives.
A number of demographic questions were added to the questionnaire to characterize the respondents. It includes 11 items about sex, age, nationality, educational level and qualification, specialty, place of work, position “job title”, years of experience, any formal palliative care training, and previous discussion with the patients or their families. Now, the preliminary instrument was then ready for piloting in a sample of physicians and included 57 items [16, 18].
Evaluation of the preliminary questionnaire “pilot study”
Five hundred questionnaires were distributed to the physicians working in primary care clinics and general hospitals all over Kuwait. The questionnaire was distributed manually for each clinic or hospital with the request that they complete and return them and add any comments that might occur to them. Of the 500 questionnaires, only 46.4% (N = 232) were returned and completed. The results of the pilot study were analyzed both quantitatively for item difficulty, item discrimination and internal consistency, and qualitatively which involves looking at comments made by respondents [16, 18]. This leaded to dropping of nearly one-third (n = 20) of the original items.
Evaluation of validity and reliability of the final questionnaire (PCAK) Additional files: 1 and 2
Based on the analysis described above, the number of items was reduced to 37. The next step was to test construct validity [33, 34] of the final version by administering it to two groups known to differ in their attitude and knowledge toward palliative care. The first group was oncologists working in Kuwait Cancer Control Center and consist of 47 and other group was physicians working in primary care clinics and general hospitals and consist of 82. This ensured that one group had a better attitude and knowledge (the oncologists) than the other group (other physicians), while other demographic characteristics were fairly similar for both groups.
PCAK was administered on two separate occasions, with an interval of 2 weeks between them. 2 weeks were expected to be long enough for participants to have forgotten their original responses, but not sufficiently long for much real change in their attitude and knowledge towards palliative care. Participants were not aware of the intended second administration at the time of the first [18]. The responses from the first administration were used to assess construct validity and internal consistency. The two sets of responses were used to measure test-retest reliability [33, 34].
Ethical consideration
The approval of the ethical committee of the Ministry of Health was taken prior to the study. Informed written consent was obtained from all participants. The aim of the study and expected outcomes were explained with guaranteeing of the privacy of the data.
Statistical analysis
All data manipulation and analysis were performed using the SPSS (Statistical Package for Social Science) SPSS version 20. P-values less than 0.05 were regarded as a sign of statistical significance. Categorical variables were represented as numbers and percent while continuous variables were represented as means and standard deviations or medians and interquartile ranges as appropriate. Chi-square test or Fisher’s Exact when appropriate was used to compare between qualitative variables. Independent t-test was used to compare the quantitative variables between two groups.
Regarding pilot study
The results were analyzed both quantitatively (for item difficulty, item discrimination and internal consistency) and qualitatively (which involved looking at comments made by respondents) [18]. For item difficulty, According to Kline (2000) [33] items are not useful if they are answered correctly by more than 80% or fewer than 20%. Pre-tests using that item run the risk of a ceiling effect in which performance on the pretest cannot be improved upon. For item discrimination, Pearson correlation was used to compare each item in attitude or knowledge with each subtotal score. An item-to-total-score correlation < 0.2 was rejected to discriminate between people with different levels of knowledge or attitude during testing of the questionnaire in the pilot study [33, 35]. Internal consistency using Cronbach’s alpha was measured separately for the different sections, each of which was tapping a different area; attitude, self-knowledge, and general knowledge. The minimum requirement for internal consistency has been accepted as 0.6 or more for research purpose [33]. Comments made by respondents were carefully revised and some changes were done upon.
Regarding the final questionnaire (PCAK)
The results of final survey were tested for construct validity by comparing two groups of different knowledge [33, 34] and attitude (oncologists and other physicians). Chi-square test or Fisher’s Exact when appropriate was used to compare between qualitative variables. Independent t-test was used to compare the quantitative variables between two groups.
Test-retest reliability was done to verify that the results produced were consistent over time. Paired t-test was used to compare the response of the same group before and after 2 weeks. More than 0.8 was considered accepted cut point for reliability and consistency over time. Dates of birth were used to match the two sets of questionnaires. Internal consistency was tested using Cronbach’s alpha as above. The responses from the first administration were used to assess construct validity and internal consistency. The two sets of responses were used to measure test-retest reliability [33–36].
Factor analysis was performed and repeated many times throughout the development of the questionnaire. Kaiser-Meyer-Olkin Measure of Sampling Adequacy more value > 0.5 is acceptable and indicating that pattern of correlation between items relatively compact and suitable for factor analysis. Bartlett’s Test of Sphericity was done. Extraction method used was principal axis factoring [37]. Variables were exclude if they have low Communalities (< 0.4) as it means that the variable didn’t contribute much to measuring the underlying factors.
Scoring of attitude responses (11 items): strongly disagree (1), disagree (2), not sure (3), agree (4), strongly agree (5). So, the score ranged from 11 to 55 points so the difference (44 points) was divided into three equal parts for scoring. Negative attitude if the participant scored = < 25, uncertain attitude if scored > 25 but < 41, positive or favorable attitude if scored > =41. Regarding self-knowledge, 5 points likert scale was used. Participant for each item was scored (5) for excellent response, (4) for very good, (3) for good, (2) for weak and (1) for none. Regarding basic knowledge scoring; each correct answer was scored one and wrong answered was scored zero. Poor knowledge was calculated if participant scored less than 50% of the total score (12 points) (= < 5 points), fair knowledge if > = 50% to = < 75% (6–9 points), good knowledge if scored > 75% (> = 10 points).
Although by factor analysis, the weight of each item in subscale analysis was not equal and some items were weighting more than others but from clinical point of view, we considered each item is important and relevant and has same weight like others in each subscale.
