Sage Journals: Discover world-class research

Abstract

Background

The need for improving knowledge and practice of palliative care delivered by health workers become an agenda in several countries. In order to measure the practice, an instrument is needed. The study analyzed the validity and reliability of the instrument to assess the physician’s practice in the management of patients with terminal diseases.

Methods

This was a cross-sectional study involving 89 physicians practicing in primary health care. The instrument of practice has been developed and resulted 5 domains consist of 20 items. An overview of reliability, construct validity, uni-dimensionality, and hierarchy of the person-items of the instrument were analyzed using Rasch Model.

Results

The reliability of the instrument is excellent with a person measure reliability of 0.85 and the item measure reliability of 0.96. Construct validity is confirmed with the MNSQ outfit values in the range of 0.54 to 1.59 and Pt Measure Corr. values in the range of 0.31 to 0.8. This instrument has a value of more than 20% unidimensionality which indicates the level of independence for items is good.

Conclusion

The instrument has good validity and reliability to assess physician’s practice in the management of patients with terminal disease.

Keywords

instrument palliative care practice validity

Background

Access to the health services at the end of life including access to palliative care is patient’s right.¹ Primary health care as a closest health facility in the community has a great opportunity to provide palliative care.² Health workers in primary health care have a significant role in providing palliative care, because most families of the patients prefer to take care for family members at home rather than in hospitals.^3,4 However, several studies show that some health workers in primary health care are not equipped with appropriate knowledge and skills for palliative care.^1,5-8 In some countries, palliative care has been not implemented yet due to limited number of trained health workers.¹ One study highlights that only physicians who were interested in the end of life care have good knowledge regarding the care.⁶

Health workers who demonstrate optimal practice in palliative care had a stronger effect to patient’s outcome.⁹ The quality of palliative care influences the patient’s quality of life.¹⁰

The need for knowledge and quality of good palliative care by health workers become an agenda in several countries, including Indonesia.^11,12 The knowledge and skills for good palliative care include pain and symptom management, psychological, and spiritual support,¹³ holistic communication,^4,14-16 decision making and timely referral.¹⁷ In order to assess knowledge and practice, an instrument is needed. To our knowledge, there is no instrument measures the practice of palliative care or care for patients with terminal diseases in Indonesia. The study was aimed to develop the instrument to assess the physician’s practice in the management of patients with terminal diseases and analyzed its validity and reliability using Rasch Model.

Rasch model is statistical model to examine the observed data against a standard model.¹⁸ The model measures the data at the response-level that indicate to what extent these ideals are realized within any particular data set.¹⁹ Rasch analysis measures person ability and item difficulty to provide information regarding performance of the instrument.¹⁸ Therefore, Rasch model analysis could evaluate the strengths and weakness of the instrument.²⁰

Methods

Design

The overall study was designed through 2 phases. Phase 1 was qualitative study using in-depth interview to explore the experience of health workers. Phase 2 was cross-sectional study to analyze instrument’s validity and reliability. This paper will focus to report the result of phase 2. The study was conducted in Bandung District between September and November 2018. The study has been approved by The Medical Research Ethics Commission Faculty of Medicine, Universitas Padjadjaran with registration number 1198/UN6.C10/PN/2017.

Participants

The in-depth interviews were conducted to health workers exploring a reflection of their experience and skills needed to care patients with terminal diseases. Primary care physicians, specialists, psychologist, and nurses were selected purposively. The reflection of participants was used to develop domain of practice for the instrument.

The participants for validity test were physicians practicing in primary health care, at least 1-year experience in practice and willing to participate. Eighty-nine physicians were selected from 5 selected districts. Sample size of 89 was considered adequate to be analyzed by Rasch Model.¹⁹ Sociodemographic characteristics of respondents is described in Table 1.

Table 1.

Characteristics of Participants.

Characteristics	Frequency (n)	Percentage (%)
Sex
Male	30	34
Female	59	66
Age (mean; SD)
39.6; 13.44
Education
Undergraduate	78	88
Postgraduate	11	12
Length of practice
<5 years	30	34
6-10 years	18	20
>10 years	41	46

The Instrument

The instrument was developed through a qualitative research. Practice is described by 5 domains and 20 questions.¹ The domains are case notification (4 questions), comprehensive and integrated care (10 questions), follow up (2 questions), and interdisciplinary collaboration (4 questions). Response categories for all items included yes, sometimes and never. Yes answer is defined as health personnel almost always performed an activity when dealing with terminal diseases. Sometimes answer is defined as health personnel almost never performed an activity when dealing with terminal diseases. Never answer refers to health personnel never performed an activity when dealing with terminal diseases.

Data Collection

Sampling frame were obtained from the district health office. The sampling method used was total sampling by selecting all doctors practicing in the selected area. The data was collected by trained enumerators. The enumerator made an appointment by telephone to meet and asked permission to distribute the questionnaire. After permission was granted, all participants were provided with written informed-consents, signed them and filled out a questionnaire. The participants completed the instrument in average of 10 to 15 min.

Data Analysis

Rasch analysis was performed to determine the validity and reliability using the Winstep software 3.73. Rasch analysis can produce good and accurate instruments to assess a person’s confidence in doing practice and determine person and items validity simultaneously.²⁰ The Rasch analysis provides an overview of reliability, construct validity, uni-dimensionality and hierarchy of the person-items.^20-22 The following fit criteria were used in the present analysis.

Reliability of the instruments is needed to assess the degree of consistency. A good measurement will show a high degree of reliability when producing a consistent score. Reliability is measured by (1) Cronbach’s alpha, to measure the interaction between person and items as a whole; (2) item reliability index and person reliability.^22-24 Item and person reliability are estimated with values more than 0.7.²¹

Item validity refers to items that are fit and misfit using values INFIT MNSQ (information-weighted mean square). Validity for each item refers to OUTFIT MNSQ (outlier-sensitive mean square), Z score standard (ZSTD) and Point Measure Correlation (Pt Mean Corr).^19,20 The values for OUFIT MNSQ between 0.5 and 1.5; OUTFIT ZSTD between −2.0 and 2.0; and Pt Mean Corr between 0.32 and 0.85 are acceptable.^19,25 Infit is used to indicate unexpected patterns of response for person ability and outfit indicates the unexpected rater responses.^19,26

The Person-Item Map describes the respondent’s ability to answer the question correctly. This map illustrates the ability of respondents to be categorized from lowest to highest and the quality of items to be categorized from easy items to difficult items visually.^19,20,26

One of the most important considerations in designing a measurement is to ensure unidimensionality based on PCA (Principal Component Analysis).^19,27 The questionnaire have to meet criteria for unidimensionality with minimum diversity requirements of 20%.¹⁹

Results

Reliability

Based on analysis, the reliability of the instrument is excellent with a person measure reliability of 0.85 (Table 2) and the item measure reliability of 0.96. The item separation index is 4.63 and the respondents’ separation index is 2.42. Cronbach’s alpha value is 0.91.

Table 2.

Summary of Person and Items Reliability and Index Separation.

	Person					Items
	Logit	Infit		Outfit		Logit	Infit		Outfit
	Logit	MNSQ	ZSTD	MNSQ	ZSTD	Logit	MNSQ	ZSTD	MNSQ	ZSTD
Mean	1.67	0.98	0.0	0.96	0.1	0.00	1.02	0.0	0.96	−0.1
Separation	2.42					4.63
Reliability	0.85					0.96

Validity

Based on the results of the Rasch analysis, MNSQ outfit values (0.5-1.5), and Pt Measure Corr (0.32-0.85) of the instrument are acceptable (Table 3). The item difficulty ranges from −3.87 to 2.25 logits. Item number 1 has a logits value of −3.87 which means this is a very easy question. This item also has Pt mean corr 0.19 and categorized as misfit item. Therefore, this item is removed from the final instrument.

Table 3.

Descriptive Statistics of Items.

No.	Items	Logit	Infit		Outfit		PT-measure
No.	Items	Logit	MNSQ	ZSTD	MNSQ	ZSTD	CORR.	EXP.
1	Identify physical problems	−3.87	0.99	0.2	0.54	−0.1	0.19	0.19
2	Identify psychological problems	−1.09	1	0.1	0.77	−0.4	0.48	0.49
3	Identify the psychological, spiritual, socio-economic need of patients and family members	0.30	1.43	2.5	1.42	1.7	0.5	0.61
4	Provide information and discuss the patient’s problems	−1.69	1.38	1.4	0.97	0.2	0.32	0.42
5	Provide palliative care in the early stage of disease	1.49	0.91	−0.6	1.04	0.3	0.64	0.67
6	Give pain drugs based on guideline and standard	−0.88	1.57	2.4	1.59	1.4	0.38	0.51
7	Discuss the patient’s problems with psychologist	1.77	1.11	0.8	1.06	0.4	0.68	0.68
8	Provide psychological support to patients and family members	−0.05	0.87	−0.7	1.40	1.5	0.54	0.58
9	Involving family members in case management	−1.02	0.89	−0.5	0.65	−0.8	0.56	0.49
10	Consider patient’s preference in management plan	−0.05	1.02	0.2	1.24	0.9	0.53	0.58
11	Assess financial source of patients and family	1.22	1.37	2.4	1.31	1.7	0.53	0.66
12	Discuss patient’s problems from spiritual aspect	2.25	0.72	−2.1	0.73	−1.6	0.73	0.69
13	Assess patient’s perception of the diseases, death and how to deal with them	0.47	0.57	−3.3	0.67	−1.7	0.73	0.62
14	Prepare process of dying and death with comfort	0.82	1.05	0.4	0.86	−0.7	0.71	0.64
15	Regular follow up to assess patient’s and family’s problems	0.39	0.78	−1.5	0.67	−1.7	0.72	0.62
16	Provide care at the clinic, home, and community setting	0.30	1.06	0.4	1.11	0.6	0.61	0.61
17	Collaborate with other professionals for case management	0.26	0.71	−2.0	0.59	−2.1	0.72	0.61
18	Consult or refer patients to specialist when needed	−1.32	1.07	0.4	0.84	−0.2	0.45	0.46
19	Be care coordinator when referring patient to other professionals	0.66	1.11	0.8	0.93	−0.3	0.64	0.63
20	Share responsibility in the case management	0.04	0.8	−1.2	0.73	−1.1	0.66	0.59

The Person-Items Maps

The left side of the person-items map describes the respondent’s ability and the right side is the quality of the item (Figure 1). There are 5 people who have a highest person logit. The 20 items of the instrument are presented from the easiest (item number 1, bottom) to most difficult (item number 12, top).

Figure 1.

Person-items map.

Uni-dimensionality

Based on Rasch analysis, each item is analyzed to identify whether item could measure the intended construct (uni-dimensionality). The measurement results 49% diversity of data and meet the criteria of unidimensionality (more than 20%).

Discussion

The purpose of palliative care for patients will be achieved if the physician provides appropriate care. To our knowledge, there are no instruments assessing practice for palliative care in Indonesia. This study aimed to develop and analyze the validity and reliability of instruments to assess physician’s practice in palliative care using the Rasch analysis. Items in this instrument have been selected and correspond to the construct model of the Rasch.

Rasch analysis can predict source of measurements’ error from respondents and items. The more information about the person ability and items difficulty will suggest the less the measurement error.¹⁸ Overall, all items (20 questions) show good person and items reliability. The value of the person reliability test is 0.85, indicates that the instrument has strong reliability to distinguish practice. The value of the item’s reliability item is 0.96, indicates the items in this instrument could be distinguished based on the level of difficulty. An alpha Cronbach indicates that data are in accordance with the Rasch analysis requirements and the instrument can be used in different conditions. Cronbach’s alpha value of 0.91 indicates that the interaction between respondents and items shows a good reliability. The high separation value indicates that the instrument is sensitive enough to distinguish participant with good and poor practice.¹⁹

Based on Rasch analysis, validity can be assessed from the person validity and items validity. The person validity can be seen from the average value of logits as shown in Table 2. The logits of +1.67 (a value of more than 1.0) indicates that the respondents have a tendency to approve the item in the instrument. The infit mean-square value of 0.98 (Table 2) indicates the observed data has 2% less variation than predicted by the Rasch model. Index separation of respondents of 2.21 means that respondents can be divided into 2 groups based on the level of practice. The greater the separation index, the better the quality of the instrument, because it can identify the group of person and items.²¹

The items validity is seen from the average logits value of 0.0 (Table 2). It shows the instrument can assess practice. Items separation of 4.83 means that the item can be used with 4 categories or scale of response. Items fit can be seen from the MNSQ value of each item (Table 3). The items in this study have MNSQ outfit value in the range of 0.54 to 1.59. In addition, the accuracy of items is evaluated from the polarity analysis of the items. It is assessed from Pt Measure Corr value. Most items in this instrument have values in the range of 0.31 to 0.8. Based on the result this instrument is appropriate for assessing practice.

The person-items map provide information regarding how well the items are distributed with regard to the ability of respondents.^20,23 Figure 1 indicates the items match to the practice of physicians. The analysis shows that the mean of items measure is close to the mean of person measure. This arrangement indicates good test-item targeting.

Unidimensionality is used to determine the extent to which the diversity of instrument measures what should be measured.²¹ Unidimensionality show that the scale of the instrument measure only one construct. This instrument has a value of more than 20% unidimensionality which indicates the level of independence for items is good.

Conclusion

Overall, analysis with Rasch model revealed that the instrument has good validity and reliability. Therefore, this instrument could be used to assess physician’s practice in the management of patients with terminal diseases.

Footnotes

Acknowledgements

The authors would like to thank to Rector of Universitas Padjadjaran for supporting this study. We also thank to Sekar Ananggadipa Putri and Ginulur Adgani for valuable contribution in data collection.

Authors’ Contributions

NA, EPS, VP, and DH designed the concept of this study and contribute in study preparation. NA conducted data collection and drafted the manuscript. All authors performed data analysis, reviewed the draft and approved the final manuscript.

Availability of Data and Materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The authors would like to thank to Rector of Universitas Padjadjaran to financially support the research.

Ethical Approval

The Medical Research Ethics Commission Faculty of Medicine, Universitas Padjadjaran approved the study with registration number 1198/UN6.C10/PN/2017.

ORCID iD

Nita Arisanti

References

Arisanti

Sasongko

Pandia

Hilmanto

. Implementation of palliative care for patients with terminal diseases from the viewpoint of healthcare personnel. BMC Res Notes. 2019;12:1-5.

Pavlič

Aarendonk

Wens

Simões

Lynch

Murray

. Palliative care in primary care: European forum for primary care position paper. Prim Health Care Res Dev. 2019;20:1-6.

Herce

Elmore

Kalanga

, et al. Assessing and responding to palliative care needs in rural sub-saharan Africa: results from a model intervention and situation analysis in Malawi. Eldabe S, ed. PLoS One. 2014;9:1-17.

Tan

Radzniwan

Khairani

Ednin

. Palliative care for a child: role of a primary care doctor. Malays Fam Physician. 2011;6:26-28.

Rochmawati

Wiechula

Cameron

. Current status of palliative care services in Indonesia: a literature review. Int Nurs Rev. 2016;63:180-190.

Spice

Lau

Perez

Turley

Turin

. Hospice care in Calgary: survey of family physicians on their knowledge, experience, and attitudes. Can Fam Physician. 2016;62:484-494.

Budkaew

Chumworathayi

. Knowledge and attitudes toward palliative terminal cancer care among Thai generalists. Asian Pac J Cancer Prev. 2013;14:6173-6180.

Cheng

. Chinese oncologists’ knowledge, attitudes and practice towards palliative care and end of life issues. BMC Med Educ. 2016;16:149.

Thompson

Bott

Boyle

Gajewski

Tilden

. A measure of palliative care in nursing homes. J Pain Symptom Manag. 2011;41:57-67.

10.

Yun

Kang

Lee

, et al. Development and validation of the quality care questionnaire—palliative care (QCQ-PC): patient-reported assessment of quality of palliative care. BMC Palliat Care. 2018;17:40.

11.

Sandsdalen

Rystedt

Grøndahl

Hov

Høye

Wilde-Larsson

. Patients’ perceptions of palliative care: adaptation of the quality from the patient’s perspective instrument for use in palliative care, and description of patients’ perceptions of care received. BMC Palliat Care. 2015;14:54.

12.

Arisanti

Hilmanto

Setiawati

Pandia

. The need for palliative care in primary health care. Rev Prim Care Pract Educ. 2018;1:103-104.

13.

Doyle

Woodruff

. International Association for Hospice & Palliative Care Promoting Hospice & Palliative Care Worldwide The IAHPC Manual of Palliative Care. 2nd ed. 2008.

14.

Richardson

. Spirituality, religion and palliative care. Ann Palliat Med. 2014;3:150-159.

15.

Lawless

Freeman

Bentley

Baum

Jolley

. Developing a good practice model to evaluate the effectiveness of comprehensive primary health care in local communities. BMC Fam Pract. 2014;15:99.

16.

Harris

Herrel

Healy

Wancata

Perumalswami

. Milestones for the final mile : interspecialty distinctions in primary palliative care skills training. J Pain Symptom Manag. 2018;52:345.e5-352.e5.

17.

Luckett

Phillips

Agar

Virdun

Green

Davidson

. Elements of effective palliative care models: a rapid review. BMC Health Serv Res. 2014;14:136.

18.

Daher

Ahmad

Than

Selamat

. Impact of rating scale categories on reliability and fit statistics of the Malay spiritual well-being scale using rasch analysis. Malays J Med Sci. 2015;22:48-55.

19.

Linacre

. A user’s guide to WINSTEP ministep; rasch-model computer programs. Program Manual 3.73. 2011.

20.

Boone

. Rasch analysis for instrument development: why, when, and how? CBE—Life Sci Educ. 2016;15:rm4.

21.

Farzad

Layeghi

Hosseini

Whiteneck

Asgari

. Using the rasch model to develop a measure of participation capturing the full range of participation characteristics for the patients with hand injuries. J Hand Microsurg. 2017;9:84-91.

22.

Nielsen

Kyvsgaard

Sildorf

Kreiner

Svensson

. Item analysis using rasch models confirms that the Danish versions of the DISABKIDS® chronic-generic and diabetes-specific modules are valid and reliable. Health Qual Life Outcomes. 2017;15:1-10.

23.

Chan

Ismail

Sumintono

. A rasch model analysis on secondary students’ statistical reasoning ability in descriptive statistics. Procedia Soc Behav Sci. 2014;129:133-139.

24.

Sandham

Medvedev

Hedgecock

Higginson

Siegert

. A rasch analysis of the integrated palliative care outcome scale (IPOS). J Pain Symptom Manag. 2019;57:291-296.

25.

Brown

Obasi

Barrett

. Rasch analysis of the WURSS-21 dimensional validation and assessment of invariance. J Lung Pulm Respir Res. 2016;33:557-573.

26.

Classen

Medhizadah

Romero

Lee

. Construction and validation of the 21 item fitness-to-drive screening measure short-form. Front Public Health. 2018;6:339.

27.

Trakman

Forsyth

Hoye

Belski

. The nutrition for sport knowledge questionnaire (NSKQ): development and validation using classical test theory and Rasch analysis. J Int Soc Sports Nutr. 2017;14:26.

The Development of Instrument to Assess Physician’s Practice in the Management of Patients With Terminal Diseases

Abstract

Background

Methods

Results

Conclusion

Keywords

Background

Methods

Design

Participants

The Instrument

Data Collection

Data Analysis

Results

Reliability

Validity

The Person-Items Maps

Uni-dimensionality

Discussion

Conclusion

Footnotes

Acknowledgements

Authors’ Contributions

Availability of Data and Materials

Declaration of Conflicting Interests

Funding

Ethical Approval

ORCID iD

References