Sage Journals: Discover world-class research

Abstract

Objectives

The Assessment of Positive Occupation 15 (APO-15) is a scale initially developed to evaluate participation in well-being-promoting occupations among individuals with mental illness. This study examines the psychometric properties of the Japanese version of the APO-15 for individuals with physical illness, focusing on its validity, reliability, and interpretability.

Methods

We evaluated the psychometric properties of the APO-15 using a sample of 1,259 individuals with physical illness living at home, in hospitals, and in nursing homes. Validity was evaluated using confirmatory factor analysis (CFA) to confirm the factor structure, and concurrent validity was examined by correlating APO-15 scores with measures of happiness and psychological distress. Reliability was assessed by internal consistency analysis using Cronbach's alpha, and item difficulty was analyzed using the Rasch Rating Scale Model (RSM). Interpretability was established through a T-scale-based five-level criterion for client status.

Results

The APO-15 showed strong psychometric properties. CFA indicated a good fit for the factor structure (RMSEA= 0.100, CFI= 0.944, TLI= 0.930). Internal consistency was high, with a Cronbach’s alpha of 0.914. Item difficulty analysis revealed a bias towards easier items, consistent with previous findings in mental illness populations. The T-scale-based five-level criterion effectively classified client status, though significance was noted between most levels except levels 1 and 2.

Conclusion

The Japanese version of the APO-15 is a reliable and valid tool for assessing occupational participation and promoting well-being among individuals with physical illness. It provides a robust framework for occupational therapists to support clients' well-being through targeted occupational engagement.

Keywords

Well-being measurement occupational engagement participation mental health

Introduction

In recent years, individual well-being has received increasing attention and support globally (Diener & Chan, 2011). One contributing factor is the aging of the global population, with mental health problems expected to increase among older adults (World Health Organization, 2016). In addition, symptoms of depression are 2–10 times more common among individuals with chronic illness or disability, with such conditions serving as major risk factors for depression (Centers for Disease Control and Prevention, 2020; World Health Organization, 2021). Previous research has shown that promoting well-being can effectively prevent disease and increase life expectancy (David et al., 2013; Diener & Chan, 2011). Therefore, promoting clients' well-being is an important component and is considered one of the outcomes to be targeted in occupational therapy (American Occupational Therapy Association, 2020).

Occupational therapy is a client-centered health profession that promotes health and well-being through occupation. The primary goal of occupational therapy is to enable clients to participate in occupations that promote well-being (World Federation of Occupational Therapists, 2013). Occupations are the center of human experience and activities that make up daily life, such as work, leisure, play, self-care, and social interaction, and are intended uses of time, energy, interest, and attention (Wilcock, 2015). Well-being is defined as the perceived state of harmony in all aspects of life (Law et al., 1998). Wilcock (2015) also states that well-being is a multidimensional phenomenon that integrates biological, psychological, social, and spiritual aspects, and is an emotional and cognitive aspect of subjective experience. Occupational well-being, defined as a state of perceived satisfaction and joy from daily experiences, represents beliefs about the relationship between occupation and well-being (Charles & Townsend, 2013). This study refers to occupations that promote human health and well-being as positive occupations and defines them as human experiences that constitute a state of physical, mental, and social well-being (Noguchi & Kyougoku, 2019).

Research on occupations that promote client well-being has been explored not only in occupational therapy but also in various disciplines, including positive psychology. Robinson et al. (2012) focus on the potential for occupational therapy to contribute more to well-being through occupations that utilize the findings of positive psychology from an occupational science perspective. Specifically, they suggested the possibility of applying positive psychology findings to occupational therapy, such as (1) gratitude, (2) flow, (3) positive relationships, (4) meaning in life, and (5) achievement. Recent developments in the quantitative assessment of occupational therapy have resulted in a number of tools to capture an individual’s occupational experience. For example, the Engagement in Meaningful Activities Survey (Eakman, 2011), and the Meaningful Activity Participation Assessment (Eakman et al., 2010). However, these scales focus on occupational experiences that are meaningful to the individual and not on those that have been scientifically proven to enhance human well-being.

Therefore, we developed a measurement tool called the Japanese version of the Assessment of Positive Occupation 15 (APO-15) (Noguchi et al., 2016; Noguchi & Kyougoku, 2021). The APO-15 was developed with reference to the PERMA model (Seligman et al., 2005) to assess the state of participation in occupations that promote well-being. It consists of 15 items grouped into four factors: Positive relationships, Achievement, Meaning, and Engagement. “Positive relationship” is defined as rich human relationships that provide happiness and satisfaction. “Meaning” is defined as the ability to find significance in particular activities and life. “Achievement” is defined as an attempt to achieve a goal in life. “Engagement” is defined as the experience of flow and the process leading to that state. Explaining these aspects in relation to occupation, for example, being able to engage in positive relationships not only reduces the likelihood of experiencing isolation and loneliness but may also contribute to fulfilling a sense of companionship and affirming self-worth (Doble & Santha, 2008). Moreover, engaging in positive occupations that promote well-being, as exemplified earlier, has been proposed to alleviate psychological distress, including depression and anxiety (Bolier et al., 2013). However, it has been suggested that individuals facing physical, mental, or social limitations are less likely to participate in such occupations (Meulenkamp et al., 2019).

The APO-15 is unique in that it consists entirely of occupations known to have the potential to enhance human well-being, as supported by positive psychology and occupational science research. In addition, the primary goal of the APO-15 is to identify individuals who experience limitations in their ability to participate in these occupations and link them to occupational therapy practice. This unique feature positions the APO-15 as a valuable tool in occupational therapy practice focused on promoting well-being (Noguchi & Kyougoku, 2021). The psychometric properties of the APO-15 have been examined in individuals with mental illness using statistical evidence provided by data analyses such as exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and item response theory (IRT). Overall, the psychometric properties of the Japanese and Turkish versions of the APO-15 were very good, and this instrument demonstrated a high degree of sensitivity in identifying individuals with mental illness who experience limitations in participating in occupations that promote well-being (Noguchi et al., 2016; Noguchi & Kyougoku, 2021; Özkan et al., 2024). Occupational therapy contributes to mental health, including the promotion of well-being in clients with physical illness (Reitz et al., 2020). Therefore, the ability to apply the APO-15 to individuals with physical illness may allow occupational therapy to further contribute to the promotion of clients' well-being. Based on the above, this study aimed to determine the psychometric properties of the Japanese version of the APO-15 in individuals with physical illness.

Methods

Participants

This study collected data from individuals with physician-diagnosed physical illnesses living at home, in hospitals, or in nursing homes. The selection criteria were (1) individuals with a prescription for occupational therapy between the age of 20-100 years old, and (2) individuals capable of independently or with minimal caregiver assistance understanding and completing the questionnaire. Exclusion criteria were (1) individuals with unstable medical conditions who were deemed by the physician or facility director to be unable to cooperate and (2) individuals who were unable to understand and complete the questionnaire alone or with minimal assistance from a caregiver. Recruitment involved displaying posters within the facilities, requesting voluntary participation, and screening individuals of interest by occupational therapists against the aforementioned selection criteria. Eligible participants received a full explanation of the study and an information sheet and were interviewed before giving their consent. Data collection was conducted face-to-face by an occupational therapist with an understanding of the APO-15. The study was approved by the ethics committee, conducted from May 2016 to March 2022, and adhered to the tenets of the Declaration of Helsinki.

Measures

Demographic information

Demographic data including sex, age, diagnosis, and nursing care level (needing care: 1-5, needing support: 1-2, or none) were collected.

APO-15

The APO-15 is a self-administered instrument that assesses the extent to which individuals with mental illness can participate in occupations that promote well-being. It comprises 15 items, rated on a 4-point Likert scale (1 = disagree to 4 = agree) for the past 2–3 weeks. The total APO-15 score ranges from 15 to 60 points, and a higher score is associated with greater subjective well-being (Noguchi, 2021; Noguchi et al., 2016; Noguchi & Kyougoku, 2021). The APO-15 is a paper-based tool, taking 5–10 minutes to complete. The APO-15 has good reliability, with a Cronbach’s alpha coefficient of 0.89. Please see the Appendix for the APO-15 (Supplemental files).

Happiness Scale

We measured happiness, an important factor in measuring the aspects of human well-being. Happiness reflects how satisfied or happy people feel about their lives and is closely related to psychological health and life satisfaction (Diener & Chan, 2011; Helliwell et al., 2021; Lyubomirsky et al., 2005). To minimize client burden, we employed a single-item happiness scale, validated for reliability and validity, using a five-point Likert scale (1 = not at all happy to 5 = very happy) referring to prior research (Abdel-Khalek, 2006). The scale assessed happiness in recent weeks.

The Japanese version of the Kessler Psychological Distress Scale (K6)

We used the 6-item Kessler Psychological Distress Scale (K6) to assess nonspecific psychological distress and its connection to mental health. This scale has strong psychometric properties and can discriminate psychiatric cases from non-cases. The K6 is a single-factor model scored on a 5-point Likert scale (0 = not at all present to 4 = always present), with total scores ranging from zero to 24 (Cornelius et al., 2013). We used the official Japanese translation of K6, and the cutoff point was based on 5 points. The K6 has good reliability, with a Cronbach’s alpha coefficient of 0.85.

Statistical analysis

Materials

This study adhered to the standards established by the COnsensus-based Standards for the selection of health Measurement INstruments (ColoradoSMIN) (Mokkink et al., 2019). COSMIN proposes a comprehensive framework for examining scale properties, including reliability, validity, responsiveness, and interpretability. In this study, we assessed the applicability of the APO-15 to clients with physical illness by evaluating its reliability, validity, and interpretability based on the COSMIN guidelines. The statistical software used in this study was HAD (https://norimune.net/had), Microsoft Excel 16.64 (https://www.microsoft.com/ja-jp/microsoft-365/excel), Exametrika (https://shojima.starfree.jp/exmk/), jamovi (https://www.jamovi.org/) module snowIRT 4.8.8, and R 4.2.1 (https://www.r-project.org) package lavaan 0.6–18.

Sample characteristics

Demographic data were summarized using descriptive statistics, including the calculation of the mean and standard deviation for participant age. Frequencies and percentages were calculated for participant sex, living place (home, hospital, or nursing home), diagnosis, and care level. In most cases, the elderly are defined as individuals aged ≥65 years; therefore, individuals aged ≥65 years were considered elderly in this study (Centers for Disease Control and Prevention, 2022). To determine whether there were statistically significant differences in the demographic data between the group aged ≤64 years and the group aged ≥65 years, an independent t test was performed for the continuous variable (age), and chi-squared tests were used for the categorical variables (sex, living place, diagnosis, care level, and happiness).

Item validity

Item validity was assessed using polyserial correlation coefficients. These coefficients measured the correlation between scale scores and total scores. Items were considered valid if the polyserial correlation coefficient was 0.5 or higher (Olsson et al., 1982). We also conducted normality tests using the Shapiro-Wilk test for the APO-15 total score. A non-significant result (p > .05) indicated a normally distributed total APO-15 score.

Structural validity

We tested the fit of the factor structure identified by Noguchi and Kyougoku (2021) to the present sample using CFA. The APO-15 factor structure is based on a first-order model called “positive occupational participation,” consisting of four subfactors: positive relationship (items: 4, 6, 10, 11, 14), achievement (items: 1, 2, 3, 5), engagement (items: 7, 12, 13), and meaning (items: 8, 9, 15). The factor structure of the APO-15 was determined by CFA using a weighted least squares mean and variance adjusted estimation (WLSMV) with missing data. WLSMV is appropriate for the analysis of categorical data. Five indicators were examined to assess the model fit of the APO-15. The first indices were the root mean square error of approximation (RMSEA) and the standardized root mean square residual (SRMR). Critical values for RMSEA indicating a good fit were <0.06, with values between 0.08–0.10 indicating a mediocre fit, and values >0.10 indicating a poor fit (Hu & Bentler, 1999). For SRMR, a value ≤0.08 indicated a good fit. The third and fourth indicators were the comparative fit index (CFI) and the Tucker-Lewis index (TLI), with critical values of CFI and TLI >0.90 indicating a good fit and >0.95 indicating a better fit (Kline, 2023). The fifth indicator is the chi-squared test of model fit, which considers the chi-squared value and the degrees of freedom of the model; a non-significant result indicates a good fit (p > .05).

Concurrent validity

Previous studies have shown that APO-15 is positively correlated with happiness (Noguchi & Kyougoku, 2021). Happiness has also been found to be negatively correlated with psychological distress (Massé et al., 1998). Thus, in this study, we assumed moderate correlations among these two variables. Concurrent validity was assessed using Spearman’s rank correlation coefficient to measure associations between each APO-15 item and the total score, Happiness Scale, and K6. Correlation coefficients were categorized as follows: r < 0.3 (none or very weak), 0.3 < r < 0.5 (weak), 0.5 < r < 0.7 (moderate), and 0.7 < r < 0.9 (strong).

Internal consistency reliability

Internal consistency reliability was assessed using Cronbach’s alpha coefficient. Cronbach’s alpha coefficient can be used to assess reliability based on the factor structure assumptions of the scale. A critical value of >0.7 indicates good internal consistency (DeVellis, 2016). In addition, to examine whether any items within the APO-15 were redundant, Cronbach’s alpha was also calculated when items were deleted. This analysis helps to identify items that may not uniquely contribute to the internal consistency of the scale and could be considered for removal (DeVellis, 2016).

Item difficulty

We assessed item difficulty using marginal maximum likelihood estimation (MMLE) within the framework of the Rasch Rating Scale Model (RSM) (Yamashita, 2022). RSM is appropriate for polytomous data from the Likert response format used in this study (Andrich, 1978). The advantage of using the Rasch approach is the ability to establish a hierarchy of item difficulty within the unidimensional structure of the APO-15. In this study, RSM estimated item difficulty, information-weighted fit (Infit), and outlier-dependent fit (Outfit) for each APO-15 item. Infit means a response pattern-sensitive fit to an item for an individual and vice versa. Outfit means a fit that is sensitive to outliers. This is more sensitive to responses to items with difficulty far from an individual and vice versa (Linacre, 2002). These criteria suggest that each item is informative for the measurement if its value is between 0.5 and 1.5. In addition, the assumption of one-dimensionality to support the application of the Rasch model was confirmed in this study by the person reliability, the mean of absolute values of the centralized Q3 statistic (MADaQ3), and the CFA. Person reliability is considered a Rasch-like indicator of Cronbach’s alpha, with >0.80 being considered acceptable (Fox & Jones, 1998). MADaQ3 is one of the indicators used to assess statistical model fit (Yen, 1984). The MADaQ3 criterion is < 0.1, indicating that the model fits the data well. The CFA, assessing one-dimensionality, used WLSMV with missing data. Four indices (RMSEA, SRMR, CFI, TLI) assessed CFA fit, following the same criteria as in the structural validity approach. To better understand APO-15, we created a weight map linking item difficulty to participant ability on a measurement plot.

Interpretability

The APO-15 evaluation criteria were designed for APO-15 total score interpretability. This evaluation criterion considered data non-normality and introduced a normalized T-scale (mean 50, standard deviation 10) (de Beurs, Boehnke, & Fried, 2022). The evaluation criteria process included these steps: 1) Calculate percentile ranks and T-scores from APO-15 total scores. 2) Compare the results to the T-scale and classify the state of participation in occupations that promote well-being (levels 4–6). 3) Using the Kruskal–Wallis test, a nonparametric method, to assess the appropriateness and characteristics of the number of levels. This analysis used APO-15 total scores and the Happiness Scale as the dependent variable and the number of levels of the APO-15 as the independent variable. In cases where significant differences between levels were observed, rank differences were examined using Holm’s method of multiple comparisons. 4) Each level was examined for the name that best describes the state of participation in an occupation that promotes well-being.

Results

Sample characteristics

This study included 1,259 participants, of whom 154 (12.23%) were aged ≤64 years and 1,105 (87.77%) were aged ≥65 years. The mean ages of the participants were as follows: overall (77.40 ± 11.25), ≤64 (55.13 ± 9.98), and ≥65 (80.18 ± 7.77). The majority of participants' diagnoses were orthopedic and cranial nerve-related. Statistical analysis revealed significant differences (p < .01) between the two age groups (≤64 years or ≥65 years) across all demographic data, including age, sex, living place, diagnosis, care level, and happiness. Other participant characteristics are shown in Table 1.

Table 1.

Characteristics of the Participants (n = 1,259).

Characteristics		Mean (SD)	Age ≤64 (%)	Age ≥65 (%)	Total, n (%)	p-value
Participant			154 (12.23)	1,105 (87.77)	1,259 (100)
Age	Overall	77.40 (11.25)				p < .01
	≤64	55.13 (9.98)
	≥65	80.18 (7.77)
Sex	Male		95 (61.69)	475 (42.99)	570 (45.27)	p < .01
	Female		59 (38.31)	630 (57.01)	689 (54.73)
Living place	Home		115 (74.68)	1001 (90.59)	1116 (88.64)	p < .01
	Hospital		38 (24.68)	80 (7.24)	118 (9.37)
	Nursing home		1 (.64)	24 (2.17)	25 (1.99)
Diagnosis	Orthopedics		44 (28.57)	426 (38.55)	470 (37.33)	p < .01
	Cranial nerve		75 (48.70)	319 (28.87)	394 (31.29)
	Internal		10 (6.49)	178 (16.11)	188 (14.93)
	Neurosurgery		21 (13.64)	110 (9.95)	131 (10.41)
	Psychiatry		1 (.65)	56 (5.07)	57 (4.53)
	Rheumatology		3 (1.95)	16 (1.45)	19 (1.51)
Care level	Care 5		1 (.65)	13 (1.18)	14 (1.11)	p < .01
	Care 4		15 (9.74)	40 (3.62)	55 (4.37)
	Care 3		11 (7.14)	88 (7.96)	99 (7.86)
	Care 2		34 (22.08)	268 (24.25)	302 (23.99)
	Care 1		14 (9.09)	257 (23.26)	271 (21.53)
	Support 2		17 (11.04)	184 (16.66)	201 (15.96)
	Support 1		20 (12.99)	206 (18.64)	226 (17.95)
	None		42 (27.27)	49 (4.43)	91 (7.23)
Happiness	Very good		13 (8.44)	163 (14.75)	176 (13.98)	p < .01
	Good		39 (25.33)	439 (39.73)	478 (37.96)
	Average		54 (35.06)	335 (30.32)	389 (30.90)
	Fair		34 (22.08)	128 (11.58)	162 (12.87)
	Poor		14 (9.09)	40 (3.62)	54 (4.29)

Note. The general physical and mental status of the care level is as follows: Support 1 = Able to perform basic activities of daily living (ADLs) independently, but requires supervision or assistance with some activities. Support 2 = Able to perform basic ADLs independently but requires supervision and assistance with more complex activities. Care 1 = Requires some assistance with ADLs, such as getting up and walking. Slight cognitive decline may be observed. Care 2 = Requires more assistance with ADLs than Care 1, and cognitive function is declining. Care 3 = Requires full assistance with ADLs and may use a cane, walker, or wheelchair. Cognitive function is declining and supervision is required. Care 4 = Requires assistance with all aspects of daily living beyond Care 3, including more intensive assistance. Care 5 = Requires full assistance with all aspects of daily living and has severe cognitive or physical impairments.

Statistical differences were found between the two groups of age ≤64 years and age ≥65 years for all demographic data (age, sex, living place, diagnosis, care level, and happiness) (p < .01).

Item validity

The APO-15 had polyserial correlation coefficients that met the criterion of 0.553–0.792 for all items. In other words, the scale scores and total scores were related and validated for all items of the APO-15. However, the normality test did not confirm normality for the total APO-15 score (p < .05).

Structural validity

The CFA of APO-15 estimated a model fit with mixed results (RMSEA = 0.100; SRMR = 0.049; χ2/df = 0.000; CFI= 0.944; TLI = 0.930) (Table 2). While the RMSEA was at the upper end of acceptable fit, the SRMR, CFI, and TLI values supported the four-factor structure of the APO-15. Although the χ2 test indicated model misfit, this test is known to be sensitive to sample size. Overall, the results provide partial support for the proposed factor structure of the APO-15.

Table 2.

Structural Validity and internal Consistency Reliability.

APO-15 items (α = 0.914)	Estimate	Std.Err	z-value	Two-tailed	Cronbach’s alpha if deleted
APO-15 items (α = 0.914)	Estimate	Std.Err	z-value	p-value	Cronbach’s alpha if deleted
Latent variables
Factor 1: Positive relationship (α = 0.911)
Item 4	.743			<.001	.908
Item 6	.574	.035	21.881	<.001	.913
Item 10	.739	.027	36.820	<.001	.909
Item 11	.816	.027	41.372	<.001	.907
Item 14	.770	.027	38.834	<.001	.908
Factor 2: Achievement (α = 0.911)
Item 1	.827			<.001	.906
Item 2	.772	.020	46.418	<.001	.909
Item 3	.829	.020	49.997	<.001	.906
Item 5	.832	.020	50.865	<.001	.905
Factor 3: Meaning (α = 0.911)
Item 8	.802			<.001	.907
Item 9	.832	.021	48.475	<.001	.906
Item 15	.792	.024	41.560	<.001	.907
Factor 4: Engagement (α = 0.911)
Item 7	.789			<.001	.908
Item 12	.873	.026	43.024	<.001	.906
Item 13	.763	.027	36.419	<.001	.910
Factor correlation
Factor 2
Factor 1	.753	.018	25.390	<.001
Factor 3
Factor 1	.862	.019	26.972	<.001
Factor 2	.776	.019	24.477	<.001
Factor 4
Factor 1	.728	.019	22.724	<.001
Factor 2	.755	.018	26.828	<.001
Factor 3	.737	.020	23.894	<.001
Model fit information
RMSEA	.100 [90% CI = .094-.105]
SMRM	.049
χ²/df	.000 [χ² = 18731.944, df = 105]
CFI	.944
TLI	.930

Note. α = cronbach’s alpha coefficient, Estimate = standardized estimates, Std.Err = standard error, CI = confidence interval, χ² = chi-Square, df = degrees of freedom.

The change in Cronbach’s alpha with the removal of each item ranged from 0.905–0.913, and the overall Cronbach’s alpha (0.914) was not exceeded with the removal of any item.

Concurrent validity

Concurrent validity was confirmed by correlations between the APO-15, the Happiness Scale, and the K6. Each factor scores and the total score of the APO-15 showed a moderate positive correlation with participants' happiness (r = 0.362 to 0.501, p < .01). APO-15 also showed a very weak negative correlation with the single factor score of K6 (r = −0.233 to −0.125, p < .01). In other words, we found a correlation with happiness, as hypothesized, but not with K6.

Internal consistency reliability

The overall Cronbach’s alpha for the APO-15 was 0.914, and all of four subfactors showed high internal consistency at 0.911 (Table 2). This indicates a high degree of consistency in participants' responses across all items on the APO-15. Furthermore, when each item was removed individually, the overall Cronbach’s alpha remained in a similar range (0.905–0.913), suggesting that removing each item does not improve the reliability of the scale. This supports the robustness in the item structure of the scale and the lack of redundancy between items in the overall high Cronbach’s alpha of the APO-15.

Item difficulty

The assumption of one-dimensionality supporting the application of the Rasch model was sufficient for all indicators (Person Reliability = 0.884, MADaQ3 = 0.092, CFA [RMSEA = 0.053; SRMR = 0.063; CFI= 0.980; TLI = 0.977]) (Table 3). The RSM results showed that the item difficulty of the APO-15 items ranged from −4.431 to −2.776, indicating an overall bias in the negative direction. Specifically, item 6 was identified as the easiest item (difficulty = −4.431), while items 1 (difficulty = −2.776) and 2 (difficulty = −2.872) were among the more difficult items, suggesting that these items were positive occupations that which participants found it difficult to participate in.

Table 3.

Item statistics of the rating scale model.

APO	Item difficulty	S.E.	Infit	Outfit
Item 1	−2.776	.042	.846	.855
Item 2	−2.872	.042	1.174	1.149
Item 3	−3.089	.043	.997	.964
Item 4	−2.990	.043	.973	.978
Item 5	−2.957	.043	.813	.806
Item 6	−4.431	.052	1.307	1.423
Item 7	−3.585	.046	1.130	1.126
Item 8	−3.897	.048	.985	.916
Item 9	−3.406	.045	.825	.807
Item 10	−3.556	.046	1.116	1.096
Item 11	−3.689	.046	.945	.908
Item 12	−3.005	.043	.960	.924
Item 13	−2.940	.043	1.319	1.317
Item 14	−2.887	.042	.893	.921
Item 15	−3.127	.043	.868	.878
Model fit information
Person reliability		.884
MADaQ3		.092
CFA
RMSEA		.053 [90% CI = .049-.060]
SRMR		.063
CFI		.980
TLI		.977

Note. S.E. = standard error, Infit = information-weighted fit, Outfit = outlier-sensitive fit, MADaQ3 = Mean of absolute values of centered Q3.

The Infit values ranged from 0.813 to 1.319, and the Outfit values ranged from 0.806 to 1.423. The Infit values indicate that the response patterns for most of the items were consistent with expectations, indicating that participants' responses to the items generally fit well within the predictions of the model. The Outfit values, which are more sensitive to outliers, also indicate that the responses to the items were mostly consistent, with only slight deviations that might be attributed to occasional unexpected responses. These results confirm that all items on the APO-15 were useful for measurement and contributed meaningfully to the overall assessment of positive occupational participation (Table 3). In addition, the weight map suggested that all items on the APO-15 functioned at item difficulty levels tailored to clients who are unable to participate in occupations that promote well-being (Figure 1).

Figure 1.

Note. The figure on the left shows the participants’ ability scores, while the figure on the right shows their item difficulty levels of the APO-15. When the vertical axis registers a positive score, it indicates higher participant ability or greater item difficulty. Conversely, negative scores indicate the opposite. Overall, the APO-15 has a high number of items of low difficulty, suggesting that it is usefull in identifying clients who having limitations in participation in occupations that promote well-being.

Interpretability

After examining the APO-15 evaluation criteria, we determined that the five-level criteria had the highest clinical practicality. Specifically, there was an average improvement in percentile rank of approximately 20%, and there was reduced variability in the number of applicable clients compared with the four- or six-level classifications. The analysis assessing the appropriateness of the five levels in the evaluation criteria confirmed the statistical significance (p < .001) of the APO-15 total scores and the Happiness Scale. Multiple comparisons further revealed significant differences between all levels, except for some instances within the APO-15 (level 1-2; p > .05) (Table 4). The interpretation of the measurement results was based on the total score of the APO-15, leading to the following criteria: a total score of 40 or less corresponded to a “Level 1 = very weak participation state” (T-score ≤41 points; percentile rank ≤19%; n = 279), for scores between 41 and 45 points, the classification was “Level 2 = weak participation state” (T-score 42 to 47 points; percentile rank 22–38%; n = 282), for scores between 46 and 49 points, the classification was “Level 3 = moderate participation state” (T-score 49 to 52 points; percentile rank 45–59%; n = 222), for scores between 50 and 54 points, the classification was “Level 4 = strong participation state” (T-score 53 to 58 points; percentile rank 62–79%; n = 264). Finally, scores of 55 or above indicated a “Level 5: Very strong participation state” (T-score ≥60 points; percentile rank ≥83%; n = 212), representing a favorable state toward Level 5. These results suggest that the APO-15 can effectively determine client status using an evaluation criterion of five levels.

Table 4.

Results of Kruskal-Wallis test and multiple comparisons.

Dependent variable (APO-15)
Variable	χ²	df	p-value	Sig
APO-15 total score	1133.990	4	.000	Yes
Multiple comparisons
Comparisons		R.Diff	ES	95% CI	z-value	p-value	adj p-value
Level 1	Level 2	−47.500	−.015	−.070; .040	−.537	.591	.591
	Level 3	−219.500	−.077	−.131; −.022	−2.723	.006	<.05*
	Level 4	−588.500	−.209	−.261; −.156	−7.419	.000	<.001***
	Level 5	−1035.000	−.367	−.414; −.318	−13.020	.000	<.001***
Level 2	Level 3	−172.000	−.101	−.155; −.046	−3.587	.000	<.01**
	Level 4	−541.000	−.333	−.381; −.283	−11.823	.000	<.001***
	Level 5	−987.500	−.604	−.638; −.568	−21.441	.000	<.001***
Level 3	Level 4	−369.000	−.375	−.421; −.326	−13.290	.000	<.001***
	Level 5	−815.500	−.814	−.831; −.794	−28.866	.000	<.001***
Level 4	Level 5	−446.500	−.517	−.556; −.475	−18.330	.000	<.001***
Dependent variable (happiness)
Variable	χ²	df	p-value	Sig
Happiness score	303.978	4	.000	Yes
Multiple comparisons
Comparisons		R.Diff	ES	95% CI	z-value	p-value	adj p-value
Level 1	Level 2	−131.037	−.065	−.120; −.010	−2.299	.022	<.05*
	Level 3	−328.133	−.175	−.228; −.121	−6.225	.000	<.001***
	Level 4	−510.772	−.275	−.326; −.223	−9.767	.000	<.001***
	Level 5	−657.066	−.328	−.376; −.278	−11.640	.000	<.001***
Level 2	Level 3	−197.096	−.165	−.218; −.111	−5.859	.000	<.001***
	Level 4	−379.735	−.324	−.373; −.274	−11.513	.000	<.001***
	Level 5	−526.029	−.378	−.424; −.329	−13.406	.000	<.001***
Level 3	Level 4	−182.639	−.207	−.259; −.154	−7.351	.000	<.001***
	Level 5	−328.933	−.284	−.334; −.232	−10.060	.000	<.001***
Level 4	Level 5	−146.294	−.129	−.183; −.074	−4.569	.000	<.001***

Note. R.Diff. = rank difference, ES = Effect size r (0.1 < small, 0.3 < medium, 0.5 < large), 95% CI = 95% confidence interval, adj p-values = Adjusted p-values, Sig = significance. The results of the analysis indicated that the dependent variable (total score on the APO-15 and the happiness scale) and the independent variable (level of the APO-15) were statistically significant. In addition, multiple comparisons also showed statistical differences among all levels except between levels 1-2 of APO-15.

Discussion

This study aimed to determine the validity and reliability of the APO-15 in individuals with physical illness. In conclusion, the APO-15 was shown to have good scale properties for individuals with physical illness. The findings provide occupational therapists with a reliable and valid assessment tool that will enable them to focus their practice on occupations that promote well-being.

The validity of the APO-15 was evaluated using item analysis, structural validity, and concurrent validity. Overall, the APO-15 demonstrated noteworthy validity. Specifically, in terms of structural validity, while the RMSEA value of 0.100 is at the upper end of the acceptable range, goodness-of-fit indices, such as the SRMR (0.049), CFI (0.944), and TLI (0.930) were within satisfactory ranges. These results generally support the factor structure of the APO-15 and suggest that it may be possible to replicate it in a population of individuals with physical illness. However, it is important to recognize that, given the demographic characteristics of this sample, the results may primarily reflect the experiences and conditions of clients aged ≥65 years. In this sample, 87.77% of participants were aged ≥65 years (n = 1,105), with a higher proportion of females (57.01%) and a prevalence of orthopedic diagnoses. In contrast, 12.23% of participants were aged ≤64 years (n = 154), with a higher proportion of males (61.69%) and a greater prevalence of cranial nerve-related diagnoses. This suggests that the demographic characteristics of this sample may have influenced the factor structure of the APO-15. To address this potential limitation, future research should evaluate the validity of the APO-15 in more diverse clinical populations, with particular attention to younger adults. Such studies may provide valuable insight into whether the factor structure observed in this study holds across different age groups and diagnostic categories, or whether different patterns emerge. This would enhance our understanding of the scale’s applicability and sensitivity in detecting occupational well-being across various clinical settings.

The study included the Happiness Scale and the K-6 to examine the concurrent validity of the APO-15. The use of single-item happiness scales to measure client happiness may raise concerns regarding validity. However, it is worth noting that the Happiness Scale was consistent with our expectation of a moderate positive correlation between APO-15 and happiness. Occupational therapy holds the belief that occupational engagement is essential for human well-being (Hammell, 2017). In other words, the results of this study may support that the phenomenon of the four factors of APO-15 being satisfied through participation in positive occupations, which increases an individual’s sense of well-being, also occurs in individuals with physical illness. Contrary to the predictions, a weak negative correlation was found between APO-15 and K-6, suggesting a limited relationship between APO-15 and client psychological distress. Previous research has shown that mental health problems, such as loneliness, depression, and anxiety disorders, are associated with social isolation and lack of social support (Hawkley & Cacioppo, 2010). In other words, the participants in this study were already receiving social support, which may have limited the relation between APO-15 and K-6.

The reliability of the APO-15 was evaluated through an analysis of internal consistency and item difficulty. Internal consistency was assessed using Cronbach’s alpha, which yielded highly satisfactory results. In addition, the item difficulty of the APO-15 was skewed towards the low end (−4.431 to −2.776), indicating that the scale is tailored to clients who face challenges in participating in occupations that promote well-being, as reflected in the weight map. These results show a difference compared to a previous study conducted with individuals with mental illness (Noguchi & Kyougoku, 2021), which reported item difficulties ranging from −3.352 to 1.813. Specifically, our study found item 6 to be the easiest (difficulty = −4.431), while items 1 and 2 were among the more difficult (difficulty = −2.776 and −2.872). In contrast, the previous study found item 14 to be the most difficult (difficulty = 1.813). The differences in difficulty reported between the two studies may be due to the different challenges the participant groups face. For individuals with physical illnesses, particularly older clients, items related to having motivation (item 1) and achieving goals (item 2) may be perceived as more difficult due to physical limitations and age-related declines that may limit their ability to set and pursue personal goals. However, for people with mental illness, item 14, which focuses on maintaining a positive outlook, may be particularly challenging due to the psychological and cognitive barriers often associated with mental illness. These variations highlight the sensitivity of the APO-15 to the diverse challenges faced by different client populations and suggest the versatility of the APO-15 to assess well-being through occupational participation under a variety of conditions.

Finally, the T-scale-based criterion of the APO-15 showed that the client’s state could be interpreted on five levels. In a previous study by Noguchi (2021), the APO-15 used latent rank theory to standardize participation in occupations that promote well-being into five ranks for 4,159 clients. These ranks are presented as five distinct groups: Rank 1 (very weak participation group) to Rank 5 (very strong participation group), approaching rank 5 is good status. Therefore, in light of the above results, the T-scale-based criteria developed in this study may be able to classify clients into five levels of status, although they did not show significance only between levels 1 and 2 of the APO-15. Furthermore, this criterion, supported by statistical evidence, has clinical utility in assessing a client’s state, including their happiness. Therefore, the five-level criteria of the APO-15 based on the T-Scale can be a useful tool for occupational therapists to explain the importance of improving the well-being of our clients.

Limitations and future research

This study’s limitations include a mainly older sample, limiting the findings' generalizability. In addition, the Happiness Scale used in this study consisted of a single item, which may not have adequately measured the participants' level of happiness. Therefore, future research should more comprehensively measure well-being and psychological distress through surveys that include a more diverse range of participants and examine the relationship with the APO-15 more broadly. Furthermore, it is important to closely examine the factor structure of the APO-15 in individuals with physical illness, as the findings of this study suggest that the factor structure of the APO-15 may differ from its original model when applied to this population. Therefore, further research is needed to assess whether a more appropriate factor structure exists for groups other than those with mental illness.

Conclusions

This study aimed to examine the psychometric properties of the APO-15 in individuals with physical illness. Findings show it’s a reliable, valid tool for evaluating well-being-enhancing occupations in this group. Occupational therapists can use the APO-15 to enhance clients' well-being through occupation-focused practice.

Supplemental Material

Supplemental Material - Psychometric properties of the assessment of positive occupation 15 for individuals with physical illness

Supplemental Material for Psychometric properties of the assessment of positive occupation 15 for individuals with physical illness by Takuya Noguchi and Makoto Kyougoku in Hong Kong Journal of Occupational Therapy

Footnotes

Acknowledgements

We thank the clients and cooperating research facilities that participated in this study.

Contributorship

All authors researched literature and conceived the study. TN and MK were involved in gaining ethical approval, patient recruitment, and data analysis. TN wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical statement

ORCID iD

Takuya Noguchi

Supplemental Material

Supplemental material for this article is available online.

References

Abdel-Khalek

A. M.

(2006). Measuring happiness with a single-item scale. SBP Journal, 34(2), 139–150. https://doi.org/10.2224/sbp.2006.34.2.139

American Occupational Therapy Association . (2020). Occupational therapy practice framework: Domain and process. American Journal of Occupational Therapy, 3(Suppl 2), p1–p87. https://doi.org/10.5014/ajot.2020.74S2001

Andrich

(1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573. https://doi.org/10.1007/BF02293814

Bolier

Haverman

Westerhof

G. J.

Riper

Smit

Bohlmeijer

(2013). Positive psychology interventions: A meta-analysis of randomized controlled studies. BMC Public Health, 13(1), 119. https://doi.org/10.1186/1471-2458-13-119

Centers for Disease Control and Prevention . (2020). The mental health of people with disabilities. Centers for Disease Control and Prevention. https://www.cdc.gov/ncbddd/disabilityandhealth/features/mental-health-for-all.html.

Centers for Disease Control and Prevention . (2022). Older adults. Centers for Disease Control and Prevention. https://www.cdc.gov/cdi/indicator-definitions/older-adults.html.

Charles

Townsend

(2013). Introduction to occupation: The art of science and living (2nd ed). Pearson.

Cornelius

B. L.

Groothoff

J. W.

Van Der Klink

J. J.

Brouwer

(2013). The performance of the K10, K6 and GHQ-12 to screen for present state DSM-IV disorders among disability claimants. BMC Public Health, 13(1), 128–138. https://doi.org/10.1186/1471-2458-13-128

David

S. A.

Boniwell

Ayers

A. C.

(2013). In Oxford Handbook of happiness. Oxford University Press.

10.

de Beurs

Boehnke

J. R.

Fried

E. I.

(2022). Common measures or common metrics? A plea to harmonize measurement results. Clinical Psychology & Psychotherapy, 29(5), 1755–1767. https://doi.org/10.1002/cpp.2742

11.

DeVellis

R. F.

(2016). Scale development: Theory and applications (4th ed). Sage Publications.

12.

Diener

Chan

M. Y.

(2011). Happy people live longer: Subjective well-being contributes to health and longevity. Appl Psychol Health Well Being, 3(1), 1–43. https://doi.org/10.1111/j.1758-0854.2010.01045.x

13.

Doble

S. E.

Santha

J. C.

(2008). Occupational well-being: Rethinking occupational therapy outcomes. Canadian Journal of Occupational Therapy, 75(3), 184–190. https://doi.org/10.1177/000841740807500310

14.

Eakman

A. M.

(2011). Convergent validity of the engagement in meaningful activities survey in a College sample. OTJR (Thorofare N J), 31(1), 23–32. https://doi.org/10.3928/15394492-20100122-02

15.

Eakman

A. M.

Carlson

M. E.

Clark

F. A.

(2010). The meaningful activity participation assessment: A measure of engagement in personally valued activities. Int J Aging Hum Dev, 70(4), 299–317. https://doi.org/10.2190/AG.70.4.b

16.

Fox

C. M.

Jones

J. A.

(1998). Uses of Rasch modeling in counseling psychology research. J Couns Psychol, 45(1), 30–45. https://doi.org/10.1037/0022-0167.45.1.30

17.

Hammell

K. W.

(2017). Opportunities for well-being: The right to occupational engagement. Canadian Journal of Occupational Therapy, 84(4-5), 209–222. https://doi.org/10.1177/0008417417734831

18.

Hawkley

L. C.

Cacioppo

J. T.

(2010). Loneliness matters: A theoretical and empirical review of consequences and mechanisms. Annals of Behavioral Medicine, 40(2), 218–227. https://doi.org/10.1007/s12160-010-9210-8

19.

Helliwell

J. F.

Layard

Sachs

J. D.

Neve

J. E. D.

(2021). World happiness report 2021. https://www.wellbeingintlstudiesrepository.org/cgi/viewcontent.cgi?article=1004&context=hw_happiness

20.

Bentler

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(2), 1–55. https://doi.org/10.1080/10705519909540118

21.

Kline

R. B.

(2023). Principles and practice of structural equation modeling (5th ed). Guilford Press.

22.

Law

Steinwender

Leclair

(1998). Occupation, health and well-being. Canadian Journal of Occupational Therapy, 65(2), 81–91. https://doi.org/10.1177/000841749806500204

23.

Linacre

J. M.

(2002). What do Infit and Outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16(2), 878. https://www.rasch.org/rmt/rmt162.pdf

24.

Lyubomirsky

King

Diener

(2005). The benefits of frequent positive affect: Does happiness lead to success? Psychological Bulletin, 131(6), 803–855. https://doi.org/10.1037/0033-2909.131.6.803

25.

Massé

Poulin

Dassa

Lambert

Bélair

Battaglini

(1998). The structure of mental health: Higher-order confirmatory factor analyses of psychological distress and well-being measures. Soc Indic Res, 45(1/3), 475–504. https://doi.org/10.1023/A:1006992032387

26.

Meulenkamp

Rijken

Cardol

Francke

A. L.

Rademakers

(2019). People with activity limitations' perceptions of their health condition and their relationships with social participation and experienced autonomy. BMC Public Health, 19(1), 1536. https://doi.org/10.1186/s12889-019-7698-9

27.

Mokkink

L. B.

Prinsen

C. A.

Patrick

D. L.

Alonso

Bouter

L. M.

de Vet

H. C.

Terwee

C. B.

(2019). COSMIN checklist manual. COSMIN. https://www.cosmin.nl/wp-content/uploads/COSMIN-study-designing-checklist_final.pdf.

28.

Noguchi

(2021). Latent rank structure in the assessment of positive occupation 15 (APO-15) (in Japanese). The Japanese Journal of Clinical Occupational Therapy, 8(1), 57–64. https://kenkyuukai.m3.com/journal/FilePreview_Journal.asp?path=sys%5Cjournal%5C20210426062613%2D10E7E713D9408356E03A8230FE77547C25A5ABA9F164E991DDAB6AAFB5CAE315%2Epdf&sid=848&id=3797&sub_id=55942&cid=471.

29.

Noguchi

Kyougoku

(2019). Program development and preliminary methods of applying the positive occupation-based practice to clients with mental disabilities. The Japanese Occupational Therapy research, 38(1), 54–63. (in Japanese). https://doi.org/10.32178/jotr.38.1_54

30.

Noguchi

Kyougoku

(2021). Psychometric properties of the Assessment of Positive Occupation 15 final version in individuals with mental illness. Hong Kong Journal of Occupational Therapy, 34(1), 3–12. https://doi.org/10.1177/1569186120985413

31.

Noguchi

Kyougoku

Teraoka

(2016). Development of the assessment of positive occupation (APO) participation to promote well-being (in Japanese). Sogo Rihabiriteshon, 44(1), 1097–1106. https://doi.org/10.11477/mf.1552200795

32.

Olsson

Drasgow

Dorans

N. J.

(1982). The polyserial correlation coefficient. Psychometrika, 47(3), 337–347. https://doi.org/10.1007/BF02294164

33.

Özkan

Ercan Doğu

Noguchi

Örsel

(2024). Validity and reliability of the Turkish adaptation of the assessment of positive occupation-15 (APO-15) in serious mental illness. OTJR (Thorofare N J), 0(0), Article 15394492241265377. https://doi.org/10.1177/15394492241265377

34.

Reitz

S. M.

Scaffa

M. E.

Dorsey

(2020). Occupational therapy in the promotion of health and well-being. American Journal of Occupational Therapy, 74(3), p1–p14. https://doi.org/10.5014/ajot.2020.743003

35.

Robinson

Kennedy

Harmon

(2012). Happiness: A review of evidence relevant to occupational science. Journal of Occupational Science, 19(2), 150–164. https://doi.org/10.1080/14427591.2011.634780

36.

Seligman

M. E. P.

Steen

T. A.

Park

Peterson

(2005). Positive psychology progress: Empirical validation of interventions. American Psychologist, 60(5), 410–421. https://doi.org/10.1037/0003-066X.60.5.410

37.

Wilcock

A. A.

(2015). An occupational perspective of health (3rd ed.). SLACK Incorporated.

38.

World Federation of Occupational Therapists . (2013). About occupational therapy. https://wfot.org/about/about-occupational-therapy

39.

World Health Organization . (2016). Mental health and older adults. World Health Organization. https://www.who.int/news-room/fact-sheets/detail/mental-health-of-older-adults.

40.

World Health Organization . (2021). Depressive disorder (depression). World Health Organization. https://www.who.int/news-room/fact-sheets/detail/depression.

41.

Yamashita

(2022). Analyzing Likert scale surveys with Rasch models. Research methods in applied linguistics, 1(3), 100022. https://doi.org/10.1016/j.rmal.2022.100022

42.

Yen

W. M.

(1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Appl Psychol Meas, 8(2), 125–145. https://doi.org/10.1177/014662168400800201

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB

Psychometric properties of the Assessment of Positive Occupation 15 for individuals with physical illness

Abstract

Objectives

Methods

Results

Conclusion

Keywords

Introduction

Methods

Participants

Measures

Demographic information

APO-15

Happiness Scale

The Japanese version of the Kessler Psychological Distress Scale (K6)

Statistical analysis

Materials

Sample characteristics

Item validity

Structural validity

Concurrent validity

Internal consistency reliability

Item difficulty

Interpretability

Results

Sample characteristics

Item validity

Structural validity

Concurrent validity

Internal consistency reliability

Item difficulty

Interpretability

Discussion

Limitations and future research

Conclusions

Supplemental Material

Supplemental Material - Psychometric properties of the assessment of positive occupation 15 for individuals with physical illness

Footnotes

Acknowledgements

Contributorship

Declaration of conflicting interests

Funding

Ethical statement

ORCID iD

Supplemental Material

References

Supplementary Material