Comments on “Caregiver Burden and Disability in Somatoform Disorder: An ExploratoryStudy”

Quasi-controlled studies continue to be frequently conducted and published in India despite the serious limitations of this research design and the constraints that the design places on strategies for data analysis. ¹ In a quasi-controlled study, Chinneimawi et al ² compared 28 patients with somatoform disorder with 28 patients with schizophrenia. They found no significant differences between groups in patient disability and caregiver burden. They concluded in their abstract that patients with somatoform disorders and those with schizophrenia are comparable in these two regards. We consider potentially problematic issues in this article.

Any unusual finding merits deeper examination. Schizophrenia is a major mental illness and, unlike patients with somatoform disorder, many patients with schizophrenia are incapable of independent living. So, it is surprising that somatoform disorders and schizophrenia patients did not differ significantly in disability and caregiver burden. There are three possible explanations. Because this was a tertiary care, hospital-based study, the sample would have been biased toward inclusion of patients with more rather than less severe somatoform disorder. Burden and disability would be higher in such patients than in somatoform disorder patients in the community. In contrast, almost all patients with schizophrenia would receive hospital care, regardless of illness severity. Furthermore, because the sample was recruited from specialty clinics, the schizophrenia patients may have been less severely ill than those in wards. To their credit, the authors recognized and accepted this limitation in their discussion. However, the conclusion section of their abstract remained misleading.

Another possible explanation for the surprising findings is that the authors did not state whether or not consecutive patients were recruited; so, it is possible that somatoform disorder patients with greater burden/disability were selectively referred and recruited, thereby increasing the sampling bias.

Here is a third explanation, something that few investigators realize. Somebody who experiences a severe toothache will rate it 10/10 for severity of pain. Somebody who experiences 50% third-degree burns will also assign a 10/10 rating for severity of pain. This does not mean that the two are equal. This is an important reason why comparison of ratings between groups can be fallacious in quasi-controlled studies, and why similar ratings may have characterized somatoform disorder and schizophrenia for burden and disability.

It is surprising that the authors were not cued to the incongruity of their findings despite their observation that the schizophrenia patients in their sample were significantly less likely to be employed. That, itself, should have suggested that disability and burden should have been greater in schizophrenia.

Moving to a second issue, the authors presented their sample size estimation to support n = 28 per group. Now, here is something that many investigators do not know. Regardless of what they are studying, and regardless of what mean difference they want to detect between two groups, and regardless of what the standard deviations are in the two groups, they need a sample size of 29 per group to detect an effect size of 0.75 with 80% power at P<0.05, and they need a sample of 63 per group to detect an effect size of 0.5 with 80% power at P<0.05. The sample size in the study was 28/group, which would allow Chinneimawi et al (2021) to detect an effect size of somewhere between 0.75 and 0.80. In other words, their study was only powered to detect a large effect size. It was underpowered to detect a medium effect size, and very seriously underpowered to detect a small effect size. In such a situation, it is fallacious to conclude that because the groups did not differ significantly in burden and disability, they were comparable.

Moving to a third issue, the statistical methods that the authors used were problematical. They converted continuous variables into categorical variables for analysis, which is discouraged for many reasons. ³ They used t-tests when distributions were non-normal, such as for comparing caregiver burden between groups; a Mann–Whitney test should have been employed. They used the chi-square test in most of their tables when the expected frequency assumption for chi-square testing was not met. Last but not least, they examined relationships between apparently every variable and every other variable with no a priori hypotheses. This was, therefore, a fishing expedition, something that is sometimes classified as research misconduct. ⁴

We hope that readers will benefit from this perspective on how to read a journal article critically. This letter was stimulated by a discussion of the study in eJournalClubIndia. ⁵