Comment on: Cytokines (IL1β,IL6,TNFα) and serum cortisol levels may not constitute reliable biomarkers to identify individuals with post-acute sequelae of COVID-19

Dear Editor,

I read the paper by Fleischer et al. ¹ with interest. The authors report on their investigation of four selected biomarkers (IL1β, IL6, TNFα, and serum cortisol) for the identification of post-acute sequelae of COVID-19 (PASC).

The authors state that this is a prospective cohort study, which is obviously observational in nature. Based on the reference numbers given for the local ethics committee, the study appears to be closely related to another study by Fleischer et al., ² where the same reference numbers are given and which was also described by the authors as a prospective observational cohort study. Unfortunately, neither of the two studies was deposited in a registry for clinical studies, even though this is recommended for observational studies today^3–5 and the current registries also include many observational studies. This makes it impossible to determine the relationship between the two studies. It is also not possible to verify whether the reported results correspond to the original objectives of these studies. Particularly in the case of observational studies, great importance should be attached to this, and this can therefore be seen as a clear methodological weakness leading to a high risk of bias. ⁶ It is also unclear whether sample size calculations were performed and what sample sizes were originally planned for the very unbalanced study groups.

Furthermore, it is not explained in more detail in which sense the study was planned prospectively. For the four biomarkers investigated and their significance in the context of PASC, reference is made to studies whose results were published in 2021 and 2022. In particular, reference is also made to the WHO Delphi consensus criteria of October 2021 for the definition of PASC. Since the inclusion of patients began in January 2021, at a time when neither the definition of the PASC group was possible nor the significance of the selected biomarkers in this context was known, it must be assumed that the study objectives were originally different.

Another important methodological weakness lies in the insufficient consideration of the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline, ⁷ which describes how observational studies should be reported. Important points of the STROBE checklist, which were not adequately addressed in the article, are for instance points 9 (Bias), 12 (Statistical methods), and 13 (Participants). It is not clear, for example, how an attempt was made to avoid selection bias, whether and how potential confounders were examined, and how exactly the 178 participants examined in the study were came about (e.g. numbers examined for eligibility, confirmed, included in the study, and analyzed).

In addition, the description of the statistical analysis also reveals methodological weaknesses. The important results of the Kruskal–Wallis test were not reported. Suitable post hoc tests in this case would not be Dunnett’s multiple comparison tests, but Dunn’s multiple comparisons tests. Strictly speaking, however, these post hoc tests would not be necessary if the Kruskal–Wallis tests were not significant, as post hoc tests are usually only applied in the case of a significant result. They are used to uncover the specific differences between three or more groups.

Since a nonparametric test was used to compare the groups, it is also unclear why the Shapiro–Wilk test was also applied, which is a test of normality (not parametric distribution). Based on the data distributions shown in Figure 1 of Fleischer et al., ¹ it can be assumed that there was clearly no normal distribution, at least for the cytokines. Accordingly, it is doubtful that Pearson’s correlation is applicable in this case, as the necessary assumption of a linear relationship between the parameters investigated is clearly questionable. In view of the (right-)skewed data distributions for the cytokines and some visible extreme data points (outliers), which are also the causes of the very large standard deviations of the ‘No prior COVID-19’ group in Figure 1 of Fleischer et al., ¹ it must be assumed that the results given for the correlation are distorted. It is known that Pearson’s correlation is very sensitive to outliers. ⁸ Due to the skewed data distributions and since the Kruskal–Wallis test is not a test for the mean value, but generally examines a shift in the location of the distribution (more precisely, stochastic dominance), the specification of mean ± standard deviation in Table 2 and Figure 1 of Fleischer et al. ¹ can be considered as not appropriate from a statistical point of view.

In observational studies, as in the present study, confounders must always be taken into account, since they can have a decisive influence on the result. ⁹ However, there is no detailed clinical description of the study cohort, such as information on comorbidities that could substantially influence the concentrations of the biomarkers investigated. Nothing is mentioned in the Statistics section either, and only a few demographic parameters are listed in Table 1 of Fleischer et al., ¹ the relevance of which for the results does not appear to have been examined in detail. Overall, the impression is that not enough attention was paid to confounding factors.

Due to the methodological and statistical flaws described above, it must be concluded that the reported results are highly likely to be biased and that the existence of false negative results does not seem unlikely. The final conclusion made is that ‘the above-mentioned cytokines and cortisol are not appropriate biomarkers. The results of this study are consistent with our previous findings and those of others who did not find any laboratory changes and have suggested a nonorganic/psychosomatic genesis of PASC.’ The first part of this conclusion is already implied in the title of the paper and must, therefore, be clearly questioned. It also ignores the long-established fact that the absence of evidence is not the evidence of absence. ¹⁰ Further studies are certainly needed to be able to exclude IL1β, IL6, TNFα, and serum cortisol as biomarkers associated with PASC. Finally, it is not sufficiently argued why one can conclude a ‘nonorganic/psychosomatic genesis of PASC’ on the basis of only four negative biomarkers when there is contradictory evidence in the literature, as also stated in the introduction of Fleischer et al., ¹ and when there are tens of thousands of possible biomarkers that could prove an organic genesis of PASC.