Abstract
Background:
Preclinical studies often provide the evidence base for clinical studies. However, the design and reporting of preclinical trial results are inadequate, resulting in poor reproducibility and clinical translatability. We aimed to systematically evaluate the methodology and reporting quality of animal studies of acupuncture for cancer pain.
Methods:
About 7 databases were searched for animal research articles on acupuncture for cancer pain from the beginning of the database to January 31, 2022. ARRIVE guidelines, STRICTA, and SYRCLE risk of bias tools were used to assess the reporting quality and risk of bias of the selected studies.
Results:
A total of 18 studies were evaluated. Of the 22 items on the SYRCLE tool, only 6 items had a positive reporting rate of more than 50%. Of the 39 items in the ARRIVE guidelines, 14 were rated excellent, and the least frequently reported checklist items were 7. Out of the 17 STRICTA checklist items analyzed, 10 were considered appropriately reported in more than 80% of the studies, while 4 were correctly reported in less than 20%.
Conclusions:
Some crucial points in the design, implementation, and reporting of the experiments included in the study were not well developed, which could significantly affect the clarity, reproducibility, and translatability of the experiments. There is a need to fully implement scientific tool guidelines for future experimental studies in order to improve the quality of preclinical studies and facilitate effective translation of their results to the clinic.
Introduction
Improvements in early detection and treatment have led to a growing prevalence of survivors of cancer worldwide. 1 Pain is one of the most frequent and problematic symptoms in cancer, with an incidence rate of 50.7% for all cancer stages, and 66.4% for advanced stages. 2 Although the World Health Organization analgesic ladder provides effectual approaches to relieve cancer pain, addiction to analgesics and the adverse effects of pharmacological interventions pose critical challenges to pain management. 3 Therefore, to relieve the adverse drug reactions of cancer pain and the huge economic burden caused by the treatment process, it is urgent to find a suitable alternative treatment plan. Acupuncture, as the most common method of traditional Chinese medicine in physical intervention, has been widely used for chronic pain management. 4 In the last 20 years, there have been many clinical reports on the utility and safety of acupuncture for the treatment of cancer pain, and acupuncture therapy is a widely recognized alternative measure for the treatment of cancer pain. 5 A recent systematic review and meta-analysis showed that acupuncture was significantly associated with cancer pain alleviation and reduced the use of analgesics with a moderate level of evidence. 3 It is notable that in recent years, there has also been an increasing number of studies on the mechanisms of acupuncture for cancer pain.
Animals are used in the life sciences to elucidate normal biology, to improve understanding of disease pathogenesis, and to develop therapeutic interventions. 6 Importantly, the beneficial effects achieved in animal models rarely fully translate into functional improvements in patients. 7 Throughout the process of a scientific experiment, from planning to design, execution, and reporting, various barriers that create obstacles in the path to translation may arise. 8 Increasingly, reviews highlight that inadequate experimental reporting can result in such studies being un-interpretable and difficult to reproduce.9-11 Thus, it is crucial for high quality of reporting and methodology to ensure trustworthy, transparent and accurate explanation of evidence.
Animals in Research Reporting In Vivo Experiments (ARRIVE) guidelines is a checklist intended to provide transparent and accurate reports of animal studies. It was developed in 2010 and updated in 2020 (ARRIVE 2.0) to solve the reproducibility problem in animal research. 12 The STRICTA (Standards for Reporting Interventions in Clinical Trials of Acupuncture) reporting guidelines, first published in 2001, were designed to improve the completeness and transparency of reporting of interventions in controlled trials of acupuncture, in order that such trials may be more accurately interpreted and readily replicated. 13 Systematic Review Centre for Laboratory animal Experimentation (SYRCLE), based on the Cochrane Collaboration RoB tool, is an assessment instrument to evaluate the risk of bias and methodological quality of animal studies. 14 To date, few studies have evaluated the methodological and reporting quality of experiments on acupuncture for cancer pain.
Therefore, the purpose of this study was to assess the methodological and reporting quality of experimentalstudies of acupuncture for cancer pain, which will contribute to future advances in experimental design and provide a valuable basis for clinical research.
Methods
Strategy Searching
An electronic search was conducted through 7 databases PubMed, Embase, Scopus, Web of science, Database for Chinese Technical Periodicals (VIP), China National Knowledge Infrastructure (CNKI), and Wanfang Data, toidentify all published animal research up to January 31, 2022. The keywords used were as follows: cancer pain, bone cancer pain, cancer-induced bone pain, acupuncture, electroacupuncture, wrist-ankle acupuncture, auricular acupuncture.
Inclusion Criteria
The inclusion criteria were (1) animal models of cancer pain; (2) acupuncture or electroacupuncture as an treatment for cancer pain, while the control group received standard treatment or placebo.
Exclusion Criteria
The exclusion criteria were (1) the use of acupuncture for cancer pain relief, which contained clinical studies; (2) experiments not studying cancer pain; (3) reviews, publications without full texts, duplicated studies.
Data Extraction
A reviewer (XXH) developed a search strategy. The database was searched and a list of all records was made. About 2 researchers (YLL and LBJ) independently checked the articles according to the inclusion criteria and extracted data for analysis. The following data were extracted: publication year, strain, sex, the last name of the first author, model of cancer pain, stimulator parameters (method of treatment with acupuncture), and the results of each article. Differences were solved together through discussions (XXH, LBJ, HW).
Methodological and Reporting Quality Assessment
Risk of bias of the included studies was assessed by the reviewers using SYRCLE’s Risk of Bias tool. This review focused on 6 different types of bias, which included selection bias, performance bias, detection bias, attrition bias, reporting bias and other potential sources of bias. 15 A third author was consulted to resolve discrepancies related to risk of bias. The judgments of “L” denoting low risk of bias, “H” denoting high risk of bias and “U” denoting unknown risk of bias, were reported to properly assess the risk of bias.
ARRIVE and STRICTA were used to evaluate the quality of reports. About 2 trained researchers with no interest conflicts extracted the data independently and evaluated the report item by item as “reported,” or “not reported.” In case of disagreement, a third researcher with no interest conflicts should arbitrate. The number of research studies eligible for each item was calculated, as well as the percentage of reported items. We calculated a ratio quality score/maximum score, generating 3 possible quality intervals of which 0.8 to 1 was considered “excellent,” 0.5 to 0.8 was considered “average,” and scores below 0.5 were considered “poor.” 16
Data Analysis
We used Microsoft Excel for the descriptive statistical analysis, and summary statistics were given as percentages. Kappa testing for agreement among reviewers was performed using the Statistical Package for the Social Sciences (SPSS) version 28.0 (IBM Corporation, Armonk, NY, USA). A kappa index <0.4 suggested poor agreement, 0.4 to 0.75 general agreement, and ≥0.75 or higher/excellent agreement.
Results
Study Selection
The database search strategies obtained 1083 records, and 714 duplicates were identified and excluded. Another 369 articles were excluded after examining the title and abstract for clinical trials or irrelevance to cancer pain. Further full-text reading excluded 10 articles as they were duplicate studies and inappropriate outcome indicators. Finally, 18 studies were included in this review.17-34 The literature selection process is detailed in Figure 1.
Article selection flow chart.
Basic Characteristics of Included Studies
All the studies were reported from 2008 to 2021. Of these studies, 12 were published in Chinese journals and the remaining 6 were published in English journals. Regarding the strain and sex of animals, 9 studies selected Sprague-Dawley rats, of which 7 used male and 2 used female animals; 7 selected Wistar rats, of which 5 used female and 2 used male animals; 1 study selected Copenhagen male rats, and 1 study selected BALB/c male mice. Of these studies, 3 used subcutaneous xenograft models, 1 used a neuropathic cancer pain model, and the others used cancer-induced bone pain models. As for the inoculated tumor cells, 1 study used AT-3.1 prostate cancer cells, 1 study used S-180 sarcoma cells, while the other studies used Walker 256 breast cancer cells.
A range of acupuncture techniques was employed with regard to the combinations of acupoints and the stimulation method (electroacupuncture [EA] or wrist-ankle acupuncture [WAA]). About 10 studies used acupoints ST36 (zusanli acupoint) and BL60 (Kunlun acupoint) pointed. About 3 studies chose to apply acupuncture treatment to a single acupoint ST36 (Zusanli), 3 studies selected L3-5 Jiaji points, 2 studies selected GB30 (Huantiao acupoint), and 1 study selected BL2 (Cuanzhu acupoint). About 1 study chose wrist-ankle acupuncture as the method of stimulation, while other studies used electroacupuncture. The acupuncture intensity of the studies was between 1 and 2 mA, and the frequency was almost 2/100 Hz.
To evaluate the analgesic effect of acupuncture, paw withdrawal latency (PWL), paw withdrawal pressure threshold (PWTs), and tail lick latency (TFL) were tested. To elucidate the mechanisms acupuncture analgesia, the indicators of μ-opioid receptor (MOR), κ-opioid receptor (KOR), δ-opioid receptor (DOR), pro-opiomelanocortin (POMC), pro-dynorphin (PDYN), glial fibrill aryacidic protein (GFAP), CXCL12, and NF-κB were measured. To indicate a role of acupuncture in immunity, Splenic Con A (Concanavalin A), Percentages of splenic T cell subsets and Splenic NK cells activity were tested. Table 1 indicated these characteristics of the included studies.
Basic Features of Included Literature.
Abbreviations: C, Copenhagen rats; M, male; F, female; W, wistar rats; SD, Sprague-Dawley; CIBP, cancer induced bone pain; MOR, μ-opioid receptor; KOR, κ-opioid receptor; DOR, δ-opioid receptor; POMC, pro-opiomelanocortin; PDYN, pro-dynorphin; PWL, paw withdrawal latency; GFAP, glial fibrill aryacidic protein; TFL, tail flick latency; WAA, wrist-ankle acupuncture; Con A, concanavalin A; PWTs, paw withdrawal pressure threshold; ↑, up-regulation; ↓, down-regulation; EA, electroacupuncture; /, not mentioned; ST36, Zusanli; BL60, Kunlun; GB30, Huantiao; BL2, Cuanzhu.; NCP, neuropathic cancer pain; SIP, subcutaneous tumor pain; B, BALB/c.
No obvious change.
Risk of Bias Assessment
Based on the evaluation by the SYRCLE tool (Figure 2), the main observation from the risk of bias and methodological quality assessments is the many “unclear” scores, indicating that most items were not sufficiently reported, resulting in an unknown risk of bias. With respect to selection bias (item 1), the sequence generation process was reported in only 4 studies,17,18,23,24 Although many studies mentioned that the animals were randomly assigned to exposure groups, the randomization method was unclear. As a result, the risk of bias on Item 1 could not be judged for many articles. Baseline similarities were reported 9 studies19-21,23,26,29,32-34 (items 2-4), whereas information about allocation concealment was not reported at all (item 5). Random placement of cages or animals in animal facilities was stated for all studies (items 6, 7).17-34 However, 3 studies described blinding animal breeders and researchers (item 8).24,28,34 As a result, performance bias could not be judged. Regarding detection bias (items 9-11), 2 studies28,32 described a random outcome assessment for relevant outcome measures (item 9). In addition, the outcome assessor was reported to have been blinded in 4 studies (items 10, 11).21,23,30,31 Incomplete outcome data were adequately addressed in 5 studies18,21,22,32,34 (items 12-15), and other studies were scored with an unclear risk of bias on this item, resulting in the risk of attrition bias in these studies could not be judged. With respect to reporting bias (items 16, 17), a high risk was identified for 1 studies. 21 All other studies were scored with a low risk of bias on these items. Additionally, other sources of bias were not clearly described, and could lead to a high risk of bias (items 18-22). Overall, the methodological quality of animal experiments is poor in terms of the selection bias, performance bias, detection bias, attrition bias and other potential sources of bias. These risks of bias are mainly due to inadequate description. The interrater reliability was excellent between the 2 assessors (kappa = 0.781). The details can be found in Supplemental File 1 and Figure 2.
SYRCLE risk of bias tool for included studies.
Quality Assessment of the Studies
Figure 3 presents the percentage of the individual items in the ARRIVE guidelines checklist to assess the quality of the studies selected for this review. According to ratio of reported score/maximum score, 14 items were scored as excellent (found in ≥80% of the studies): (1) study design, (2a) sample size, (6a) clearly define all outcome measures assessed, (8) experimental animals, (9) experimental procedure, (10) results, (11) abstract, (12) background, (13) objectives, (16a) animal care and monitoring, (17a) interpretation/scientific implications, (21b) declaration of the funder(s) in the design, analysis, and reporting of the study. The least frequently reported checklist items (only found in ≤20% of the studies) were items (3) inclusion and exclusion criteria, (4) randomization, (5) blinding, (7a) experimental procedures, (19) protocol registration, (20) data access. Overall, of the 39 items of ARRIVE, 14 items were assessed as poor and 14 items were assessed as excellent. As is evident from Figure 4, none of the analyzed studies fully complied with the ARRIVE guidelines. The mean checklist score of the studies published from 2008 to 2021 was 62%. See Supplemental File 2 for detailed scoring. The interrater reliability was excellent between the 2 assessors (kappa = 0.904).
Reporting quality assessment using the ARRIVE2.0 tool.
Bar chart displaying the scores of the analyzed articles.
STRICTA guidelines were used to assess the quality of included article reports, as shown in Table 2. The positive reporting rates of items such as “acupuncture rationale,” “reasons for acupuncture treatment,” “acupoint names,” “Needle stimulation,” “needle retention time,” “treatment regimen,” “complementary interventions for the acupuncture group,” and “precise description of the control intervention” were above 80%. However, the positive reporting rates of items such as “needle type” and “responses sought (arrival of Qi)” were below 60%. None of the articles reported the setting and context of the treatment, the background of the practitioners, the rationale for the control, the rationale for the choice of a control or comparison group in the context of the research question, with sources to justify this choice. The interrater reliability was excellent between the 2 assessors (kappa = 0.935).
STRICTA 2010 Checklist for the Included Studies.
Abbreviations: Y, yes; N, no.
Discussion
Preclinical studies play an important role in scientific progress and the discovery of new and potentially successful treatments, assuming that they are designed, conducted, evaluated, and adequately reported in accordance with internationally agreed guidelines. 35 Fundamental experimental design components are often overlooked in scientific papers, which result in such studies being uninterpretable and difficult to reproduce, 6 and preclinical studies on acupuncture for cancer pain do not appear to be an exemption.
This review indicates that the methodological quality of experimental studies of acupuncture for cancer pain is generally poor and the quality of reporting requires improvement. In particular, bias due to insufficient randomization and lack of blinding is evident. All studies mentioned randomization methods, but the methods used for randomization were unknown or not reported in sufficient detail. Whether this is related to inadequate reporting where adequate methods were used or inadequate methods is uncertain. Since randomization is an important tool for creating comparable study groups, which assign potential risk factors equally across groups, the lack of adequate reporting is of particular concern, as lack of randomization may affect study results. In animal studies, disease is usually induced rather than naturally present. Therefore, the timing of randomization is more important than in the patient setting. It needs to be assessed whether disease is induced prior to actual randomization and whether the order of induction is randomly assigned. In addition, lack of allocation concealment prior to disease induction may lead to variations in study conditions prior to the intervention between groups, leading to biased results. Another important potential source of bias is lack of blinding. Investigators and caregivers are not blinded or do not report blinding at all, which may consciously or unconsciously influence further treatment/care throughout the treatment protocol and ultimately affect outcomes. Detection bias occurs when the assessment of outcomes is not the same for each study group, and blinding of randomized outcomes and outcome assessors is designed to reduce this bias. Lack of blinding is an important and fundamental methodological weakness, which may raise doubts about the validity of the study. In addition to the above types of bias, other risks of bias including contamination, potentially inappropriate influence of funders, unit of analysis errors, design-specific risks of bias, and addition of new animals to groups to replace drop-outs from the original population also require attention.
It is worth noting that risk of bias and reporting quality should be considered distinct from each other. While the former relates to the internal validity of the trial, the latter points to how the researchers reported their findings. Although personal communication with the authors of a study may be intended to clarify questionable or missing information, this does not guarantee the correctness of the information provided. To ensure scientific rigor, criteria for inclusion and exclusion of animals and data should be determined prior to the start of the experiment and collection of data. Reporting these criteria helps the reader interpret the data and provides key information for other researchers wishing to adopt the model. The statistical analysis methods implemented will reflect the goals and the design of the experiment. The statistical analysis methods implemented will reflect the objectives and design of the experiment. The description of the statistical analysis should provide sufficient detail so that another researcher can reanalyze the original data using the same methods and obtain the same results. Akin to the approach taken for clinical trials, protocol registration has emerged as a mechanism that is likely to improve the transparency of animal research. It enhances scientific rigor and protects the researcher against concerns about selective reporting of results. Reporting of data sharing statements ensures reproducibility of experimental data and provides more value for money in preventing unnecessary use of animals. In the STRICTA guidelines, most items are reported at 80% or higher. Nevertheless, these following aspects still need to be noted, the first being the needle details for finding the acupuncture response after needle entry (arrival of Qi). The lack of needle details not merely reduces the credibility of the study results and the objectivity of the efficacy, it also hinders the promotion and international development of acupuncture therapy. Secondly, there is insufficient description of the therapist and treatment setting and context, which may affect the generalizability of the trial results. Lastly, in terms of explaining the reasons for the control group intervention, the selection of the control group intervention should be combined with medical ethics and the scientific nature of the study and should be justified.
Acupuncture for cancer pain has been studied mainly on models of cancer-induced bone pain. The mechanisms involved in the modulation of cancer pain by acupuncture have not been fully elucidated. These studies have investigated a number of possible mechanisms, including actions on the opioid system and inflammatory factors. Due to the diversity and complexity of cancer pain mechanisms, research on the mechanisms of acupuncture for cancer pain has gradually increased. However, the methodological and reporting quality of most studies is not high, which seriously hinders the reproducibility of experiments and clinical translation. Therefore, there is a necessity for strict implementation of animal research guidelines in future studies.
This systematic review has several limitations. First, being limited to English and Chinese databases, other language studies may be missed. The absence of other language studies may introduce selective bias. Second, the total sample size was still not large enough, although we made a concerted effort to search all the studies that met the inclusion criteria. Third, the quality of the included studies was unsatisfactory, which had an important influence on the results of the systematic. In addition, the subjective bias in the evaluation will also have a certain impact on the evaluation.
Conclusions
This review shows that the methodological and reporting quality of experimental studies of acupuncture for cancer pain is unsatisfactory. Especially, the randomization and blinding methods are inadequate. It is recommended that future experimental study designs be completed under the guidance of animal research guidelines to facilitate the production of manuscripts with scientific merit and clear conclusions.
Supplemental Material
sj-docx-1-ict-10.1177_15347354221123788 – Supplemental material for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment
Supplemental material, sj-docx-1-ict-10.1177_15347354221123788 for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment by Liangbo Jiao, Xiaoxia He, Jihong Zhang, Yali Liu, Yanan Luo and Hu Wei in Integrative Cancer Therapies
Supplemental Material
sj-docx-2-ict-10.1177_15347354221123788 – Supplemental material for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment
Supplemental material, sj-docx-2-ict-10.1177_15347354221123788 for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment by Liangbo Jiao, Xiaoxia He, Jihong Zhang, Yali Liu, Yanan Luo and Hu Wei in Integrative Cancer Therapies
Supplemental Material
sj-docx-3-ict-10.1177_15347354221123788 – Supplemental material for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment
Supplemental material, sj-docx-3-ict-10.1177_15347354221123788 for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment by Liangbo Jiao, Xiaoxia He, Jihong Zhang, Yali Liu, Yanan Luo and Hu Wei in Integrative Cancer Therapies
Research Data
sj-xlsx-4-ict-10.1177_15347354221123788 – Supplemental material for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment
sj-xlsx-4-ict-10.1177_15347354221123788 for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment by Liangbo Jiao, Xiaoxia He, Jihong Zhang, Yali Liu, Yanan Luo and Hu Wei in Integrative Cancer Therapies
Research Data
sj-xlsx-5-ict-10.1177_15347354221123788 – Supplemental material for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment
sj-xlsx-5-ict-10.1177_15347354221123788 for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment by Liangbo Jiao, Xiaoxia He, Jihong Zhang, Yali Liu, Yanan Luo and Hu Wei in Integrative Cancer Therapies
Research Data
sj-xlsx-6-ict-10.1177_15347354221123788 – Supplemental material for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment
sj-xlsx-6-ict-10.1177_15347354221123788 for Effects of Acupuncture on Cancer Pain in Animal Intervention Studies: A Systematic Review and Quality Assessment by Liangbo Jiao, Xiaoxia He, Jihong Zhang, Yali Liu, Yanan Luo and Hu Wei in Integrative Cancer Therapies
Footnotes
Author Contributions
XXH and LBJ contribute the same amount to the article. XXH, YLL, and LBJ selected, extracted, and evaluated the data. HW, YNL, and JHZ commented on and edited all drafts. All the authors read and approved the final manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
