Critical Appraisal of Published Clinical Trials Regarding the Effect of Complementary and Alternative Medicine in Reducing Pelvic Pain Related to Endometriosis

Background: Clinical trials are considered as the best method of scientific research.; So critical evaluation of the quality of clinical trials is necessary to achieve evidence-based medicine.

Materials and Methods: This critical appraisal of published clinical trials was conducted on the effect of complementary and alternative medicine in reducing pelvic pain related to endometriosis until November 2023. Out of the 2,037 retrieved articles of related databases, ten studies were evaluated. Review Manager (Rev Man) version 5.3 and the Cochrane risk of bias tools and ROB tool for randomizes trials were used for evaluating quality of articles.

Results: The results of pooled studies based on The Cochrane ROB tool (Good, Fair, and Poor) showed that the quality of two of the studies had Good, two of the studies had Poor quality and the rest of the studies had Fair quality.

Conclusion: Considering of the side effects of conventional treatment and with the aim of use the results of clinical trials in evidence-based medicine associated with the effectiveness of complementary and alternative medicine in reducing pelvic pain related to endometriosis, improving the quality of articles is essential. Clinical trials should be conducted according to standard principles and critically evaluated with relevant tools before publication.

As the health profession progresses and studies on the prevention, diagnosis and treatment of diseases are published, it is critical to be able to identify and distinguish the best existing evidence [1]. With the extension of medical sciences and the increasing number of research journals and articles published in them, the structure of published articles and their compliance with the reported standards has received more attention than before [2]. Controlled randomized trial is considered as the best method of scientific research to comprehend the effects of different interventions in individuals and patients and answer therapeutic questions [3, 4]. However, systematic review studies have shown that many of these trials do not have the desired reporting quality, and unfortunately systematic errors are found in many of these published studies, leading to invalid estimates of the effectiveness of clinical treatment [1, 5]. Examination of the quality of reports of clinical trial articles shows that the most problematic and ambiguous cases are in determining the sample size, the method used for random allocation of study participants, the details of any restrictions on randomization (e.g., classification and blocking) and how to blind and use statistical methods [3, 6, 7]. When evaluating any research, it is significant to remember that there are “three general areas to be considered: validity, results and relevance” [8]. To critically evaluate a journal article, it is essential to start by considering the research methods used in the study. This could be carried out using critical evaluation checklists which are specific to the study designs [9]. Trials with poor methodology and bias can mislead health policymakers and providers at all levels from national public health policy to the treatment of a disease, and critical evaluation of the quality of clinical trials is probable only if the design, guidance, and analysis of clinical trials are fully defined in published articles [10]. On the other hand, in latest years, the emphasis on evidence-based care as a way to raise standards of care and improve health services has been emphasized by health system policymakers [11]. In this transition, proper design and detailed reporting of research has a particular importance, and their protocol must be designed carefully and their results be reported fully [8]. The perception of evidence based medicine (EBM), well-defined as the ‘‘combination of the greatest research data with medical proficiency and patient ideals” [12]. In other words, evidence based medicine contains evaluating studies critically, combining research results and applying demanding precise proof in practice [13]. 

Given these facts in this article we want to evaluate articles which have considered the effect of alternative and complementary medicine in reducing pelvic pain related to endometriosis. Endometriosis is an inflammatory disease procedure, considered by endometrial-like tissue external the uterus and it may be related to cyclical and/or noncyclical pelvic pain and/or infertility [14]. It affects 70 million cases worldwide which parallels to 10-15% of women of reproductive age [15, 16]. The social and economic impacts of endometriosis are considerable [17]. Regarding to the treatment approaches of Endometriosis, surgical removal of lesion (laparoscopy), Pharma logical and medical treatment is directed to induce hypoestrogenism and amenorrhea, anovulation for which purpose oral contraceptive pills (OCPs), progestogens ordered by a multiplicity of methods, and gonadotropin-releasing hormone (GnRH) agonists and antagonists are used. Both OCPs and NSAIDs have inadequate efficiency for symptom controlling [18, 19], and the substantial side effect of medical treatments consequences rates of 25%–50% [20]. A significant point to be considered is the high occurrence of pain and sexual fluctuations initiated by endometriosis in sexually active women. Data display that dyspareunia is the main gynecological complaint that can perceived in endometriosis, followed by chronic pelvic pain (CPP). Further, 46% of these women did not experience effective reactions to existing conventional treatments [21, 22].

Considering of the side effects of conventional treatment and its disappointment to prompt a systematic effective response, improvement of complementary and integrative therapeutic medicines and methods is required to reduce the distress of patients [23]. For this purpose, it is necessary to evaluate studies not only focus on the influence of complementary and alternative medicine in reducing pelvic pain associated with endometriosis, but also have a good quality. Therefore, the present study intended to critically review published studies on the effect of complementary and alternative medicine in reducing pelvic pain related to endometriosis.

Identification and Selection of Articles

This study is a critical review of the literature, with the aim of examining the quality of randomized control trials in the ground of complementary and alternative medicine in women with pelvic pain correlated to endometriosis. To discover relevant studies published until November 2023, international and national databases, such as Cochran library, Medline (via PubMed), Web of Science, Scopus, and Persian databases including SID, Magiran were individually searched by two researchers without any time limit. Finally, the reference list of retrieved studies were manually searched to discover further relevant studies. To search databases, keywords containing: Complementary and Alternative medicine, Herbal therapy/medicine, Cupping, Message, Yoga, Homeopathy, Acupuncture, Traditional medicine, Reflex therapy, pelvic pain and Endometriosis reviews together with their parallels in Mesh and relating words with AND, OR functions were searched using search operators. The PRISMA 2020 flowchart for systematic reviews was used to extant the selection procedure for the studies. Inclusion criteria include a human clinical trial performed in the field of alternative and complementary medicine such as Complementary and Alternative, Herbal therapy/medicine, Cupping, Message, Yoga, Homeopathy, Acupuncture, Traditional medicine, Reflex therapy in Persian and English, and exclusion criteria were dearth of access to the English full text and nonconformity of the principles of randomized controlled trial.

First systematic search was performed using keywords and Booleans in databases: Cochran library: (224 articles), PubMed (110 articles), Web of Science (1,512 articles), Scopus (78 articles), and Persian databases including SID (75 articles), Magiran (38 articles).

Figure 1: PRISMA Flow diagram of the study

The titles and /or abstracts of the studies retrieved by the search strategy and those from extra sources obtained were independently selected by the two authors to classify studies that actually meet the aims of this review. The full text of these eligible articles was retrieved and evaluated for eligibility individually by two authors (S.M. and T.KH). They reviewed the quality and acceptability of the articles separately, and any disputes were resolved through consensus with a third author (N. M). Two authors evaluated the risk of bias separately based on the Cochrane risk of bias tools for randomizes trials [24], in terms of allocation sequence, blinding of participants, allocation concealment, , incomplete outcome data, personnel, and outcome assessors, and selective reporting bias (Figure 2). Review Manager (Rev Man) 5.3 was used for drawing the summary and graph of risk of bias charts (Figure 2, 3).

Figure 2: The graph of risk of bias.

Figure 3: Summary of risk of bias.

The Cochrane ROB tool was applied using specific thresholds to align it with AHRQ (Agency for Healthcare Research and Quality) standards of good, fair, and poor. In this tool, good quality implies all criteria are satisfied (i.e., low risk for each domain). Fair quality indicates one criterion is not met (i.e., high risk of bias for a domain) or two criteria are unclear, with an evaluation that this is unlikely to bias the outcome, and no significant known limitations that could compromise the results. Poor quality suggests one criterion was not met (i.e., high risk of bias for a domain) or two criteria are unclear, with an evaluation that this is likely to bias the outcome, and there are significant limitations that could invalidate the results, or two or more criteria are identified as high or unclear risk of bias. (Table 1).

Table 1: Characteristics of the Include Studies

The present study extracted articles focused on the effect of alternative and complementary medicine in reducing pelvic pain related to endometriosis. To compile the present study, first 2,037 articles were identified through databases according to the study question; among them, 1,628 articles were removed because of various reasons, including: unrelated with title and ineligibility; such as proceeding paper, being book, conference papers, document, review; then 183 articles were screened. Sixteen full-text articles were evaluated for eligibility which three of them included from references lists. Among them five articles were excluded because they examined the effect of alternative and complementary medicine on dysmenorrhea. An article examined the effect of alternative and complementary medicine on pelvic pain but it was in Chinese language; therefore, it did not enter the study. Finally, ten articles were included in critical appraisal of published clinical trials. In this critical appraisal study, the included studies have been performed between "2008-2023".  Included studies have been conducted in Brazil [23, 25], Australia [26, 27], the United Kingdom [28], Germany [29], and the United States [30, 31]. An article examined the effect of abdominal acupuncture on pain of pelvic cavity in women with endometriosis but because we could not access the English text of this article; so it was not included in this study [32]. Articles were indexed in ISI, all of them were Q2 [23, 25-28, 30, 31, 33, 34] except Meissner, K. et. al which was Q1 [29]. 

based on Cochrane risk of bias tools for randomizes trials [24], random sequence generation (selection bias) in all studies were low risk [23, 25, 26, 28, 29, 33, 34] except three of  them that were unclear risk [27, 30, 31]. Allocation concealment (selection bias) of was unclear risk in four study [27, 29-31] and low risk in others [23, 25, 26, 28, 33, 34]. Blinding of participant and personnel's (performance bias) were unclear risk in three study [27, 29, 33] and low risk in others [23, 25, 26, 28, 30, 31, 34]. Blinding of outcome assessment (detection bias) in fifty percent of studies were unclear risk [23, 27-29, 33] and the others were low risk [25, 26, 30, 31, 34]. Incomplete outcome data (attrition bias) in two study were high risk [26, 28] in one study unclear risk [27] and in the others were low risk [23, 25, 29-31, 33, 34]. Regarding selection reporting (reporting bias) all studies were low risk [23, 25, 26, 28-31, 33, 34] except of one study that were unclear risk [27]. Based on The Cochrane ROB tool in view of Thresholds for Converting the Cochrane Risk of Bias Tool to AHRQ Standards (Good, Fair, and Poor), two of the articles were evaluated as Good quality [25, 34], two of the articles were Poor quality [27, 29] and the rest were Fair quality [23, 26, 28, 30, 31, 33]. Pain measuring scales to measure pelvic pain in all studies were, a 0–10-point numerical scale, VAS (visual analogue scale).

The present study aims to evaluate critically published clinical trials on the impact of alternative and complementary medicine in reducing pelvic pain related to endometriosis. Based on the outcomes of this study, the quality of the most of the articles in the field of complementary and alternative medicine in reducing pelvic pain related to endometriosis were fair. The most defects were in blinding of outcome assessment (detection bias), allocation concealment (selection bias), blinding of participant and personnel's (performance bias) and random sequence generation (selection bias) respectively in clinical trial studies. Although no critical research has been done in this field so far; so it is difficult to make decision and conclusion about the findings, but we tried to discuss this issue by using other articles in similar fields and using review articles. These results of present study may be due to the lack of reporting of clinical trials and researchers' unfamiliarity with standard tools on how to write and report articles, although the tools are constantly reviewed, which are similar to the results of Sharifi et al. [35] study; Manouchehri et al. [36] and the study of Mehrazmay et al. [37]; sarayloo et al. [38] and Irani et al. [39] in the field of clinical trial quality. 

Although the random allocation sequence and its various ways in different studies are the main components in RCT studies, but sometimes this issue is not well reported. In the present study this issue was well reported in some studies [23, 25, 26, 28, 29, 33, 34]; but not in others [27, 30, 31]. Previous studies have not mentioned this matter too [6, 38, 40, 41]; even studies do not mention who managed the random allocation sequence, in the present study this issue was well reported in some studies [23, 25, 26, 28, 33, 34]; but not in others [27, 29-31], which seems to be because of the lack of consciousness of the authors about the significance of reporting this section and the principles of writing articles. About blinding of participants and personnel, who has been blinded by the intervention and how to do it, in the present study this issue was well reported in some studies [23, 25, 26, 28, 30, 31, 34]; but not in others [27, 29, 33] or outcome assessor was well reported in some studies [25, 26, 30, 31, 34]; but not in others [23, 27-29, 33]. These results were present in some other studies that examined the quality of the trials [38, 42] that it shows the weakness of the authors' knowledge in the importance of reporting this section. In the existing study, in all of the comprised studies, the results of them in terms of primary and secondary outcomes were expressed, except for the Rubi-Klein et al. study. However, the use of additional statistical analyzes was limited, which may be due to the lack of need to use them. In this study, all articles were indexed in very reputable databases, which are consistent of Vickers et al. study [43]. 

Finally, the results of this study generally showed the fair quality of reports and the weakness of the authors in the standard report of clinical trial. Although the included articles in this study were from reputable scientific sites, but most of them in terms of quality were fair or poor and just two of them had good quality. This may indicate that the authors did not use the accurate principles of conducting and reporting of clinical trials. It should be noted that in some evaluations of descriptive and cohort observations, the quality of articles was not satisfactory [44, 45], which indicates the need for researchers and authors to be familiar with the principles of research and training to use standard tools to perform and report different stages of research. The powers of this study were the use of the identical pain measuring instruments in all of the articles. In the critical appraisal of the articles, extensive searches in electronic databases and the validity and novelty of most of the included studies and their publication in journals with international indexes. One of the limits of this study was the exclusion of articles written in non-English (Chinese) languages. Also, the use of different complementary and alternative medicine in reducing pelvic pain related to endometriosis. The critical evaluation is influenced by individual views, which in this regard tried to overcome it by using the appropriate tools of critical appraisal and the opinion of a third researcher.  The number of studies comprised in this study was low, which makes it difficult to consider findings. Owing to the low number of studies in this issue, further studies with stronger design are recommended.

According of the results of this study in evaluating of critically published clinical trials on the effect of alternative and complementary medicine in decreasing pelvic pain related to endometriosis, the quality of the most of the articles were fair or poor. Considering to the economic and social impacts of endometriosis and substantial side effects of medical treatments rates and its disappointment; developing of complementary and integrative therapeutic medicines and approaches is required to reduce the suffering of this patients. According to the findings of this study and considering that the results of clinical trials are based on evidence-based medicine and considering the importance of complementary medicine, the existing researches in this field should be performed and reported with standard and correct principles. Be critically evaluated for publication with accurate and valid tools. Researchers should consider conducting studies with an accurate and standard method. Educating researchers to conduct appropriate studies and using a standard checklist to improve the quality of articles is recommended.

Acknowledgement

We express our appreciation to everyone for assistance in providing informational support for this study.

Conflict of interest

The authors declare that they have no conflict of interest.