A systematic review of sham acupuncture validation studies

Background Acupuncture is widely used worldwide; however, studies on its effectiveness have been impeded by limitations regarding the design of appropriate control groups. In clinical research, noninvasive sham acupuncture techniques can only be applied through validation studies. Therefore, this systematic review aimed to evaluate the scope of existing literature on this topic to identify trends. Methods We queried Pubmed, EMBASE, and the Cochrane Central Register of Controlled Trials databases from inception to July 2022 for relevant articles. Author names were used to identify additional relevant articles. Two independent reviewers assessed the identified articles based on the inclusion and exclusion criteria. The following data were extracted: study design, information regarding acupuncturists and participants, general and treatment-related characteristics of the intervention and control groups, participants’ experience of acupuncture, and research findings. Results The database query yielded 673 articles, of which 29 articles were included in the final review. Among these, 18 involved the use of one of three devices: Streitberger (n = 5), Park (n = 7), and Takakura (n = 6) devices. The remaining 11 studies used other devices, including self-developed needles. All the included studies were randomized controlled trials. The methodological details of the included studies were heterogeneous with respect to outcomes assessed, blinding, and results. Conclusions Sham acupuncture validation studies have been conducted using healthy volunteers, with a focus on blind review and technological developments in sham acupuncture devices. However, theren may be language bias in our findings since we could not query Chinese and Japanese databases due to language barriers. There is a need for more efforts toward establishing control groups suitable for various acupuncture therapy interventions. Moreover, there is a need for more rigorous sham acupuncture validation studies, which could lead to higher-quality clinical studies.


Background
Acupuncture, a widely used therapy worldwide, involves inserting needles into the body for healing purposes [1].Worldwide, numerous studies have been conducted to evaluate the efficacy of acupuncture; however, acupuncture-related clinical studies have been impeded by difficulties in designing an appropriate control group [2,3].When comparing the therapeutic efficacy of acupuncture and non-treatment controls, considering the general placebo effect and potential bias is crucial.Since the therapeutic efficacy of acupuncture is generally exaggerated, Page 2 of 18 Lim and Go BMC Complementary Medicine and Therapies (2024) 24:215 the specific effect of acupuncture remains to be established.To mitigate the problem regarding control groups, noninvasive sham acupuncture (SA) interventions, including the Streitberger's and Park sham needles, have been developed and used [4].
To facilitate the application of these noninvasive SA techniques in clinical research, relevant clinical validation studies are warranted.Accordingly, we aimed to conduct a systematic review of SA validation studies to investigate their characteristics, including participants, intervention and control group settings, and evaluation indicators.Our findings could inform the development and validation of novel and improved SA techniques.

Information sources and search engines
We performed a query of three databases (Pubmed, EMBASE, and the Cochrane Central Register of Controlled Trials) for relevant articles from inception to July 2022.We used the following search string: (acupuncture or needle) AND (sham or placebo) AND (validation or validity or validating or validate or credible or credibility).Author names were used to identify additional relevant articles.This study adheres to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement, and the research protocol has been published in a previous paper [5].

Selection criteria
To select eligible articles for this systematic review, two independent reviewers (SML and EJG) assessed the retrieved articles based on the following inclusion criteria: 1) original articles, 2) clinical trials, and 3) SA validation studies using SA control groups.We excluded studies unrelated to manual acupuncture or those testing the effects of acupuncture.In the primary title/abstractbased screening, articles considered irrelevant to the research topic were excluded.Subsequently, a secondary full-text screening was performed on articles with unclear abstracts.Disagreements were discussed until a consensus was reached.

Data extraction and risk of bias assessment
Data extraction was conducted by two independent reviewers (SML and EJG) using a predetermined data extraction form.The following data were extracted from the selected studies: 1) study design; 2) information regarding acupuncturists and participants; 3) general and treatment-related characteristics of the intervention and control groups; 4) participants' experience of acupuncture; and 5) research outcomes.
The literature quality was assessed using the Cochrane risk of bias assessment tool.The assessment items included random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection bias), incomplete outcome data (attrition bias), selective reporting (reporting bias), and other bias.Additionally, two researchers (SML and EJG) independently evaluated the literature quality, with disagreements resolved through discussion.

Data analysis
Descriptive analyses (mean, standard deviation, and frequency analysis) were conducted on the outcomes of the SA validation studies.

Search and article selection
The database query yielded 673 articles, of which 644 articles were excluded during the screening process based on title/abstract and full texts.Finally, 29 studies were included in this systematic review (Fig. 1) .

Characteristics of the selected studies
The 29 selected articles were published between 1998 and 2016.Among them, five, seven, and six studies described validation tests for the Streitberger, Park, and Takakura devices, respectively.The remaining 11 studies described validation tests for other devices, including self-developed needles.Specifically, six studies used a blunted placebo needle and a block, cylinder, or pad foam [24,26,27,30,32,33], one study used a toothpick and guide tube [25], two studies used an endermic acupuncture device with a flat, non-puncturing needle tip [28,31], one study used a blunt, noninvasive needle that comprised a diamond honing stone and a guide tube [29], and one study used a sham device designed to prevent skin penetrations of needles using a hollow inner tube with a central base channel [34].All studies were randomized controlled trials (RCTs) (Table 1).
Regarding participants, 21 studies involved healthy adults, with seven studies (including all studies that used the Takakura device) attempting to blind the acupuncturists.Among the remaining eight studies, four involved patients and four involved both healthy adults and patients.Moreover, 17 studies included both intervention and control groups, while 12 administered both acupuncture therapy (AT) and SA to the intervention group.Notably, three studies that used the Takakura device performed validation experiments on two SA types: skintouch and non-touch.
The most frequently used acupoint for SA validation was LI4, followed by BL23, TE5, and ST36.Further, 14 and 13 studies involved single and multiple acupoints, respectively.Four of the 13 studies that used multiple acupoints assessed acupoint-dependent differences in outcomes.Two studies did not mention the acupoint chosen.
Twenty studies considered the participants' acupuncture experience.Among them, 11 and nine studies recruited participants with and without acupuncture experience, respectively.The most frequently used SA validation method was guessing the applied acupuncture type (n = 21).Other SA validation methods included penetration, pain, and deqi sensation.

Reliability of acupuncturist blinding
All six studies that used the Takakura device evaluated acupuncturist blinding, with one study using a different device.These studies tested whether the acupuncturists could correctly guess the AT type after administering two (AT/SA) or three (AT/skin-touch SA/non-touch SA) different acupuncture treatments by providing a guessed (correct/incorrect) or "don't know" (DK) response.
Among the studies that used the Takakura device, incorrect and DK answers outnumbered correct answers in four [18][19][20]23] and two studies [21,22] with AT and SA treatments, respectively, suggesting that the Takakura device is effective in acupuncturist blinding.In studies that identified three AT types, non-touch SA led to more incorrect answers than skin-touch SA [20,21,23].In the study that used a different device, the rate of incorrect     ( and correct answers was higher when the needle was shown before and after treatment, respectively [28].
In the study conducted by Takakura et al. [20], participants were instructed to indicate the reason for the answer, with the most frequent reason being deqi sensation.

Reliability of participant blinding
Participant blinding was evaluated in two, five, four, and eight studies using the Streitberger, Park, Takakura, and other devices, respectively.In all these studies, the participants were instructed to answer in the same aforementioned format as the acupuncturists.Among these studies, the rate of incorrect answers was higher for AT and SA in four [14,15,17,28] and 14 [8, 11, 13, 15, 19, 21-23, 25, 27, 29, 30, 33, 34] studies, respectively.In the remaining study, most participants gave the answer 'DK' , which contributed to a low rate of correct answers for SA [9].
Two studies compared the blinding success according to the selected acupoint.Participants were more likely to correctly guess the acupuncture type when it was administered to the upper limbs (vs.lower limbs), limbs (vs.torso), and traditional acupoints (vs.non-traditional acupoints) [13,30].Chae et al. [14] measured the penetrating force using a computerized system and observed that it was associated with the blinding outcome.

Blinding Index
The blinding effect was analyzed in 24 studies, with five studies being excluded owing to failure to provide data for calculating the Blinding Index [35] (Table 2).Among these, 11 studies had blinding scenarios of "unblinded" and "opposite guess" in the experimental (AT) and control (SA) arms, respectively.Additionally, two studies had a blinding scenario of "random guess" in both arms.Accordingly, 13 of the 24 (54%) studies were considered to have applied effective blinding scenarios.Moreover, six studies were unblinded in the experimental arm (AT), and random guessing was applied in the control arm (SA), while three studies were unblinded in both arms.Furthermore, one study applied random guessing in the experimental arm (AT) and was unblinded in the control arm (SA), while another study applied random and opposite guessing in the experimental (AT) and control (SA) arms, respectively (Table 3).

Participants' responses to acupuncture-related sensations
Twenty studies evaluated participants' acupuncturerelated sensations.Among these, five, four, five, and six studies used the Streitberger, Park, Takakura, and other devices, respectively.Participants were asked to rate the acupuncture-related sensations, including pain and penetration, on a 1-10 or 1-100 visual analog scale (VAS).
Fifteen studies evaluated the participants' penetration sensation.Among these, 12 and three studies evaluated the presence/absence and level of penetration sensation, respectively.Eleven studies performed pain evaluation, of which four and seven studies evaluated the presence/ absence and level of pain, respectively.In six studies that reported the penetration sensation, most participants perceived the penetration in both AT and SA.The perception of penetration sensation was lesser in SA and AT in six [6-8, 24, 25, 32] and two [10,16] studies, respectively.In four studies, more participants reported the penetration sensation only with AT [12,18,21,30].Notably, in the studies conducted by Chae et al. [14] and Lee et al. [15], participants who received AT and SA in the LI4 acupoint reported significantly stronger penetration sensation with AT; however, no significant differences were observed in the CV12 and ST36 acupoints [15,19].
Takakura et al. [22] reported that most participants experienced pain with both AT and SA; however, the perceived pain was lesser in SA.Fink [26] showed that all participants reported pain with both AT and SA.In contrast, Kreiner et al. [30] reported that only 7.8% and 3.1% of the participants felt pain with AT and SA, respectively.Another study showed that 59.6% of the participants reported only AT-induced pain [22].Regarding the pain level, three studies reported stronger pain in AT than in SA [6,9,14].Liang et al. [16] reported that only group A (AT → wash out → SA) perceived significantly stronger pain with AT.In the remaining three studies, the pain level did not significantly differ between AT and SA [10,32,34].Moreover, responses were sought regarding the feelings of relief, pleasure, facial temperature, acceptability, and comfort.Notably, only the facial temperature measurements showed differences between AT and SA.

Participants' report on deqi sensation
Fifteen studies evaluated the participants' deqi sensation, of which three, five, four, and three studies used the Streitberger, Park, Takakura, and other devices, respectively.Notably, twelve and three studies evaluated the presence/absence and level of deqi sensation, respectively.Six studies reported greater deqi sensation with AT than with SA [6,11,12,16,18,24].Five studies reported that most patients lacked deqi sensations with AT, which was even lower with SA [10,19,21,23,30].Fink et al. [26] showed that 84.4% and 34.4% of participants reported deqi sensation with AT and SA, respectively.Chae et al. [14] reported that participants felt significantly stronger deqi sensations with AT than with SA. White et al. [7] reported no differences between the two groups.Lee et al. [15] reported some differences in deqi sensation at LI4 but no differences between the two groups at CV12 or ST36.

Quality assessment
Figure 2 presents the results of the assessment items of the overall risk of bias.In all included studies, 192 "low risk" and 11 "unclear risk" assessments were performed in seven domains.The risk of bias was low for random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection

Sham Device
bias), incomplete outcome data (attrition bias), selective reporting (reporting bias), and other bias in 27, 22, 29, 29, 28, 29, and 28 studies, respectively.Among the assessment items, allocation concealment (selection bias) had the highest frequency of "unclear risk" evaluation (n = 7) due to the lack of a specific description of the method of concealing the allocation sequence.Random sequence generation (selection bias) had the second highest frequency of "unclear risk" evaluation (n = 2) due to an unmentioned or unclear randomization method.Similar distributions were noted for the low and unclear risks of bias in studies using the Streitberger, Park, and Takakura devices.

Discussion
Invasive control groups involving needle insertion into an area other than a traditional acupuncture point or a traditional acupuncture point unrelated to the treatment objective may be unsuitable as placebo control groups since the procedure can induce physiological effects similar to invasive AT [36].Noninvasive SA needles were developed to overcome these limitations.Noninvasive SA devices, including the Streitberger, Park, and Takakura devices, are characterized by blunt needle tips that cannot penetrate the skin but have the same shape as needles used for AT, which ensures participant blinding [4].Validation studies on SA devices used across acupoints and participants are important for improving acupuncturerelated clinical research that involves SA control groups [37,38].
All included SA validation studies in this review had an RCT design involving randomly assigned intervention (AT) and control (SA) groups of healthy volunteers or patients.Blinding was influenced by the participants' acupuncture experience, acupuncturist's experience, acupoint, and type of SA (skin-touch or non-touch).A higher rate of blinding success was observed for participants without acupuncture experience, experienced acupuncturists, acupoints in body parts other than the hand, non-traditional acupoints, and skin-touch SA.Including DK as a response option may influence the results and their interpretation; therefore, this should be carefully considered.
Other aspects of blinding that were evaluated included penetration, pain, and deqi sensations.Specifically, the presence/absence and level of sensations were evaluated through yes/no responses and a VAS, respectively.Although the evaluation items for deqi varied across studies, it was mostly evaluated based on the level of sensations such as dull pain, heat, stinging, and tingling.Since AT-and SA-related sensations are important factors in studies involving patients, future studies should comprehensively consider the influence of the disease on sensations based on validation study outcomes using healthy volunteers.
In clinical studies evaluating the therapeutic effect of AT, establishing an appropriate control that allows the exclusion of the placebo effect is important, and thus, evaluation of the AT-specific effects.However, in real practice, precise assessment of the AT-specific effects is difficult owing to the multiple and complex factors that influence the AT-related experiences and expectations of patients [39].Therefore, using an SA control intervention that allows effective blinding of patients and assessment of AT-specific effects is crucial for obtaining highly reliable clinical findings [40].Meta-analyses conducted by Vickers et al. [41,42] revealed that the AT intervention group showed clinically significant outcomes compared with the SA control group, which indicates that appropriate SA controls can allow high-quality clinical evidence.Moreover, compared with noninvasive SA interventions, penetration of a real acupuncture needle can achieve a significant analgesic effect for a specific condition such as pain [43].Therefore, future SA-controlled clinical trials that use the optimal AT protocol and adequate sample size for the desired effect size could further improve evidence-based medicine.Additionally, for RCTs that include a no-intervention group, it would be helpful for validation of the SA control.
According to White et al. [8], compared to healthy participants, patients experience a stronger needle sensation for both real and sham needles and are more likely to report both as real needles.Thus, differences in sensation during AT or differences in treatment expectations between patients and healthy participants could affect the results.Consequently, generalizing the results of validation studies for sham needles in healthy adults or patients could be inappropriate.Future studies should focus on identifying the most suitable sham needles for specific diseases.
SA devices that involve skin contact or minimal insertion may pose limitations in controlled clinical studies owing to potential neurophysiological effects via skin contact or SA.Ideally, SA controls should have physical features and psychological effects identical to those of AT, which minimizes the physiological effects on the human body and maintains blinding of both participants and acupuncturists even in long-term clinical studies.Since SA validation studies are conducted using a single-or double-randomized design, establishing suitable control groups, including electroacupuncture and intradermal acupuncture, for various AT interventions is crucial to validate their therapeutic efficacy.
A limitation of this study is the possibility of language bias since we did not query Chinese and Japanese databases due to language barriers.

Conclusions
More efforts are required to establish control groups suitable for various acupuncture therapy interventions.Moreover, more rigorous sham acupuncture validation studies are necessary, potentially improving the quality of clinical studies.

Fig. 1
Fig.1 Flow chart of the trial selection process

Fig. 2
Fig. 2 Risk of bias summary

Table 1 (
continued) RCT Randomized control trial, SIS Shoulder impingement syndrome, CBP Chronic back pain, RWI Recent whiplash injury, AT Acupuncture therapy, SA Sham acupuncture, ST Skin touch, NSQ Needling sensation questionnaire, DK Don't know, Remove Removal of the needle or sham needle, Facial temperature Measurement of facial temperature using DITI (digital infrared thermal imaging), Acceptance acceptability of needle using a 5-point scale, Visceral pain stimulation pain scores and rectal balloon volumes following rectal balloon distensions

Table 2
Blinding index values computed from 24 validation studies VBI blinding index of real acupuncture group, SBI blinding index of sham acupuncture group