Intra and Inter Examiner Reliability Study for the Characteristics of Evaluation of the SA201

Introduction: There is no published account of the reliability of the percussion technology on human participants. Our goal was to measure the reliability of the instrument’s analysis protocol on an inert substance and on human participants with expert and novice evaluators. Method: 15 participants were evaluated by six evaluators, three experienced and three novices. Results: No participants were excluded from the analysis, based on the NDI, ODI and BMI. Even if the global result from the durometer testing were surprising in their range, the effect size score for the pre and post value on the durometer indicate very good reliability. Conclusion: The intra-evaluator reliability is very good when testing on the durometer but less predominant when evaluating the participant. In addition a novice chiropractor can use the instrument with a certain degree of ability that will develop over time if he uses this technology on a regular basis.


Introduction
The introduction of tools to assist chiropractors in the treatment of spinal dysfunctions started in 1910 with radiology and was followed in 1913 with the first pneumatic percussion [1]. The ancestor of contemporary instrumentation utilization in chiropractic was born. The SA201 (Sigma Instruments Inc. (Crawnberry Township, Pennsylvania, USA) is homologated in Canada, by Health Canada.
The SA201 has an analysis and a treatment function. In this study only the analysis portion was analysed. The SA201 technology is based on tissue resonance properties that stipulate that tissue vibration is a scientific measurable entity [2][3][4][5][6]. The piezoelectric sensor of the SA201 device relays the analog data to be transformed into digital data that is presented onto a computer screen.
In the literature the piezoelectric sensor has been utilized as a noninvasive tool to study bone dynamic movement [7] (Figures 1 and 2).
The wavelength corresponds to the frequency of the area under investigation. In the SA201 the ideal frequency for the spine is considered to be between 45 and 55 Hz with an average around 48 Hz [2] (Figure 3).
The height of the wavelength corresponds to the resistance to movement in the area evaluated. The ideal resistance based on a duromètre 40 [3] would be situated between 15 and 25.
Finally, Sigma Inc. concluded from their in house studies [2,3] that; the precision of the analysis of the instrument was at 99.62% reproducible, there was an evaluator error of 3.3% and that repeat factor created an augmentation of the mobility and once this factor was corrected the inter-evaluator precision was 94.8%.
The literature review on clinical studies in relation to the SA201 is limited to three published studies and the literature on the analysis protocol of the SA201 is limited to the in house studies from the manufacturer [8][9][10]. There is no published account of the reliability of this technology on an inert substance like the durometer 40 or on human participants. Our goal was to measure the reliability of the instrument's analysis protocol on an inert substance and on human participants with expert and novice evaluators. 15 adult participants were selected. They received an information session where it was explained what was expected of them before and during the experimentation. A research consent form was explained and signed by each participant. The sample size was based on previous studies from the literature [11,12].

Participants
Inclusion criteria were: adult between the age of 18 and 65, in good health. Exclusion criteria were: a score above 40% on the Oswestry disability index (ODI) and a score above 48% on the neck disability index (NDI) on the day of the experimentation; presentation of asymmetric range of motion of more than 10 degrees or uncomfortable pain either in the cervical or lumbar region on the day of the experimentation and finally a body mass index (BMI) superior to 40. All participants were evaluated by a licensed chiropractor.
first test at C-1 which was at 90 degrees and the second test at 45 degrees like the rest of the spine. The head was held according to the teaching principle from the manufacturer. The thumb and the major finger are on the side of the fork shape head and the index finger is between the two branches of the fork and the head is slid from one segment to the next. In order to obtain a reading a progressive and sustained force up to six pounds; preset by the manufacturer; is produce to produce a release of the head of the instrument producing recoil toward the sensor inside the instrument. This results in the recording of the data for that segment. That procedure is repeated for every segment. We evaluated 30 sites as it is recommended by the manufacturer from C-1 to S-5. The measures were recorded twice on each participant by each evaluator. The sequence of the analysis and the rest period in between each analysis is presented on Table 1.

Evaluators
We had six evaluators, divided in 2 groups (n=3 per group). The first group was with experienced chiropractors (E) having respectively 10, 3 and 1 years of experience with the instrument. A novice (N) group composed of two chiropractors with 20 and 3 years of clinical experience but no experience with instrumentation and a nonprofessional person with no clinical training or experience in palpation or instrumentation. They received an information session where it was explained what was expected of them before and during the experimentation. The researchers were not aware of the clinical experience difference in the status of the evaluators.
The variables evaluated are frequency and resistance. The compilation of the data was done in the Excel software.
Training of the N group, the day before the experimentation the three novice evaluators received a four hour training session. This training session is the manufacturer recommended introduction session to novice acquiring the instrument. The evaluators were introduced to the theory and had a hands-on practice session.
• During the practice session, the following was covered: • Patient's positioning Clinician's positioning • How to hold and stabilize the instrument with the 30 mm forked head • One exercise to experiment the pre-tension force Palpating the anatomical reference points • One exercise feeling the instrument and maintaining the approximate 45 degree angle without displacing the instrument   • One exercise feeling the instrument displacing the instrument and maintaining the approximate 45 degree angle • One exercise feeling the instrument displacing the instrument and maintaining the approximate 45 degree angle and evaluating the cervical, thoracic and sacrolumbar areas on each other.
The site of the experimentation was a private clinic.
Two communicating adjacent room with sharing one computer and instrument namely « room #1 » « room #2 » were used; the waiting room could sit comfortably fifteen persons and was available for the participants, to relax in between their evaluation sessions.
An examination room was available, where the participants could change into a gown and the technician could prepare the participants, for their evaluation session.
We had a site supervisor, to verify that the protocol was followed as erected and supervise the technician. The site supervisor was in the room whenever there was an evaluation and was in control of the software and made certain no one could see the screen as it was recording the data from the analysis. There was no comment made to the evaluator or participant during the procedure. He verified the exactness of the anatomical reference points, the flow of the participants and of the evaluator, respecting the establish time schedule (Tables 1-3). After the experimentation he transferred all the data to excel using a specially develop database query software (Chirosoft Inc., Quebec, QC, Canada).
The technician was responsible for the recruitment of the participants and of the evaluators, the training of the novice group, take care of participants and evaluators while they were on the premise, responsible to obtain the declared consent forms signed for the research project, evaluating the questionnaires (ODI, NDI and BMI) associated with the inclusion/exclusions criteria, to verify the symmetry of the cervical and lumbar ROM, execute the anatomical marking of the reference points and to direct the traffic flow during the experimentation.
Each participant was on site for a period of 90 minutes and each evaluator was on site for six hours with a one hour lunch break.
The SA201 was onsite and used with the same 30 mm forked head that the N group had practiced with.
The analysis was done in two steps. The first part consisted in the evaluator doing two complete analyses on a durometer 40 of a ¼ inch thickness which was resting on a metal base. Then he would proceed to do the same analysis on the patient. The analysis consisted on testing the cervical, dorsal and sacrolumbar areas representing 30 data per analysis. The analysis was done twice with a few seconds pause in between the analysis, and was therefore considered before and after analysis without any intervention.
The pre-tension force of the instrument was at six pounds and it was factory calibrated. The participants were seated on a sitting massage chair with their head resting on the head piece with their neck in the horizontal position.

Statistics
We measured averages with standard deviation and we completed with an ANOVA, Cohen's D and the Student T test [17]. Our analysis schematic is presented in Table 4.

Results
Our results are presented as tables.
There were no participants excluded from the analysis based on the NDI, ODI and BMI (Tables 5 and 6).

Discussion
In Tables 7 and 8    The global result from the durometer testing were within range of the manufacturer's recommendation and the effect size score for the pre and post value on the durometer indicate good reliability.
Pondering on these result for the minimal sample size, we found out that E1; while certified as an instructor in this technology, does not or rarely use the analysis function of this instrument in his practice. Thus, the adage if you do not use it you lose it. This technology is a technique and it requires constant practice. Lack of practice was reflected quite clearly in this instance. The other evaluators (E2, E3, N1, N2, and N3)

Participants # NDI (%) Oswestry (%)
Range 0 to 36 0 to 20 Average and SD 11,60 ± 10.56 7,07 ± 6,71   sample size estimation results are well under the 900 recorded and we can put more value on their results. N3 is not a health care professional and does not have any training, we consider his higher sample size estimation normal, but still within range and we kept his value in the pool we analyzed; because some neophytes may not have the same skill set as a regular practionner in the clinical field.
In Table 10, we look at the aggregate results of all the examiners. The outcome of our evaluators is 44.63 Hz ±2.51 Hz. We are at 0.82% of the inferiorly predicted value from the manufacturer. However when we look in Table 11 for the resistance values; our evaluators present a value of 22.43 ± 5.26. This is the suggested range.
The same patient receiving repeated six pound pressures on the same points of his paraspinal tissues from an instrument, this was done by six different clinicians within sixty minutes and it can become a source of variation of our results (Sigma 1998 c). This was not present on the durometer testing which is an inert substance and it is resting on a metal plate. Thus, it could possibly affect our conclusion on reliability and validity on human participants.
In Tables 12 and 13 we calculated the effect, respectively of the frequency and the resistance, of the pre and post measurement on patients. We can refer to Table 14 for an interpretation of the values obtained. When we compare E2 vs E3 we have a value of 0.16 for the frequency and 0.20 for the resistance, indicating us that there is not much difference between these two evaluators. When comparing E1 to either E2 or E3, E1 data become inconclusive and we have explained previously why and we demonstrate that his behavior is more like an evaluator of the N grouping.
Our expectations were to find Cohen's D score to demonstrate little effect within the E and N grouping respectively. N3 presented a score which makes him an outlier. We were not surprised by this result.     The human factor seems to be the great equalizer when we approach techniques of evaluation; we have Haneline et al. [18], report that the intra and inter evaluator reliability is low in static palpation.  24] demonstrated that a standardized method for palpation increased the reliability of this technique significantly. This technology has certainly brought a standardized set of evaluation with the controlled force to evaluate the spine and the training syllabus seems to have been sufficient for novice to perform to a certain standard reflected by the results.
Humphreys et al. [25] demonstrated that it was possible for an inexperienced evaluator to identify inter-segmental fixations in the cervical spine. Thus, demonstrating that our N# data could be a reflection of a natural tendency for a novice with a lack of experience.
Our results indicate a large inter evaluator variance, results similar to what was previously published in the literature [4], when done on humans. It is however probable that the continuous solicitation of the same paravertebral structure creates an adaptive change over time [5]. We did not see such a variation on the durometer. Thus, reinforcing the point, that the paraspinal tissues can be influence; by the examination itself. The clinician has to be careful in his first analysis as well as his re-evaluation after the treatment.

Limitations
The angle of the head could be an intrinsic factor creating certain variability when testing on a patient. It was not really a factor on the durometer. The application of force to reach the six pound of pretension can be another factor; if the force was applied briskly instead of progressively can account for a certain variability and inconsistency of the results with patients. We noticed that it is not the case on the durometer.

Conclusion
The intra-evaluator reliability is very good when testing on the durometer but less predominant when evaluating the participant. In addition a novice chiropractor can use the instrument with a certain degree of ability that will develop over time if he uses this technology on a regular basis.