Design and content validation of the Oral Hygiene Habits Scale .

Cite as: Rodríguez NI & Moral J. Design and content validation of the Oral Hygiene Habits Scale. J Oral Res 2016; 5(4): 159-167. Abstract: The presence of dental plaque is associated with deterioration of oral health, tooth decay, gingivitis and periodontal disease. The study of content validity of a new instrument aims to determine whether the items included in the measuring instrument are representative of the assessed construct. The objective of this research was to design and validate the content of an oral hygiene habits scale. A Likert-like scale, named Oral Hygiene Habits Scale (OHHS), containing 10 items and five response options on frequency of behavior, was developed to assess the dimensions of toothbrushing and f lossing. Six experts were recruited for the study. They were asked to conduct their evaluations individually. Four characteristics (adequacy, clarity, consistency and relevance) were evaluated for each item, using an assessment scale of four ordinal values (from 1="does not meet the criteria" to 4="high level of compliance"). Data were analyzed using the binomial test, Kendall’s W and Fleiss’s Kappa coefficient. The judges mostly indicated a high level of compliance with the characteristics evaluated for each item and their judgments were consistent with one another. It is concluded that the OHHS composed of 10 items showed content validity. Further studies are suggested to determine its reliability and construct validity.


INTRODUCTION.
According to the World Health Organization 1 the promotion of oral health as a strategy to reduce disease and maintain oral health consists of a number of elements, among which we find access to drinking water, general hygiene and proper oral hygiene.The cleaning of the oral cavity by toothbrushing is effective for removing dental plaque 2 .The presence of plaque is associated with deterioration of oral health, dental cavities 3 , gingivitis 4 and periodontal disease 5 , hence the importance to maintain the oral cavity free of plaque.
Dental cavities, one of the main oral health problems, vary with age, sex, socioeconomic status, diet and oral hygiene habits.Its severity is characterized by pain and difficulty during mastication 6 .In addition, the loss of teeth, functional and aesthetic problems, and cavities in need of treatment, negatively affect the quality of life in relation to oral health because of the intense pain and psychological distress.They also affect daily activities and labor productivity 7 .
The American Dental Association (ADA) 8 establishes as measures of daily oral hygiene: toothbrushing twice a day for two minutes with fluoride toothpaste, using a soft brush, and interdental cleaning once a day.It also recommends the use of mouthwash as part of routine hygiene habits, and changing toothbrush every four months or sooner 8,9 .The effectiveness of dental cleaning correlates with brushing time and not with the use of toothpaste [10][11][12] .
There are questionnaires for the assessment of oral hygiene applied to research in prevention programs, but these applied studies do not provide evidence of the reliability of the questionnaire regarding its internal consistency, temporal stability or interrater reliability, and its validity; its content, construct or criterion [16][17][18] .The reliability of an instrument refers to the degree to which its scores are stable, either when considering various items (internal consistency), evaluation times (temporal) or evaluators (interrater) 19 .The validity of a measuring instrument refers to degree to which the items covering the content of the construct (content), the evidence supporting the theory and the interpretation of test results (construct) yield scores that are highly correlated with other instruments that assess the same construct (criterion) 20,21 .
In developing a measurement tool, it is first necessary to define the construct being evaluated in terms of indicators and dimensions, that is, to operationally define its content 22 .
The concept of content validity helps to determine whether the items of a measuring instrument are representative of the assessed construct 23 .A strategy to establish content validity is the evaluation by a panel of expert judges, defined as the opinion of recognized people in the field who evaluate whether the content is relevant and appropriate, through criteria developed by the researcher 23 .
The objective of this research was to design and validate the contents of an oral hygiene habits scale developed from elements recommended by the ADA 8 .

Participants
The process of developing the Oral Hygiene Habits scale was conducted by two professors, researchers in the areas of Periodontology and Psychology, at the Faculty of Dentistry and Psychology at Universidad Autónoma de Nuevo León.
Items and the possible responses to each of the questions were developed from criteria established by the ADA, and from a literature review and information obtained on oral hygiene habits from patients who had already received periodontal treatment.
Subsequently, items to the defined dimensions were created.Simple interrogative sentences were preferred, using simple and direct language and avoiding juxtaposed, disjunctive, complex and compound sentences, double negatives, technicalities, localized expressions and regionalisms.Inconsistency with the construct of interest was considered as a criterion for modifying or removing items.
The scale was initially designed including four dimensions: toothbrushing, flossing, chemical control of dental plaque and professional dental care.However, the dimension of chemical control of dental plaque was removed, as it was considered an indirect measure of oral care.The dimension of professional care was also removed because it was considered irrelevant or unsuitable to represent the construct.Therefore, the scale was finally defined with only two dimensions: toothbrushing and flossing.Once designed, the scale was evaluated by the experts individually.

Content validity was established through expert opinion.
The panel of judges was composed of a total of six experts, four experts in the assessed construct and two experts in measurement techniques.The experts worked in the areas of dentistry and psychology, respectively.Four characteristics that each item should meet as an indication of their corresponding dimension were considered: adequacy (items that belong to the same dimension are sufficient to obtain the measurement of it), clarity (the wording is understandable, even for a person with low education, i.e., syntactic and semantic structures are appropriate), consistency (the item has a logical relationship with the dimension or indicator it is measuring) and relevance (the item is essential or important, i.e., it must be included).The experts were asked to judge compliance with these characteristics for each item in its dimension.Information by each of the judges was Design and content validation of the Oral Hygiene Habits Scale.

Rodríguez NI & Moral J.
gathered.Afterwards, data analysis was performed taking into account the evaluation provided by each one of experts.

Instruments
The format of expert judgment proposed by Escobar and Cuervo was used as an assessment tool 22 .This format includes the categories of adequacy, clarity, consistency and relevance, under which each item must be evaluated by each judge.Evaluation was made following an ordinal scale of four categories: 1="does not meet the criteria", 2="low compliance", 3="moderate compliance" and 4="high level of compliance".An open question was included for each item to suggest a modification or replacement in case of a low level of compliance.In addition, they had the option of suggesting additional items and dimensions.

Procedure
Six experts were selected from the Faculties of Dentistry and Psychology at Universidad Autónoma de Nuevo León.It was explained to each one of the experts that the usefulness of the scale was related to diagnostic and epidemiological research purposes.The evaluation form containing the ten items, theoretical definitions of the construct and the four characteristics to assess in each item, was also explained to the experts.

Data analysis
The distribution of the four characteristics evaluated for each item was described using a frequency table.The criterion for keeping an item unmodified was that five of the six judges evaluated it in the category "high level of compliance" and that none of them indicated that the item "does not meet the criterion" in each of the four characteristics evaluated.A percentage above 80% was used; following the high reliability criteria for interrater reliability indices 24,25 .The criterion to remove an item was that four of the six experts coincided in evaluating it within the category "does not met the criteria" at least in one of the four characteristics evaluated.This criterion of two-thirds of inadequacy was taken from a previous study conducted on the development of an instrument.
In other cases, the item had to be modified following the suggestions of the experts 26 .
The hypothesis that the proportion of high-level compliance in each characteristic was larger than or equal to .8 by the binomial test was contrasted, using the exact probability because the sample size was small (six judges).The consistency of the judgments was contrasted evaluating the four characteristics in the 10 items and rating the level of the characteristic within each item by Kendall's W test in the case of tied observations.Finally, the agreement between the judges was determined by Fleiss's Kappa coefficient 24 with a change in the estimate of the expected probability by chance, due to the concentration of most cases in the latter category, as expected.These distributions with a very clear negative skewness and peakedness made calculating this probability from the observed values inappropriate 25  It should be noted that Kendall's W is the ratio between the variance of the midranges and its maximum value in case that k judges (six judges) homogeneously assign each characteristic to a range, and that the distribution of these ranges should correspond to an arithmetic progression from the minimum to the maximum range (1 to 4).In case that maximum ranges are expected for each characteristic, then the null variance between ranges and equivalence of the midrange reflects the agreement between the judges, that is, that the null hypothesis (H 0 : W=0) is maintained 25 .

Design of the measuring instrument
From the elements recommended by the ADA 8 ten items were defined and grouped in two dimensions (Table 1): brushing (6 items) and flossing (4 items).A dimension of mouthwash was not included for two reasons: it is not a basic, but a complementary recommendation 8,12 , and to prevent that lowincome people get a low score as a result of not being able to follow this complementary or additional recommendation.

Content validity
The six judges mostly indicated a high level of compliance (83% to 100%), in assessing the 10 items in the characteristics of adequacy, clarity, consistency and relevance, using an ordinal scale of four levels.No item was rated with the ordinal category 1 (Table 2).
In the four evaluated properties of the 10 items the null hypothesis of high level of compliance (H 0 :p[x=4] ≥.80) remained, using the binomial test with a unilateral contrast because of the exact probability (see Table 3).Therefore, the 10 items have good for the evaluation of the construct according to the six judges and can be kept unmodified.The null hypothesis of equality of midranges among the six judges in each of the four attributes when assessing the 10 items, using Kendall's W test with a tailed test the exact probability (Table 4) was maintained.The null hypothesis of equality of midranges among the six judges in adequacy, clarity, consistency and relevance within each item was also maintained, using Kendall's W test with a tailed test for the exact probability (Table 5).
Fleiss's Kappa coefficient showed a high level of agreement (k=.778,95%CI, .774,.782).It should be noted that the probability expected by chance would be very high if calculated from observed values (p e =.844) due to the concentration of cases in the latter category, the value of Fleiss's Kappa resulting negative (k=-.068) (see Table 6).

DISCUSSION.
According to the six experts, the 10 items showed the characteristics of adequacy, clarity, consistency and relevance.Their judgments were consistent in evaluating these four characteristics in the items as well as to qualifying the level within each item.Giving an equivalent value to each level of response in calculating the expected probability by chance (modification), the agreement between the judges was high from a modified Fleiss's formula.According to Escobar and Cuervo's criteria 22 , it can be considered that the items of the two dimensions represent the content of the construct in a high degree, i.e., they are representative of the assessed construct.
The indicators included in this scale take into account the frequency of brushing, the use of fluoride toothpaste, time spent while brushing, interdental cleaning and the elapsed time between the changing of toothbrushes.In addition, the indicator of the use of toothpaste during brushing indicated that when performed adequately, brushing removes plaque with or without using toothpaste 11 .Regarding Slots's findings 14 on the failure to maintain oral hygiene in the long run and the short-term effect of educational interventions and to avoid an effect of acquiescence or compliance with the style of writing 21 , a reverse item was designed with respect to the frequency of occurrence of the failure to brush teeth in any given day.Often, inverse items have less internal consistency that direct items due to confusion in their interpretation and reading errors 27 .As a consequence of this, it is common that in unidimensional scales composed of the same number of direct and reverse items two factors are defined as a result of the lower internal consistency and lower average of reverse items with respect to direct items 28,29 .
In this scale there is only one reverse item.To avoid misinterpretations and reading errors negative elements were capitalized, focusing readers' attention towards the negative form of the sentence.It was also placed at the beginning to prevent the possible distraction caused by the previous sequence of affirmative sentences.Because of these precautions, internal consistency is not expected to be much lower than in the remaining items, or that it may result in a poor indicator of the frequency of brushing, affecting the internal consistency of the scale.
As a limitation of the study it should be noted the use of a non-probabilistic intentional sampling in the selection of judges, as it is usual in this type of content validity studies, since a probability sampling of experts is unworkable 22 .It is assumed that the panel of judges was representative of the universe of experts on the subject.In turn, sample size was small, but within the recommended range 22 , so nonparametric statistics was used.
In this paper, Delphi methodology, which is another option when the object of study is the consensus among experts on conceptual issues, was not used.This methodology is common in research on strategic and policy planning and it is being implemented in the field of health studies 30 .It consists in bringing consensus within a panel of experts following an iterative process that is repeated until a consensus is reached, or it is concluded that consensus is impossible and the reasons are known 31 .Future studies could use this methodology to study the content validity of this instrument.
The study of this scale with two samples is suggested, one of dental clinic population and other of general population, to determine the psychometric properties of internal consistency, stability, criterion validity, construct validity, describe their distribution and establish standards of interpretation.As a criterion for estimating the validity the Simplified Oral Hygiene Index (OHI-S) could be used 32 .

CONCLUSION.
The 10 items of the Oral Hygiene Habits scale show content validity and are appropriate to assess the construct by expert judgment.Resumen: La presencia de placa bacteriana se asocia con el deterioro de la salud oral, aparición de caries, gingivitis y enfermedad periodontal.El estudio de la validez de contenido de un instrumento nuevo, se centra en que los ítems del instrumento de medición sean representativos del constructo evaluado.El objetivo de esta investigación fue diseñar y validar el contenido de una escala de hábitos de higiene bucal.Se creó una escala de 10 ítems tipo Likert con cinco opciones de respuesta sobre frecuencia de conducta para evaluar las dimensiones de

REFERENCES.
Design and content validation of the Oral Hygiene Habits Scale.
. The modification was to give a value equivalent to each level of response (1/4=.25) in the calculation of the expected probability by chance.It was interpreted that the coefficient of Kappa value <.20 shows very low reliability, from .20 to .39 low, from .40 to .59 moderate, from .60 to .79 high, and ≥.80 very high 25 .

Table 2 .
Classification of the 10 items in four characteristics.

Table 4 .
Agreement between the six judges on judgments of adequacy, clarity, consistency and relevance for the 10 items contrasted by Kendall's W test.

Table 5 .
Agreement between the six judges on judgments of adequacy, clarity, consistency and relevance within each item contrasted by Kendall's W test.

Table 6 .
Fleiss's kappa for the use of the three ordinal categories in the 10 items between the six judges.and content validation of the Oral Hygiene Habits Scale.