PDF
Citation
Print
INTRODUCTION
Orthodontic treatment is performed mainly in adolescent patients, but economic development and new information have popularized this treatment in middle-aged adults by promoting interest in esthetics, appearance, and function.1-5 According to the 2012 statistics report of the American Association of Orthodontists, 12 million United States adults underwent this treatment, which represented an increase of 39% from the corresponding number in 1996.6 In Korea, adult patients are showing greater interest in orthodontic treatment1 with increases in their esthetic and functional needs. Baik et al.7 reported an increasing trend in the number of adult orthodontic patients aged over 19 years during a 10-year period after 1984. Im et al.8 evaluated the distribution of these patients and reported that the proportion of patients ≥ 19 years old and ≥ 25 years old increased from 34% and 4% in 1992 to 38% and 14.2%, respectively, in 2002.
Despite the increase in adults undergoing orthodontic treatment, only 41.8% of the survey respondents expressed the desire to undergo orthodontic treatment in the evaluation of their subjective needs,9 indicating that the participation rate of adult patients is still low. Some studies focused on factors that may influence the decision to undergo treatment in adults and reported that adults make their treatment decisions on the basis of their perception of the treatment rather than an expert's opinion.2,10-12 Lee2 reported that, in comparison with adolescents, adults tended rely more on their willingness and judgment while taking decisions to undergo treatment. Wedrychowska-Szulc and Syryńska12 reported that although decisions to undergo orthodontic treatment are influenced by the opinions of dentists and parents, these influences tend to decrease with increasing patient age. Bourne et al.13 reported that in comparison with specialists, patients expressed moderate or low needs or perceptions regarding orthodontic treatment. Therefore, it is essential to assess the perceptions of orthodontic treatment in adults.
Studies on the perceptions of orthodontic treatment in adults have been conducted, but these studies had many limitations. Lim et al.14 confirmed a change in the perception among orthodontic patients at Seoul National University Dental Hospital over the last decade. However, their study14 included patients who visited the hospital for orthodontic treatment, and they reported the patient’s preference for treatment methods instead of their treatment decisions. Since other studies focused on patient satisfaction with orthodontic treatment,1 knowledge and attitude,15 and the perception of this treatment among adolescents,16,17 these studies had limitations in evaluating the perception in adults.
Korea is one of the fastest aging countries according to Statistics Korea and is expected to become an elderly society, with more than 20% of the population aged over 65 years, by 2026.18 As the age of the population increases, information on changes in the perception of orthodontic treatment for adults is needed to promote healthy oral status by adequate treatment among adults. However, there is no proper tool to measure the perception of this treatment in adults. Therefore, verification of the content validity and reliability of an instrument should precede its use as a measurement tool. For this reason, this study aimed to develop a standardized instrument to measure the level of cognition for orthodontic treatment in adults and verify its reliability and validity for assessing perceptions of orthodontic treatment in adults.
MATERIALS AND METHODS
Study design
The entire procedure met the ethical standards issued by the Honam University Institutional Review Board (No. 1041223-201504-HR-039-02). This study aimed to develop a measurement instrument that could assess the perceptions of orthodontic treatment in adults over 19 years of age and to verify the reliability and validity of this instrument.
The study was designed according to the eight-step instrument-development guidelines proposed by DeVellis,19 which consists of an instrument development stage (instrument component design, initial item development, selection of the response format, content validity test, and item review) and an instrument verification stage (instrument application, evaluation of the instrument, and optimization of the instrument) (Figure 1).
Study subjects and data collection
Preliminary instrument development
To verify the content validity, a professional group consisting of 10 members, including one professor with experience in instrument development, six professors in orthodontics, and three professors in dental hygiene, were selected. All initial items were measured on a four-point Likert scale as follows: 1, not at all; 2, not like that; 3, true; and 4, very true.
Requests were made for comments on the items that were difficult to understand and needed modification. The content validity test was conducted by experts in two stages from September 1 to September 27, 2015. Face validity was verified on March 4, 2016, in a sample of five adults aged 19–48 years. For the item review, a preliminary survey of 30 male and female college students aged ≥ 19 years was conducted from March 20 to March 30, 2016 at Chonnam National University and Honam University.
Verification of the instrument
For data collection, an online survey was conducted by an internet research system of a specialized research company (Macromill Embrain, Seoul, Korea) from April 20 to May 6, 2016. To protect the rights of all subjects, before starting the online survey, the participants were provided detailed information on the purpose and method of the study, the benefits and risks associated with participation in the study, privacy- and confidentiality-related information, and voluntary consent to participate and withdraw from the study without negative consequences. Subsequently, the survey was conducted only if the subjects agreed to participate in the study.
To increase the representativeness of the sample, quota sampling was conducted by region and age. First, Korea was classified into four major areas as follows: the Capital area (Seoul, Gyeonggi, and Incheon), the Central area (Gangwon, Daejeon, Chungnam, and Chungbuk), the Southeast area (Busan, Ulsan, Daegu, Gyeongbuk, and Gyeongnam), and the Southwest area (Gwangju, Jeonnam, Jeonbuk, and Jeju). Subjects with no orthodontic treatment experience were selected based on the ratio of the adult populations between 19 and 64 years of age in 2016 and the estimated proportions of the populations in each age group.18 Exploratory factor analysis (EFA), which is a method of increasing the reliability of data by removing questions that do not measure a concept well, and confirmatory factor analysis (CFA), which is a method of verifying the validity of the EFA model, tend to overestimate construct validity and model fit when implemented on the same object.20 Thus, EFA and CFA were performed for different subjects to increase the generalizability of the instrument in this study. Previous studies showed that EFA requires at least 150–200 samples20 and for CFA, the minimum sample size appropriate for the parameter estimation process by the maximum-likelihood estimation method was generally 100–150, with an appropriate level of 200.21 In our study, a total of 420 participants were considered after accounting for potential dropouts, and among these, the data of 406 participants were included in the analysis. The data of 14 participants who did not provide sufficient responses were not used in the analysis. By random classification, the data of 206 participants were used for item analysis and EFA, and the data of the remaining 200 participants were used for CFA.
Instrument development process
Preliminary instrument development
1) Instrument component design
To clarify information for the perceptions of orthodontic treatment in adults, a literature search was conducted in a database of international papers (PubMed) from January 2000 to June 2015 using the search terms orthodontic, orthodontic treatment, dental/teeth, and awareness/knowledge/attitude/perception/need/effect. In addition, for domestic papers, a search was conducted in DataBase Periodical Information Academic (DBpedia), the Korean Studies Information Service System (KISS), the National Digital Science Library (NDSL), and the Research Information Sharing Service (RISS) using the same search terms. A total of 1,448 papers were searched and in each paper, the abstract was checked. Subsequently, 1,336 papers that were duplicates or not related were excluded. The remaining 112 articles were screened, and 29 papers dealing with the perception of orthodontic treatment were finally selected and included in the subsequent analyses.
Based on previous studies, this study defined the perception of orthodontic treatment as negative or positive perceptions related to thought, feeling, prejudice, and favorable impressions derived from the subject’s understanding of orthodontic treatment. The final component of the recognition of orthodontic treatment included four factors: general perception of orthodontic treatment (the effects of orthodontic treatment on psychological, aesthetic, and oral health), the perception of orthodontic treatment for adults, the effect of orthodontic treatment, and the retention of orthodontic treatment. Considering the four factors as a basic framework, the corresponding questions for each factor were constructed.
2) Initial item development
The first 35 preliminary items were derived. The items were reviewed by consultation with experts (two professors with experience in instrument development and one professor in orthodontics) to determine whether each component was appropriate, the appropriate item was configured to the component, and that the items did not have duplicate meanings. Fifteen items were deleted, resulting in 20 preliminary questions, including five questions each about the effect, retention, and general perception of orthodontic treatment, and the perception of orthodontic treatment for adults.
3) Selection of the response format
On the basis of the results of a previous study22 that evaluated the effect of the number of response categories on the reliability and validity of a rating scale and reported that the optimal number of alternatives was between 4 and 7, the scale was considered from 4 to 7 points. Because many responses can confuse the respondents and the even-numbered Likert scale shows a greater likelihood for identifying tendencies,17 a 4-point Likert scale was selected. The responses on this scale were rated as 1 point for “not at all”, 2 points for “not like that”, 3 points for “true”, and 4 points for “very true.”
4) Content validity test
To validate the 20-item preliminary instrument, experts performed two content validity tests and one face validity test. The content validity test was verified on the basis of two criteria, an item-level content validity index (I-CVI) and the scale-level content validity index/averaging (S-CVI/Ave), to confirm that the universe of content provides a basis for developing items that will adequately represent the content.23,24 The I-CVI was calculated as the ratio of experts who responded with 3 (true) or 4 (very true) points for each item, and a result of 0.78 or higher meant that each item is valid. The S-CVI/Ave was calculated as the ratio of items answered with 3 or 4 points by the expert, and a result of 0.90 or higher meant that the scale-level is valid. Through the first validity test by the experts, 16 items were selected by deleting four items on the basis of the criteria and modifying the expression of nine items from responses that they were difficult to understand or needed revisions to sentence vocabulary. In the second validity test by the experts, three other items were deleted. For the remaining 13 items, the I-CVI was 1.00 and the S-CVI was 0.94 to 0.97, which ensured the content validity of the 13 items by the experts. A face validity test was conducted to confirm the understanding, difficulty, and accuracy of each item by the subjects to whom the instrument was applied. In this test, the I-CVI of all items was more than 0.80. Through these steps, seven of the 20 initial items were removed, leaving 13 items in a preliminary instrument.
5) Item review
To verify the reliability of the preliminary instrument, a preliminary survey of 30 male and female college students over 19 years of age was conducted and Cronbach's α, which indicates the reliability of internal consistency, was obtained. Cronbach's α was 0.805, which is higher than the internal consistency reliability standard of 0.70. Thus, each item within the scale reflected the same concept. For the factors, Cronbach's α was 0.731 for the general perception of orthodontic treatment, 0.700 for the perception of orthodontic treatment for adults, 0.731 for the effect, and 0.640 for the retention of treatment.
Verification of the instrument
The collected data were analyzed using IBM SPSS ver. 23.0 (IBM Corp., Armonk, NY, USA) and AMOS ver. 21.0 (IBM Corp.) statistical programs.
1) Item analysis and construct validity
The general characteristics and homogeneity of the subjects were analyzed by descriptive statistics and chi-squared analysis. Item analysis with correlation coefficients > 0.30 was used to analyze the correlation between the total scores and each item.
Construct validity was tested using EFA and CFA. To confirm the fit of the EFA, the Kaiser-Mayer-Olkin (KMO) test and Bartlett’s test were performed. The KMO test indicates the suitability of samples for factor analysis and is judged to be appropriate if it is 0.70 or more. Bartlett's test indicates the suitability of the factor analysis model and indicates that the use of factor analysis is appropriate if it is statistically significant (p < 0.05). Where fit, principal component factor analysis using Varimax rotation was performed based on an Eigenvalue representing an explanatory power of the factor of ≥ 1.0, communality and factor loading of ≥ 0.40, and an accumulative variance of 60% or above.25 Principal component factor analysis, which is widely used to minimize the loss of information and extract a small number of factors that explain the relationship between the items as much as possible, was used to reduce the number of variables, and Varimax rotation was used to obtain the meaningful factors by simplifying the factor structure.
After determining the number of factors, the model fit was verified using goodness-of-fit indices, such as χ2 and df, and the root mean square error of approximation (RMSEA). Convergent validity, which confirms whether the item explains the same concept, was tested based on the criteria of β > 0.50, an average variance extracted (AVE) value of > 0.50, and construct reliability (CR) of > 0.70. Moreover, two criteria, AVE > r2 and r ± 2 × standard error (SE) ≠ 1, were checked to test discriminant validity using a low correlation between the factors.25
2) Reliability
Evaluation of the internal consistency reliability of the instrument was based on a Cronbach’s α of > 0.70, calculated for each factor and the total score.25
3) Optimization of the instrument
To optimize the validated instrument, vocabulary and readability were reviewed again through consultations with the experts. The final measurement instrument for perception of orthodontic treatment consisted of four factors with a total of 11 items, which were classified into three items for the general perception of orthodontic treatment, four items for the perception of orthodontic treatment for adults, two items for the effect of treatment, and two items for the retention of orthodontic treatment.
RESULTS
General characteristics
The general characteristics of the 406 subjects included in the study are shown in Table 1. The general characteristics of the EFA (n = 206) and CFA data (n = 200) were not significantly different (p > 0.05).
Item analysis
The mean and standard deviation for each item were analyzed on the basis of the EFA data. The results for skewness (the degree of distortion from the normal distribution) and kurtosis (the actual measure of outliers present in the distribution) based on a skewness of > 2.00 and a kurtosis of > 7.00 to determine the shape of the distribution, revealed that the skewness was within 0.87 of the absolute value in all items and kurtosis was within 3.06 in all items. Thus, all items had confirmed normality. In addition, in the correlation (corrected item-total correlation coefficient) between the individual items and the total items, no items showed correlation less than 0.30, indicating the absence of low internal consistency (no correlations among items)25 (Table 2).
Exploratory factor analysis
EFA, a multivariate analysis method, uses the correlation between variables to reduce the data measured by multiple variables to a smaller number of factors. This type of analysis provides a factor structure. In the results of the first factor analysis, the communality of items 9 and 10 (“The duration of adult orthodontic treatment is the same as that of adolescent orthodontic treatment” and “The cost of orthodontic treatment in adults is the same as that in adolescents”) was less than 0.4, indicating no significant contribution of the two items. These items were deleted after review, and only 11 items were selected for the second factor analysis. In the results of the second factor analysis, four factors were derived based on Eigenvalues ≥ 1.0, and the cumulated variance rate was 66.82% (≥ 60%). For the final 11 items, the KMO value was 0.87 (≥ 0.70), and Bartlett's test showed significant results (p < 0.001). The principal component analysis through Varimax rotation showed that the number of factors was four, the communality was 0.50–0.84 (≥ 0.40), factor loading was 0.47–0.89 (≥ 0.40), and the correlation coefficient between the individual and total items was 0.43–0.73 (> 0.30) (Table 3). By reviewing the contents of all items and confirming the internal consistency reliability, 11 items were selected, with no items excluded, by considering the changes in Cronbach's α values when deleting the items.
Confirmatory factor analysis
CFA is a statistical technique used to verify that the factor structure extracted through EFA is also valid in the new sample. In other words, it confirms the fit of the final instrument to the model. CFA was performed on the data of 200 respondents, excluding the data of the 206 respondents used in EFA. The χ2 value was 206.94 (p < 0.001), χ2/df value was 2.11, root mean square residual (RMR) value was 0.04 (< 0.05), RMSEA value was 0.04 (< 0.05), standardized RMR (SRMR) value was 0.01 (< 0.05), goodness-of-fit index (GFI) was 0.93 (> 0.90), comparative fit index (CFI) was 0.94 (> 0.90), and equivalent Tucker–Lewis index (TLI) was 0.92 (> 0.90), indicating that the fit of the model was acceptable (Table 4). Since no additional modification of the model was required, it was confirmed as the final model.
Discriminant validity
In the first discriminant validity test, the r2 value of each factor was 0.22–0.35, which was smaller than the AVE value. In the second discriminant validity test, the r ± (2 × SE) value between the factors was in the range of 0.41–0.66, and none of the factors included a value of 1. These results indicate that the discriminant validity was secured.25
Reliability analysis
In the reliability test performed to confirm the internal consistency of the measurement instrument, a Cronbach's α of 0.845 was obtained. For the individual components, Cronbach's α was 0.764 for the general perception of orthodontic treatment, 0.705 for the perception of orthodontic treatment for adults, 0.707 for the effect, and 0.701 for the retention of orthodontic treatment (Table 5). An improvement of more than 0.1 in the Cronbach's α value could not be achieved by excluding any one of the 11 items.
Optimization of the instrument
After verifying the validity and reliability, 11 items covering four orthodontic treatment factors, including general perception (three items), perception of orthodontic treatment for adults (four items), the effect of treatment (two items), and the retention of orthodontic treatment (two items) were confirmed to constitute the measurement instrument for the perception of orthodontic treatment in adults. The responses were scored on a 4-point Likert scale as follows: 1, not at all; 2, not like that; 3, true; and 4, very true. Higher scores indicated a higher level of perception of orthodontic treatment.
DISCUSSION
This study developed a measurement instrument to assess the perception of orthodontic treatment in adults aged 19 to 64 years and verified the reliability and validity of the proposed tool in accordance with the 8-step method proposed by DeVellis.19 This study presents data pertaining to the preliminary instrument development and verification of the instrument.
The aim of this study was to complement the limitations of previous studies1,15-17 that measured satisfaction, knowledge, attitude, and perceptions of orthodontics without validity and reliability testing, and to develop a standardized instrument for measuring the perceptions of orthodontic treatment in adults. Therefore, in this study, a multi-level allocation sampling method was applied to generalize the research results and secure the representativeness of the samples. The proposed instrument will allow comprehensive assessment of the perceptions of orthodontic treatment among the growing population of adults who wish to undergo the treatment.
This instrument has unique characteristics. First, item analysis, EFA, and CFA were conducted, in addition to evaluations of content validity and face validity. Moreover, verification of convergent/discriminant validity and reliability provided evidence to support the validity and reliability of the instrument. Therefore, the instrument developed in this study represents a reliable and valid measurement approach. Second, since knowledge, emotion, and attitude regarding orthodontic treatment contribute to the perception of this treatment in adults, a comprehensive evaluation is possible. Third, the proposed instrument is a self-reported measurement tool that can easily measure the perceptions of orthodontic treatment in adults on the basis of 11 items. Fourth, during the planning stage of orthodontic treatment in adults, this instrument can be used to measure the patients’ perceptions and the information obtained could be used to guide treatment according to the level of perception. Thus, more individualized treatment planning may be possible.
While orthodontic treatment improves the oral health-related quality of life (OHRQoL),26 and such treatment in adults is an important aspect in a rapidly aging society, adult patients are hesitant about receiving orthodontic treatment. To ensure appropriate adult orthodontic treatment and achieve healthy oral statuses, changes in the perception of this treatment should be clarified in adults. This study developed an effective instrument with confirmed validity and reliability to measure these changes. The instrument will allow measurement of treatment determinants in adults to achieve optimal results while providing appropriate adult orthodontic treatment. The data obtained with the proposed measurement instrument will serve as the basis for future studies investigating the role of orthodontic treatment perception in adults on their treatment decisions. In addition, it will be a cornerstone for related approaches, such as the development and application of educational materials for adults and establishment of a public relations strategy of related association, to promote proper awareness of orthodontic treatment for adults.
To increase the representativeness of the sample, quota sampling was conducted by region and age in the present study. Nevertheless, participants may have provided untruthful answers given the nature of an anonymous online survey.
XML Download