ABSTRACT

Objective:

The aim of the study was to test validity and reliability of the Human Papilloma Virus Knowledge Scale (HPV-KS) in Turkish.

Material and Methods:

The methodological study was conducted with 920 participants at a training and research hospital in Ankara, Turkey, between February and May 2019. The data collection form consisted of descriptive characteristics of the participants and HPV-KS. Guidelines for the Process of Cross-Cultural Adaptation of Self-Reported Measures were followed for the language and cultural adaptation of the scale. Content validity, exploratory, and confirmatory factor analyses were performed to test the validity of the scale. The scale’s reliability was assessed using the item-total correlation, Cronbach’s alpha, and test-retest analysis.

Results:

In line with the experts’ suggestions, two item were excluded from the scale. The scale content validity index was found to be 0.96. The exploratory factor analysis determined the four subscales of the scale with 33-items. The explained variance was found to be 64.56%. In the confirmatory factor analysis, all the goodness of fit indexes had acceptable values. The item-total correlations determined that each item was positively correlated with the total scale ranging from 0.53 to 0.80. The Cronbach’s alpha coefficient calculated for the overall scale was 0.96. It was found that there was a statistically significant positive relationship between test and retest (r=0.166; p=0.05).

Conclusion:

The scale was shown to be a valid and reliable standard measurement that can be used to accurately evaluate the efficacy of health education provided by healthcare professionals.

Keywords:

HPV knowledge, HPV screening tests, HPV vaccine, validity, reliability

Introduction

Human papillomavirus (HPV) is one of the major causes of sexually transmitted disease globally (1). HPV infections spread via direct skin-to-skin contact or through skin-to-mucous membrane contact, and the most common form of this infection is sexual contact (2). Many sexually active individuals are exposed to HPV at some point in their lives (3,4,5). HPV infections are strongly associated with development of cervical cancer (1,6). According to estimations made by the World Health Organizations, 604,000 new cervical cancer cases would be diagnosed and 342,000 deaths would be recorded in 2020 (7). Over the past five years in Turkey, it has been reported that the prevalence of cervical cancer is 7.163, cancer of the vulva is 862, anus cancer is 666, and penile cancer is 70 (8).

HPV vaccines are important for the prevention of HPV infections (9,10). The HPV vaccine is a safe and effective method for preventing most common HPV infections and HPV-related cancers (9,11). Clinical applications of the HPV vaccine demonstrated that it was highly effective against HPV infection in both sexes before the first sexual experience (12,13). In the past decade, HPV vaccines have been included in the national vaccination programs of 121 countries (13). However, the HPV vaccine is not yet included in the national vaccination program in Turkey (14).

Studies have reported that social awareness and knowledge of HPV infection, modes of transmission, and vaccination and screening programs are not at the optimal level globally (15,16,17,18,19,20,21). As a result of an increase in HPV-related diseases, and use of HPV screening tests and HPV vaccines, Waller et al. (22) developed the Human Papilloma Virus Knowledge Scale (HPV-KS) to address the need for a scale that measures and evaluates HPV knowledge. The most important factor contributing to increasing the validity and reliability of a developed scale is testing it in different cultures (17,19,22).

In studies evaluating information, attitudes, and behaviors regarding HPV and vaccines among women, medical staff, and health sciences students in Turkey, it was reported that knowledge of HPV was not at the desired level (16,17,19). Validity and reliability measures information about current HPV screening programs and HPV vaccines in both genders, and has been tested in community-based studies. It was found that there is a need for a comprehensive HPV knowledge scale with confirmed Turkish validity and reliability. Therefore, the aim of this study was to adapt the HPV-KS to the Turkish language and culture and to test its validity and reliability.

Material and Methods

This was a methodological study conducted at the obstetrics and gynecology, urology, dermatology, and general surgery clinics of a tertiary training and research hospital in Ankara, Turkey, between February and May 2019. “Guidelines for the Process of Cross-Cultural Adaptation of Self-Reported Measures” were followed in the language and cultural adaptation of the HPV-KS (23).

The scientific research Ethics Committee of a University of Health Sciences Turkey, Gülhane Training and Research Hospital approved the study protocol (approval number: 41418926-19/20, date: 17.01.2019). All the participants were informed about the study methods, and verbal and written informed consent was obtained. Written permission was obtained from the original author, Joe Waller, via e-mail to use the HPV-KS.

Results

Discussion

This study investigated the validity and reliability of the Turkish version of the HPV-KS. The scale validity was evaluated for linguistic validity, content validity, and construct validity (26,32). First, language and cultural adaptation of the scale application used Guidelines for the Process of Cross-Cultural Adaptation of Self-Reported Measures by Beaton et al. (23). After the language validity, the scope was validated to evaluate the scale’s suitability for the sociocultural characteristics of Turkish society. The Lawshe technique was used, and the CVI value was 0.96. The content validity of the Turkish version of the HPV-KS was quite high (24,25).

The construct validity of scales is assessed using factor analysis (26,32). First, the KMO value was calculated, and Bartlett’s test was used to determine whether the study data were compatible with the construct validity. The KMO value was greater than 0.80, demonstrating that the sample size was sufficient for factor analysis. The findings showed that the sample size was ideal and the correlation between the items as appropriate for factor analysis (32).

In the original HPV-KS study concurrently conducted in the UK, Australia, and the US by Waller et al. (22), the first 29 items were analyzed using EFA and three factors explained 20.13% of the total variance. In this study, the EFA determined that four factors explained 64.56% of the total variance (Table 3). The total variance above 60% in the scales was considered sufficient to define the investigated features (32,33). The total variance above 60% in this study was much higher than the original scale. This may have been because only Turkish society was included, and the language and content validity were assessed before the test implementation. In the original scale’s factor analysis, the results of three different countries were analyzed simultaneously. Therefore, the independent subscale that was created in three different ways according to the national vaccination policies of the three countries was not included in the factor analysis in the original scale study (22). In the present study, considering Turkey’s current HPV vaccine policy, the independent subscale was modified, and factor analysis was applied to the items included in the independent subscale together with other subscales.

In the present study, factor loads were examined according to the EFA of the HPV-KS results. Based on the factor loads result, no item was removed from the HPV-KS. In the original scale, although the factor load was below 0.33, it was reported that no item was removed according to the results of an advanced analysis (22). In the final stage of the EFA analysis, the factors obtained were identified based on the relationship of the meaning and the original scale. The original scale includes three main subscales (29 items) and one independent subscale (6 items).

According to the factor load distribution obtained in the present study, the General HPV knowledge scale and the HPV testing knowledge subscales had the same structure as the original scale, and the HPV vaccination knowledge subscale and the HPV vaccine availability items subscale were partially similar to the original scale. Two items (the 28^th and 29^th) in the HPV vaccination knowledge subscale of the original scale were collected under the HPV vaccine availability Items subscale (22). These items were thought to be displaced because of cultural differences.

Waller et al. (22) reported that in the validity and reliability study of the original HPV-KS, according to the CFA results, three factors were confirmed and the fit indexes were acceptable (22). In this study, the HPV-KS, which had a four-factor structure, and the CFA fit indexes had acceptable values (32) (Table 4). As a result of the CFA, we decided to preserve the scale’s structure determined using the EFA.

A scale’s reliability is determined by its invariance and internal consistency (26,32). For this study’s internal consistency, the item analysis was based on the item-total correlation.
The total score of the HPV-KS Turkish version and each scale item were positively correlated with a range of 0.53 to 0.80, so there was no need to remove items from the scale. In the original scale, no comparison was conducted because the item-total correlation was not examined.

The internal consistency analysis also calculates the reliability coefficient. For the 29 items in the original scale, Cronbach’s alpha was 0.83, and the internal consistency was reported to be high (22). In this study, Cronbach’s alpha of the 33-item HPV-KS Turkish version was 0.96, similar to the result of the original scale. This showed that the internal consistency of the Turkish version of the HPV-KS was quite high.

The test-retest method is another method for reliability analysis. In the present study, a positive and significant relationship between the HPV-KS total score and the retest score comparing the first application result and the retest result supported the scale’s internal consistency. The t-test is another method used to examine the internal consistency of the scale with the test-retest method (33,34). In the present study, a comparison of the mean scores of the HPV-KS total and subscales obtained from the test and retest results determined that there was no significant difference between the total and subscale mean scores. This demonstrates that the scale’s test and retest scale mean scores were similar, strengthening the internal consistency. The Turkish version of the 33-item HPV-KS was thus shown to be a valid and reliable measurement instrument. The total HPV-KS score may be between “0 and 33”. Higher scores indicate a thorough understanding of HPV general knowledge, HPV screening tests, and the HPV vaccine.

Studies have examined the knowledge, beliefs, attitudes, and behavior in many societies concerning HPV, HPV screening tests, and HPV vaccines (28,29,35). Studies have also examined whether sociodemographic characteristics, such as gender, age, educational status, marital status, monthly income level, and women who have had Pap tests have an impact on the results related to HPV. Studies conducted worldwide on HPV and HPV vaccines reported that women have more information on HPV and HPV vaccines than men (28,29,35). In this study, and in contrast to previous reports, no significant difference was found between the sexes in terms of comprehensive information about HPV. Thus, the public needs comprehensive information on this subject without separating the genders. The current study found that sociodemographic characteristics, such as age, educational status, marital status, and monthly income levels made no difference regarding knowledge of HPV and HPV vaccines. In the present study, women who had undergone a Pap test had significantly higher HPV knowledge than women who had not have Pap tests, in line with previous reports (16,29). Therefore, women participating in screening programs have more information about and awareness of HPV. In the present study, sociodemographic features, health history, and sexual health-related features made no difference in terms of HPV and HPV vaccine knowledge, whereas the greater knowledge level of women who underwent Pap tests supported the opinion that the HPV-KS was a consistent scale for measuring information.

Conclusion

Conducting validity and reliability analyses of a scale in different cultures contributes to the widespread use of that scale as a standard measurement instrument and provides intercultural comparison. The results of the validity and reliability analyses in this study determined that the 33-item Turkish version of the HPV-KS was a valid and reliable measurement instrument that can be used in women and men aged 18-49 in Turkish society. The HPV-KS can be used to determine the level of knowledge of healthcare professionals responsible for providing health education and healthcare students regarding HPV infection HPV screening tests, and HPV vaccines. The HPV-KS is considered a valid and reliable standard measurement instrument that can be used to accurately evaluate the efficacy of health education provided by healthcare professionals.

Stage 1: translation

The first stage in adaptation is the forward translation from the original language to the target language (23). To produce the Turkish language validation of the HPV-KS, it was first translated from English to Turkish by three academician experts in their field.

Stage 2: synthesis of the translations

Then it is expected that the translations of each item are brought together and documented, and a synthesis of the common translations is created at the end of this stage (23). In this stage, three different academicians evaluated the compatibility of the translations with each other and Turkish culture.

Stage 3: back translation

In this stage, in order to provide validity of the translation, other translators who are blinded to the original text are expected to translate the text backward to the original language (23). In this stage, the scale was translated from Turkish back to English by three different academicians.

Stage 4: expert committee

The semantics, idiomatic, experiential, and conceptual equivalence in the back translation and the original scale were examined (23). For the content validity, 13 academicians experienced in their field scored the scale according to its suitability with Turkish language and culture by comparing each item’s original English and Turkish translation (24,25).

Stage 5: test of the pre-final version

To pretest is the final stage of the adaptation process. Ideally, this field test of the new questionnaire should be tested in the pre-final version among 30 to 40 people from the target population (23). To evaluate the operability and comprehensibility of the data collection form, a pre-test was made with 20 women and 20 men. As a result of the pre-test, further corrections were made throughout the data collection form. The data obtained in the pre-test were excluded from the final analysis of the study.

Sample size

In scale validity and reliability studies, it is reported that the participant/item ratio should be at least 10/1 to satisfy factor analysis conditions when calculating sample sizes (26,27). Anticipating that some participants would drop out of the study, for this study 30% more participants were recruited than the recommended number. The present study sample size was calculated as 910, made up of 455 women and 455 men for the 35-item HPV-KS. The study sample consisted of 920 volunteers who visited the hospital’s gynecology and obstetrics, urology, dermatology, and general surgery polyclinics for any reason. Inclusion criteria for the study were subjects between the ages of 18-49 years who were literate and with no written or oral communication barriers. For the scale retest, the first 140 participants were interviewed twice, with the second round of data collected two weeks after the first survey administration.

Data collection tools

Data collection forms were developed following a literature review conducted by a researcher (15,17,20,28,29). The data collection forms were organized into a female (58 questions) and male (54 questions) version. The forms consisted of three parts. In the first part, there were 14 items for the participants’ sociodemographic characteristics. The second part consisted of questions assessing their knowledge of sexually transmitted diseases (STDs), age of first sexual intercourse, history of disease/discomfort in the genitals, and other factors that were prepared differently for both sexes.

HPV Knowledge Scale: The HPV-KS was included in the third part of the data collection form. The HPV-KS was developed by Waller et al. (22) in 2013 to measure individuals’ knowledge of HPV, HPV testing, and HPV vaccination. The HPV-KS has a total of 35 items. The first 29 items are examined in three subscales, and the remaining six items are evaluated as an independent subscale (22). This scale was chosen to be adapted into Turkish because it evaluates essential issues, such as knowledge of HPV, HPV-related cancers, HPV screening tests, and HPV vaccines. There is no short and comprehensive information scale to assess this issue in Turkey. It has been shown as an effective measurement tool for assessing these areas and it is easy to score. The first subscale of the HPV-KS is “General HPV knowledge”, with 16 questions on the subjects’ general knowledge of HPV. The second subscale, “HPV testing knowledge”, has six items related to HPV screening tests. The third subscale, “HPV vaccine knowledge” consists of seven items regarding HPV vaccine information. “HPV vaccine availability” items, the independent HPV-KS subscale, is organized in three different ways for the HPV vaccine program, which is conducted in three countries (the UK, the US, and Australia) where the scale is applied. The independent subscale can be modified in accordance with the HPV vaccine policy of each country Waller et al. (22).

Participants mark each item of the HPV-KS as “Yes”, “No”, and “I don’t know”. In the evaluation phase, each correct answer is scored as “1”, and wrong answers and “I don’t know” statements are scored as “0”. The total HPV-KS score may be between “0 and 35”. Higher scores indicate a thorough understanding of HPV general knowledge, HPV screening tests, and the HPV vaccine.

Data collection

Data collection forms were administered face to face and under appropriate conditions where the privacy of the participants was ensured. After the briefing, the participants were allowed to answer the form individually. Participant questions during form completion were answered by the researcher without giving any information about HPV. Answering the data collection forms took an average of 25-35 minutes.

Statistical analysis

The data obtained in the study are presented as number and percentage for categorical variables and as mean ± standard deviation for continuous variables. The normality of the distribution was analyzed using the Kolmogorov-Smirnov test. The compatibility of the factor analysis of compatibility of data sets was examined by Kaiser-Meyer-Olkin (KMO) and Bartletts’ tests. To test the construct validity of the HPV-KS, explanatory factor analysis (EFA) with varimax axis rotation and confirmatory factor analysis (CFA) were conducted. CFA based on polychoric correlation was applied to the data set and non-weighted least squares estimation method was chosen as a parameter estimation method. The scale’s reliability was assessed using the item-total correlation, Cronbach’s alpha, item analysis, and test-retest analysis. The average total and subscale scores were calculated. The Mann-Whitney U test and paired sample t-tests were used to analyze the relationships between dependent and independent variables. The HPV-KS scale total score average and subscale scale mean scores were calculated. IBM SPSS Statistics for Windows, version 22.0 (30) and IBM Statistics AMOS 21.0 (31) were used to analyze the data statistically. A p-value less than 0.05 was considered statistically significant.

Characteristics of participants

In the current study, 50% (n=460) of the participants were female, 52.2% (n=480) were married, and 12.4% (n=114) had an education of eight years or less (Table 1). Overall, 8.0% (n=37) of the women in the study and 33.7% (n=155) of the men reported that they had their first sexual intercourse between the ages of 12 and 18. A total of 193 (42%) of the women participants and 201 (43.7%) of men stated that they needed health education about STDs. A total of 296 (64.3%) of the male participants reported that they used condoms during sexual intercourse. Only four (0.9%) of the participants [female: (n=4), male: (n=0)] had an HPV vaccine (Table 2).

Validity analyses

When testing the content validity of the Turkish language version of the HPV-KS, the Lawshe technique was used (25). In line with the experts’ suggestions, the 32^nd item in the independent subscale “The Vaccines for Children Program provides free HPV vaccines to children who are uninsured, underinsured, or on Medicaid” and the 35^th item “The HPV vaccine is usually given to girls in school settings” were excluded because they were unsuitable for the Turkish national vaccination program. As a result of the language and content validity, 33 items were included in the Turkish version of the HPV-KS. In this study, the content validity index (CVI) score of the Turkish version of the HPV-KS was 0.96 (24,25).

KMO and Bartlett’s tests were used to measure the suitability of the sampling for factor analysis. The KMO measure of sampling adequacy of the data was 0.96, and Bartlett’s test was highly significant (χ²=3006.5, p<0.001). EFA conducted with varimax axis rotation yielded eigenvalues greater than 1, and four factors were identified that explained 64.56% of the total variance (Table 3). The EFA determined that the four subscales of the 33-item HPV-KS scale had factor loads varying between 0.54 and 0.80 (Table 3, Figure 1).

CFA was conducted to test the accuracy of the factor structure identified in the EFA within the scope of the HPV-KS construct validity to identify the relevance values. The CFA found that the χ²/SD, Goodness of Fit Index, Adjusted Goodness of Fit Index, Comparative Fit Index, Root Mean Square Error of Approximation, and Normed Fit Indexes of the Turkish version had acceptable values (32) (Table 4). The first factor included items 1 to 16 and this factor was termed General HPV knowledge. The fourth factor included items 17 to 22 and this factor was termed HPV Testing Knowledge and consisted of the original scale’s second subscale. The third factor included items 23 through 27 and this factor was termed HPV vaccine knowledge and was the original scale’s third subscale. The 28^th and 29^th items in the HPV vaccine knowledge scale in the original HPV-KS were separated from this scale because of the factor analysis. The second factor included items 28 to 33 and this factor was named as HPV vaccine availability (Table 3, Figure 1).

Reliability analysis

In the current study, item analysis was conducted to determine the internal consistency of the HPV-KS. Item-total correlations were evaluated to analyze the contribution of the items to the total score. The item analysis determined that each item was positively correlated with the total scale in a range of 0.53 to 0.80 and that there was no need to remove any items from the scale. The scale’s internal reliability coefficient was calculated to determine the internal consistency. Cronbach’s alpha of the HPV-KS was 0.96 for the total scale (Table 5). When any item of the HPV-KS was deleted, there was no change in the scale’s reliability coefficient.

In this study, Spearman’s correlation analysis was used to assess the relationship between the HPV-KS total score of the 920 participants and the HPV-KS total retest score applied to 140 participants to determine the internal consistency of the HPV-KS. The analysis found a statistically significant positive relationship between the two applications (r=0.166; p=0.05).

Relationship between the HPV-KS and some sociodemographic characteristics

A statistically significant difference was found between the female participants’ Pap tests and the HPV-KS scale total and subscale mean scores (p<0.05). The total scale and subscale mean scores of the women who had Pap tests were higher than the mean scores of the women who did not (z=2.454, p=0.01). According to the analyses between gender, marital status, marriage duration, age, family type, economic income level, perception of income, longest-lived region, health history, and sexual health-related features and the HPV-KS total scale and subscale mean scores, there was no statistically significant relationship (p>0.05).

Study Limitations

The fact that the study was conducted in a single center was considered as a limitation of the study.

References

Mavundza EJ, Iwu-Jaja CJ, Wiyeh AB, Gausi B, Abdullahi LH, Halle-Ekane G, et al. A Systematic Review of Interventions to Improve HPV Vaccination Coverage. Vaccines (Basel) 2021; 9: 687.

Cheng L, Wang Y, Du J. Human Papillomavirus Vaccines: An Updated Review. Vaccines (Basel) 2020; 8: 391.

Bretagne CH, Jooste V, Guenat D, Riethmuller D, Bouvier AM, Bedgedjian I, et al. Prevalence and distribution of HPV genotypes and cervical-associated lesions in sexually active young French women following HPV vaccine. J Gynecol Obstet Hum Reprod 2018; 47: 525-31.

Crosbie EJ, Einstein MH, Franceschi S, Kitchener HC. Human papillomavirus and cervical cancer. Lancet 2013; 382: 889-99.

Priest HM, Knowlden AP. Systematic review of primary prevention Human Papillomavirus interventions targeting college students. Int J Sex Health 2015; 27: 125-44.

Centers for Disease Control and Prevention. Cancers associated with human papillomavirus. [Cited 20 December 2021]. United States— 2012–2016. U.S. Cancer Statistics data brief, no 10. Atlanta, GA: Centers for Disease Control and Prevention, US Department of Health and Human Services. Retrieved from https://www.cdc.gov/cancer/uscs/pdf/USCS-DataBrief-No10-August2019-h.pdf

World Health Organization. Cervical cancer. [Cited January 24, 2022]. Retrieved from: https://www.who.int/news-room/fact-sheets/detail/human-papillomavirus-(hpv)-and-cervical-cancer#:~:text=HPV%20is%20the%20most%20common,populations%20eventually%20clear%20the%20infection.

GLOBOCAN. Global cancer statistics: Turkey. [Cited January 24, 2022]. Retrieved from http://gco.iarc.fr/today/data/factsheets/populations/792-turkey-fact-sheets.pdf

Apaydin KZ, Fontenot HB, Shtasel DL, Mayer KH, Keuroghlian AS. Primary Care Provider Practices and Perceptions Regarding HPV Vaccination and Anal Cancer Screening at a Boston Community Health Center. J Community Health 2018; 43: 792-801.

Kaur P, Mehrotra R, Rengaswamy S, Kaur T, Hariprasad R, Mehendale SM, et al. Human papillomavirus vaccine for cancer cervix prevention: Rationale & recommendations for implementation in India. Indian J Med Res 2017; 146: 153-7.

Donadiki EM, Jiménez-García R, Hernández-Barrera V, Sourtzi P, Carrasco-Garrido P, López de Andrés A, et al. Health Belief Model applied to non-compliance with HPV vaccine among female university students. Public Health 2014; 128: 268-73.

Chatterjee A. The next generation of HPV vaccines: nonavalent vaccine V503 on the horizon. Expert Rev Vaccines 2014; 13: 1279-90.

World Health Organization. Vaccine-preventable diseases: monitoring system. Global Summary. [Cited November 29, 2021]. Retrieved from: http://apps.who.int/immunization_monitoring/globalsummary/schedules

Selçuk I, Üstün Y. Future Feasibility of Nonavalent 9-valent HPV Vaccine in Turkey. The Journal of Gynecology - Obstetrics and Neonatology 2019; 16: 41-4.

Capogrosso P, Ventimiglia E, Matloob R, Colicchia M, Serino A, Castagna G, et al. Awareness and knowledge of human papillomavirus-related diseases are still dramatically insufficient in the era of high-coverage vaccination programs. World J Urol 2015; 33: 873-80.

Dönmez S, Öztürk R, Kısa S, Karaoz Weller B, Zeyneloğlu S. Knowledge and perception of female nursing students about human papillomavirus (HPV), cervical cancer, and attitudes toward HPV vaccination. J Am Coll Health 2019; 67: 410-7.

Guvenc G, Seven M, Akyuz A. Health Belief Model Scale for Human Papilloma Virus and its Vaccination: Adaptation and Psychometric Testing. J Pediatr Adolesc Gynecol 2016; 29: 252-8.

Kops NL, Hohenberger GF, Bessel M, Correia Horvath JD, Domingues C, Kalume Maranhão AG, et al. Knowledge about HPV and vaccination among young adult men and women: Results of a national survey. Papillomavirus Res 2019; 7: 123-8.

Özdemir E, Kısa S. Validation of the Turkish Cervical Cancer and Human Papilloma Virus Awareness Questionnaire. Int Nurs Rev 2016; 63: 465-72.

Turhan E, Cetin S, Cetin M, Abacigil F. Awareness and Knowledge Levels of 18-Year-Old and Older Individuals Regarding Human Papillomavirus (HPV) and HPV Vaccine in Hatay, Turkey. J Cancer Educ 2019; 34: 234-41.

Vaidakis D, Moustaki I, Zervas I, Barbouni A, Merakou K, Chrysi MS, et al. Knowledge of Greek adolescents on human papilloma virus (HPV) and vaccination: A national epidemiologic study. Medicine (Baltimore) 2017; 96: e5287.

Waller J, Ostini R, Marlow LA, McCaffery K, Zimet G. Validation of a measure of knowledge about human papillomavirus (HPV) using item response theory and classical test theory. Prev Med 2013; 56: 35-40.

Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000; 25: 3186-91.

Ayre C, Scally, AJ. Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Meas Eval Couns Dev 2014; 47: 79-86.

Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975; 28: 563-75.

Erdoğan S, Nahcivan N, Esin MN. Hemşirelikte araştırma; süreç, uygulama ve kritik. Nobel Tıp Kitapevleri, Inc; 2017. p. 195-232.

Norman GR, Streiner DL. Biostatistics the bare essentials. Lewiston, BC Decker, 3.rd ed, Inc; 2008.

Dodd RH, McCaffery KJ, Marlow LA, Ostini R, Zimet GD, Waller J. Knowledge of human papillomavirus (HPV) testing in the USA, the UK and Australia: an international survey. Sex Transm Infect 2014; 90: 201-7.

Wheldon CW, Krakow M, Thompson EL, Moser RP. National Trends in Human Papillomavirus Awareness and Knowledge of Human Papillomavirus-Related Cancers. Am J Prev Med 2019; 56: e117-23.

IBM Corp. Released. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp; 2013.

Arbuckle JL. Amos (Version 23.0) [Computer Program]. Chicago: IBM SPSS; 2014.

Kartal M, Bardakçı S. SPSS ve AMOS Uygulamalı Örneklerle Güvenirlik ve Geçerlik Analizleri. Akademisyen Yayınevi, Inc; 2018.

Ercan I, Kan I. Reliability and validity in the scales. Journal of Uludag University Medicine Faculty 2004; 30: 211-6.

Karakoç A, Dönmez P. Basic Principles Of Scale Development. Tıp Eğitimi Dünyası 2014; 13: 39-49.

Radecki Breitkopf C, Finney Rutten LJ, Findley V, Jacobson DJ, Wilson PM, Albertie M, et al. Awareness and knowledge of Human Papillomavirus (HPV), HPV-related cancers, and HPV vaccines in an uninsured adult clinic population. Cancer Med 2016; 5: 3346-52.

Validity and reliability of a Turkish version of the human papillomavirus knowledge scale: a methodological study