Development and validation of a questionnaire to evaluate the knowledge, attitude and practices regarding travel medicine amongst physicians in an apex tertiary hospital in Northern India
Tropical Diseases, Travel Medicine and Vaccines volume 8, Article number: 13 (2022)
Travel medicine focuses primarily on pre-travel preventive care and the conditions and diseases acquired during or after travel. There is a paucity of validated tools to assess the knowledge, attitude and practises of physicians with regard to travel medicine. We attempted to develop a tool to assess existing expertise among Medicine and Infectious Diseases resident doctors with respect to travel medicine.
Item level content validity index (I-CVI) and scale level content validity index (S-CVI/Ave) were estimated for each of the items to establish the content validity. Refined measures of inter-rater agreement (Brennan and Prediger Agreement Coefficient and Gwet’s Agreement Coefficient) were estimated for the tool.
The final version of the questionnaire had satisfactory content validity (I-CVI > 0∙6 and S-CVI/Ave > 0∙9) and possessed high agreement among the raters (Brennan and Prediger AC > 0∙7, p < 0∙01 and Gwet's AC > 0∙8, p < 0∙01) with regard to necessity, clarity and relevance of the scale.
This tool covers a wide range of questions and is scientifically validated. The final version of the tool can be used largely for the assessment of knowledge, attitude and practices among medical practitioners. This is instrumental to build targeted intervention programs to enhance the knowledge regarding travel medicine among health care providers.
After a long gap of the Spanish flu pandemic, the emergence of COVID-19 has taught many lessons to humanity. One of the key message is to realize the importance of public health from a global perspective. The health care system should be geographically inclusive and not be confined to a region or a country because a pandemic like this has no boundaries. Travel medicine globalizes health care in terms of providing preventive and curative health across boundaries.
Travel medicine or emporiatrics is the field of medicine which is concerned with the promotion and protection of health of travellers. It aims to prevent diseases and other adverse health outcomes among international travellers. It requires up-to-date information on the global epidemiology of the non-infectious and infectious health risks, health regulations and vaccination necessities in various countries along with the emerging patterns of medication-resistant infections . As international travel becomes more accessible, knowledge of this field is likely to become essential for a physician [2, 3]. Although derived from the traditional medicine disciplines, this branch of medicine is a newly emerging field given the increasing number of international travellers and reporting of various infectious and non-infectious diseases , injuries  and other health risks among international travellers [6,7,8]. Since travel medicine is a new discipline, expert opinion and experience still dominate many areas in this branch, highlighting the need for continuous investigation in the field .
International travellers are at higher risk of developing various health threats, which depend on both the health needs of the traveller and on the type of travel to be undertaken. The traveller’s triad includes the three major components that influence the risk associated with a specific travel plan i.e. place, time and person. The region of the world being visited determines the altitude, humidity, temperature and infection profile etc. The travellers’ vulnerability to these exposures may be determined by their age, general well-being, the trip's length, and the diversity of planned activities . Pre-travel health education, vaccination and prophylactic drugs may serve to mitigate these risks [10,11,12].
With rapidly evolving travel regulations, there is a need to provide training to practicing physicians to predict travel-associated health risks and recognise untoward exposures. As travel medicine gains prominence worldwide, we recognise the dearth of adequately trained field experts. In the absence of subject specialists, general physicians must be provided formal training to ensure adequate care . In this regard, an assessment of the existing knowledge among health care practitioners is necessary to develop interventions for targeting gaps in knowledge.
With this objective in mind, an attempt was made at our tertiary care facility to develop a comprehensive tool covering major aspects of travel medicine. Currently, no widely disseminated, valid instrument for assessing travel medicine's knowledge, attitude, and practices is available in India. This tool assesses the knowledge, attitude and practices (KAP) regarding travel medicine in the form of a questionnaire. We also attempted to validate this tool in primary care physicians, internist, infectious disease specialists, and other health care providers.
Material and methods
Development and validation of the travel medicine questionnaire
Step I: Conceptualization and identification of domains and sub-domains for the travel medicine knowledge, attitude and practices assessment tool
For conceptualizing and identification of domains and sub-domains to develop the initial pool of items, multiple round table discussions and focus group discussions (FGDs) were held among experts from different fields of medicine, infectious diseases and travel medicine from five tertiary care centers in different parts of India. This included subject experts with certifications in travel medicine from the International Society of Travel Medicine (ISTM) and physicians currently practicing in travel clinics across the country (Fig. 1).
Step II: Literature review to identify domains and sub-domains and generating preliminary item pool for the tool
An extensive literature review was carried out to analyse the existing evidence on travel medicine. The literature search aimed to identify domains and sub-domains required to develop the travel medicine KAP assessment tool. We used the standard textbooks, journals, and internet databases for the identification of relevant concepts. The internet search engines used were Google Scholar, PubMed, Scopus and JSTOR using the keywords string: (Travel Medicine) AND (Knowledge) AND (Attitude) AND (Practices) published in or after year 2000. Table 1 shows the details of some of the sources used to identify the domains and sub-domains and to generate a preliminary item pool for our travel medicine tool.
Step-III: Developing the structure of the questionnaire through expert review
The initial pool of 157 questions was again reviewed by the experts, and the number of items was reduced to 143. This set of identified items was organized into the form of a questionnaire. The questionnaire was constructed in a simple and lucid language, and the flow of the questions was maintained, keeping in mind the purpose of assessment of KAP.
Step-IV: Establishing face and content validity and estimation of agreement coefficients
Face validity is the lowest level of validity and represents the assumption of an expert and acceptance that a test represents the domain being assessed. After preparing the first draft, the questionnaire was reviewed by the experts in the field for face validity (by assessing for readability, comprehensibility, feasibility, completeness and layout and style).
The purpose of content validation is to reduce the bias associated with the operationalization of the instrument in the initial stages . To establish the content validity of the travel medicine tool, we chose three components to judge the overall content validity. Studies estimate the content validity indices with a single validity parameter. However, a few studies adopted a different approach and decomposed the overall validity into its components [15, 16]. We adopted this technique and judged the overall content validity in terms of—Necessity (Is the question necessary to be asked to the resident to assess their knowledge/attitude/practices of travel medicine?), Clarity (Is the question wording/structure/options given convey the meaning effectively?) and Relevance (Is the question relevant as far as branch of travel medicine is concerned with respect to knowledge/attitude/practice?). Each question was evaluated by the experts for these components separately. The rating protocol was designed into the form of a Likert scale as-
1) Necessity (N): Each item was rated as: 1 (neither useful nor necessary), 2 (useful but not necessary) and 3 (essential).
2) Clarity (C): Rate each item as 1(not clear), 2(slightly clear/needs major revision), 3 (clear/needs minor revision), 4 (very clear).
3) Relevance (R): Each item was rated as: 1 (not relevant), 2 (slightly relevant/needs major revision), 3 (relevant/needs minor revision), 4 (very relevant).
The idea behind decomposing the overall quality judgement into its components (N, C and R) was to give more freedom to the experts to judge and to provide more strength to the validation process. Furthermore, we also looked for any lack of consistency between experts for travel medicine questionnaire in terms of three above mentioned parameters.
We chose several measures of inter-rater agreement as well as indices of validity to validate the travel medicine tool.
Content validity indices (CVI)
Two indices have been proposed by researchers for judgement of content validity of a tool. This includes-item level content validity index (I-CVI) and scale level content validity index (S-CVI). The eight experts rated each of the item in terms of N, C and R as mentioned above. In the next step, these scores were dichotomized. For necessity, the dichotomous variable was categorized as ‘1’ if the item was rated as ‘3’ (essential) or ‘0’ otherwise. Likewise, for clarity and relevance the dichotomous variables were generated as ‘1’ for experts giving a rating of 3 or 4 and ‘0’ otherwise. These dichotomous variables were then used to estimate the content validity indices (I-CVI and S-CVI/Ave) for each of these characteristics. The item-level CVI (I-CVI) is computed by dividing the total number of ‘1’s by the total number of experts [17, 18]. The S-CVI/Ave is then calculated by averaging the I-CVIs estimated for each item of the instrument. This exercise was repeated for each of the parameters of content validity (necessity, clarity and relevance). Polit and Beck  recommended an S-CVI/Ave of 0∙90 or above as excellent.
Agreement Coefficients (AC)
The most popular method of quantification of inter-rater agreement among researchers has been the Cohen’s Kappa [19,20,21,22]. Recent published literature discussed the limitations of kappa statistic and proposed other measures of inter-rater agreement [19, 22]. Klein  has pointed out the limitations of Cohen’s Kappa and suggested that the Brennan and Prediger  coefficient and Gwet’s [24, 25] agreement coefficient arguably represent the data more accurately. Further, he suggested that these two agreement measures are found to be more robust than any other measure of inter rater agreement . Similarly, Wongpakaran  found Gwet’s AC to provide a more stable inter-rater reliability coefficient than Cohen’s Kappa and recommended to use for inter-rater reliability analysis. A more detailed discussion on each of the agreement coefficients may be found elsewhere [19, 22, 25]. In our study, we estimated these agreement coefficients along with the percent agreement for each of the three above mentioned components.
Content validity indices
Table 1 shows the I-CVIs and S-CVI/Ave for each of the component of content validity. Polit and Beck recommended an I-CVI ≥ 0∙78 for inclusion of an item . But, we adopted a less strict cut-off of I-CVI < 0∙60 for deletion of the items from the pool . We removed the items which had an I-CVI < 0∙6. The I-CVIs for necessity ranged from 0∙625 to 1∙000. Six items had a necessity I-CVI of 0∙625 whereas five items had 0∙750. Rest of the items had an I-CVI ≥ 0∙78 where out of total 106 items 85 items had an I-CVI of 1∙000. This reflects satisfactory ranges for necessity in terms of I-CVIs. For clarity, we observed a slightly better I-CVIs which ranged from 0∙875 to 1∙000 with six items having an I-CVI of 0∙875 and rest of the items had 1∙000. Likewise, for relevance the I-CVIs ranged from 0∙625 to 1∙000 where four items had an I-CVI of 0∙625. The overall scale level content validity index (S-CVI/Ave) was observed to be above 0∙900 for each of the dimension of content validation.
Agreement among the experts
The final version of questionnaire had 106 items after removal of items with an I-CVI < 0∙6 (Supplementary Table 1). For the remaining items, we estimated agreement indicators. The results of these agreement indicators with regard to the three dimensions of overall quality of the tool has been depicted in Table 2. We observed a high and statistically significant percentage agreement among experts with regard to the overall validity of the travel medicine tool. For each of the dimensions, the overall percent agreement among the experts was above 90 percent. The cut-offs of agreement according to Gwet’s AC as categorized by Tammaa  is as follows: < 0∙2 = poor; 0∙21–0∙4 = fair; 0∙41–0∙6 = moderate; 0∙61–0∙8 = substantial; and 0∙81–1∙0 = almost perfect. For each of the dimensions, we observed Gwet’s AC > 0∙8 showing high levels of agreement among raters (Table 3).
With the increasing number of international travellers, travel medicine has gained new significance. Studies have highlighted that the prevalence of travel related problems is surprisingly high among the travellers [7, 28,29,30] and also pointed out the inadequacies in KAP in both health providers as well as the travellers [31,32,33,34]. In order to build any specific intervention program to increase knowledge regarding travel medicine in healthcare practitioners, it becomes necessary to assess the existing knowledge of the health providers.
Earlier KAP studies in travel medicine are either disease-specific [31, 35] or conducted amongst travellers [32, 33, 35, 36]. None of these studies have elucidated on the development process of their tools, and validation data on the same is often missing. Very few studies have been published which attempted to develop and validate the questionnaire regarding travel medicine among medical practitioners. Ratnam et.al. developed and validated a questionnaire to assess the risk of developing viral infections in Australian Travellers . The study covers only a particular domain (viral infections) of travel related problems among travellers and does not establish the content validity through estimation of content validity indices (I-CVI and S-CVI).
The major strength of this study is the development and validation of a travel medicine tool, which will enable the researchers to assess the KAP among health care providers. The content collection through thorough literature review as well as several rounds of discussion with the experts ensured the quality and coverage. Further, the establishment of content validity through expert evaluation and measures of content validity and agreement coefficients made the tool robust and scientifically validated. The final set of 106 questions had satisfactory content validity indices (I-CVI > 0∙6 and S-CVI/Ave > 0∙9). The agreement coefficients (Brennan and Prediger AC > 0∙7, p < 0∙01 and Gwet's AC > 0∙8, p < 0∙01) among the raters with regard to necessity, clarity and relevance of the travel medicine KAP assessment tool were observed to be high.
This study is not free from limitations. The experts chosen for reviewing the travel medicine tool are from internal medicine, infectious disease programme and allied branches who are involved in operating clinics of travel medicine since dedicated travel medicine branch is yet to evolve in India. One who has completed a certificate course in Travel Health from International Society of Travel Medicine (ISTM) was actively involve as an expert. Although, we have taken utmost care to cover every aspect of travel medicine, since it is a vast discipline there is always a prospect of modification and improvement of this tool. Due to limited resources available for the study, only experts from India were involved to review the travel medicine KAP tool. However, AIIMS, New Delhi being an apex health care center of India has specialists from all clinical domains of human health. Therefore, their expertise was used for the development of initial pool of items and revised version of the tool.
The patterns of infectious diseases vary by geographic region and population  and differences in the climate of various regions also impact the patterns of infectious diseases  and therefore require special attention by health care providers. We suggest that the definition of travel medicine should be expanded in such a way that it covers the health problems of domestic travellers and repatriates, to prevent the spread of infectious diseases especially various kinds of respiratory tract infections (RTIs) which may be highly contagious and can give rise to a pandemic. So, comprehensive attempts should be made to make the definition more exhaustive and the possible inclusion of this aspect should be the point of consideration in future.
The pre-travel consultation has become a necessary part of the travellers’ checklist. Considering this issue, present study is a significant contribution in the field of travel medicine and provides the basis for the assessment of the knowledge, attitude and practices among medical practitioners so that adequate intervention programs may be developed to enhance the knowledge of travel medicine among health care providers. This tool covers a wide range of questions and is scientifically validated. The final version of the tool can be used globally for the assessment of knowledge, attitude and practices among medical practitioners. This is instrumental to build targeted intervention programs to enhance the knowledge regarding travel medicine among health care providers.
Availability of data and materials
The dataset supporting the conclusions of this article is available by taking prior approval of the corresponding author.
Aw B, Boraston S, Botten D, Cherniwchan D, Fazal H, Kelton T, et al. Travel medicine: What’s involved? When to refer? Can Fam Physician. 2014;60:1091–103.
Treadwell TL. Trends in travel. In: Zuckerman JN, editor. Princ Pract Travel Med. 2nd ed. United Kingdom: John Wiley & Sons, Ltd; 2013. p. 3–4.
Kozarsky PE, Keystone JS. Introduction to Travel Medicine. In: Keystone JS, Kozarsky PE, Bradley A. Connor, Hans D. Nothdurft, Mendelson M, Leder K, editors. Travel Med. 4th ed. United States: Elsevier; 2019. p. 1–2.
Torresi J, McGuinness S, Leder K, O’Brien D, Ruff T, Starr M, et al. Non-infectious Problems. Man Travel Med. 4th ed. Singapore; 2019. p. 265–96.
WHO. Injuries and violence. In: Poumerol G, Wilder-Smith A, editors. Int Travel Heal. 1st ed. Switzerland: World Health Organization; 2012. p. 51–3.
Flaherty GT, Chen B, Avalos G. Individual traveller health priorities and the pre-travel health consultation. J Travel Med. 2017;24:1–4.
Farnham A, Furrer R, Blanke U, Stone E, Hatz C, Puhan MA. The quantified self during travel: mapping health in a prospective cohort of travellers. J Travel Med. 2017;24:1–8.
Grieve S, Steffen R. Epidemiology: Morbidity and Mortality in Travelers. In: Keystone JS, Kozarsky PE, Connor BA, Nothdurft HD, Mendelson M, Leder K, editors. Travel Med. 4th ed. Elsevier; 2019. p. 3–14.
Hill DR, Ericsson CD, Pearson RD, Keystone JS, Freedman DO, Kozarsky PE, et al. The practice of travel medicine: Guidelines by the infectious diseases society of America. Clin Infect Dis. 2006;43:1499–539.
CDC. CDC Yellow Book 2018: Health Information for International Travel. Brunette GW, editor. United States: Oxford University Press; 2018.
Tessier D. Fitness to travel. In: Jane N Zuckerman, editor. Princ Pract Travel Med. 2nd ed. United Kingdom: John Wiley & Sons, Ltd; 2013. p. 27–36.
Torresi J, McGuinness S, Leder K, O’Brien D, Ruff T, Starr M, et al. Manual of Travel Medicine. 4th ed. Singapore: Springer Nature Singapore; 2019.
Kozarsky PE, Steffen R. Travel medicine education-what are the needs? J Travel Med. 2016;23:1–3.
Shrotryia VK, Dhanda U. Content Validity of Assessment Instrument for Employee Engagement. SAGE Open [Internet]. 2019;9:1–7. Available from: https://doi.org/10.1177/2158244018821751
Halek M, Holle D, Bartholomeyczik S. Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia. BMC Health Serv Res. 2017;17(1):554.
Rodrigues IB, Adachi JD, Beattie KA, MacDermid JC. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet Disord. 2017;18:1–9.
Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Res Nurs Heal. 2006;29:489–97.
Kovacic D. Using the Content Validity Index to Determine Content Validity of an Instrument Assessing Health Care Providers’ General Knowledge of Human Trafficking. J Hum Traffick. 2018;4:327–35. Available from: https://doi.org/10.1080/23322705.2017.1364905 (Routledge).
Klein D. Implementing a general framework for assessing interrater agreement in stata. Stata J. 2018;18:871–901.
de Raadt A, Warrens MJ, Bosker RJ, Kiers HAL. A Comparison of Reliability Coefficients for Ordinal Rating Scales. J Classif. 2021;1–25. Available from: https://link.springer.com/article/10.1007/s00357-021-09386-5 . Springer; [cited 29 Jul 2021].
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46. Available from: http://epm.sagepub.com
Wongpakaran N, Wongpakaran T, Wedding D, Gwet KL. A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: A study conducted with personality disorder samples. BMC Med Res Methodol. 2013;13:1–7.
Brennan RL, Prediger DJ. Coefficient kappa: Some uses, misuses, and alternatives. Educ Psychol Meas. 1981;41:687–99.
Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61:29–48.
Gwet KL. Handbook of Inter-Rater Reliability: the definitive guide to measuring the extent of agreement among raters. 4th ed. 2014. Adv. Anal.
Solans-Domènech M, MV Pons J, Adam P, Grau J, Aymerich M. Development and validation of a questionnaire to measure research impact. Res Eval. 2019;28:253–62.
Tammaa A, Fritzer N, Lozano P, Krell A, Salzer H, Salama M, et al. Interobserver agreement and accuracy of non-invasive diagnosis of endometriosis by transvaginal sonography. Ultrasound Obstet Gynecol. 2015;46:737–40.
Buss I, Genton B, D’Acremont V. Aetiology of fever in returning travellers and migrants: A systematic review and meta-analysis. J Travel Med. 2020;27:1–12.
Angelo KM, Kozarsky PE, Ryan ET, Chen LH, Sotir MJ. What proportion of international travellers acquire a travel-related illness? A review of the literature. J Travel Med. 2017;24:1–8.
Liu W, Hu W, Dong Z, You X. Travel-related infection in Guangzhou, China, 2009–2019. Travel Med Infect Dis. 2021;43:102106 (Elsevier Ltd). Available from: https://doi.org/10.1016/j.tmaid.2021.102106
Pavli A, Lymperi I, Katerelos P, Maltezou HC. Knowledge and practice of malaria prophylaxis among travel medicine consultants in Greece. Travel Med Infect Dis. 2012;10:224–9. Elsevier Ltd Available from: http://dx.doi.org/10.1016/j.tmaid.2012.09.006
Namikawa K, Kikuchi H, Kato S, Takizawa Y, Konta A, Iida T, et al. Knowledge, attitudes, and practices of Japanese travelers towards malaria prevention during overseas travel. Travel Med Infect Dis. 2008;6:137–41.
Al-Abri SS, Abdel-Hady DM, Al-Abaidani IS. Knowledge, attitudes, and practices regarding travel health among Muscat International Airport travelers in Oman: Identifying the gaps and addressing the challenges. J Epidemiol Glob Health. 2016;6:67–75. Available from: https://doi.org/10.1016/j.jegh.2016.02.003 (Ministry of Health, Saudi Arabia).
Della Polla G, Pelullo CP, Napolitano F, Lambiase C, De Simone C, Angelillo IF. Knowledge, attitudes, and practices towards infectious diseases related to travel of community pharmacists in italy. Int J Environ Res Public Health. 2020;17(6):2147.
Flaherty GT, Leong SW, Finn Y, Sulaiman LH, Noone C. Travellers with type 1 diabetes: Questionnaire development and descriptive analysis of knowledge and practices. J Travel Med. 2021;27:1–8.
Goesch JN, Simons De Fanti A, Béchet S, Consigny PH. Comparison of knowledge on travel related health risks and their prevention among humanitarian aid workers and other travellers consulting at the Institut Pasteur travel clinic in Paris France. Travel Med Infect Dis. 2010;8:364–72.
Ratnam I, Torresi J, Matchett E, Pollissard L, Luxemburg C, Lemoh CN, et al. Development and validation of an instrument to assess the risk of developing viral infections in Australian travelers during international travel. J Travel Med. 2011;18:262–70.
Wilson ME. Geography of Infectious Diseases. In: Jonathan Cohen, William G. Powderly, Opal SM, editors. Infect Dis (Auckl). 4th ed. United States: Elsevier; 2017. p. 1055–64.
Wu X, Lu Y, Zhou S, Chen L, Xu B. Impact of climate change on human infectious diseases: Empirical evidence and human adaptation. Environ Int. 2016;86:14–23.
Authors are thankful to the experts who provided that ratings of the questions in terms of Necessity, Clarity and Relevance.
This research work has not received any funding from any of the funding sources.
Ethics approval and consent to participate
Study was ethically approved by All India Institute of Medical Sciences (AIIMS) research ethics committee (Ref. No.: IECPG-326/22.07.2020, RT-34/26.08.2020). This study has not been conducted on human participants, and therefore consent to participate do not apply.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kumar, A., Rajendran, A., Usman, M. et al. Development and validation of a questionnaire to evaluate the knowledge, attitude and practices regarding travel medicine amongst physicians in an apex tertiary hospital in Northern India. Trop Dis Travel Med Vaccines 8, 13 (2022). https://doi.org/10.1186/s40794-022-00170-w