Public health informatics training in New Mexico *

The information needs and information-seeking behavior of public health practitioners has not been researched extensively. Exploratory investigations of the information needs of the US public health workforce, however, recommend providing greater awareness of and training in accessing public health information resources [1–4]. 
 
This case study describes the University of New Mexico (UNM) Health Science Library and Informatics Center's evidence-based public health (EBPH) informatics training for public health practitioners in New Mexico. The training sessions, intended both to raise awareness about and to provide hands-on searching experiences related to EBPH, occurred from August 2005 through March 2006.


INTRODUCTION
The information needs and information-seeking behavior of public health practitioners has not been researched extensively. Exploratory investigations of the information needs of the US public health workforce, however, recommend providing greater awareness of and training in accessing public health information resources [1][2][3][4].
This case study describes the University of New Mexico (UNM) Health Science Library and Informatics Center's evidence-based public health (EBPH) informatics training for public health practitioners in New Mexico. The training sessions, intended both to raise awareness about and to provide hands-on searching experiences related to EBPH, occurred from August 2005 through March 2006.

TARGET POPULATION
All professional-level New Mexico Department of Health (DoH) employees throughout the five administrative regions of New Mexico were eligible to participate in this informatics training opportunity. Table  1 provides a summary of the eighty-seven training participants' diverse occupations.

RECRUITMENT ISSUES
New Mexico is the 5th largest state in the United States and has the 6th lowest population density. Many of the 1.8 million people who live in New Mexico either reside in large population centers such as Albuquerque or in much smaller towns separated by vast deserts and mountains. Harding County in the rural Northeastern quadrant of the state, for example, only has 0.35 people per square mile [5]. The geographic isolation of some far-flung populations has serious implications for the provision of public health and library or informatics services. New Mexico ranks 45th in median household income nationally according to a 2003 US Census report [6]. Low income levels generally affect tax revenues, which translate into resource constraints for agencies providing needed public health services. Budget deficiencies had been perennial and persistent when this training program commenced during 2005. One of the most telling constraints pertained to the lack of training opportunities for continuing education: for several years prior to this training, professionals in two of the five regions had been unable to participate in quarterly continuing education * Funded by the National Library of Medicine (NLM) under NLM contract N01-LM-1-3515 with the Houston Academy of Medicine-Texas Medical Center Library.
Supplemental Appendixes A and B are available with the online version of this journal. opportunities sponsored by DoH in Albuquerque due to limited budgets [7].
Beyond these obstacles, recruiting a critical mass of participants in the project became difficult due to two further unexpected challenges. First, a new Web-based state government email system replaced the previous client email system just as the initial recruitment phase began during July/August 2005. State employees found themselves trying to learn a new email system without any formal instruction, and consequently many did not notice the announcements for the training sessions. The same email system generally failed to function for nearly a two-week period during the next major recruitment period in January 2006. Second, Hurricanes Katrina and Rita (September/October 2005) presented further obstacles to recruitment and retention in the training project as DoH professionals were responding to the immediate needs of evacuees brought to New Mexico, were being deployed to the Gulf region, or were being reassigned to fill gaps caused by the deployments of their colleagues [8,9]. Smaller scale crises, due either to emerging health conditions in certain locales or already stretched-thin personnel resources interfered with not only recruitment, but also the actual participation in the scheduled training sessions.

PARTNERS
The first author co-taught a course for the UNM Masters of Public Health Program during 2002 with the chief medical officer of the New Mexico Department of Health, who urged the author to find a means to train the DoH workforce in many of the same informatics skills taught to master's of public health (MPH) students in the course. This long-term collegial relationship later resulted in a smooth implementation of the EBPH training program. The chief medical officer oriented the DoH's regional directors to the project and secured their support, thereby making communications with these colleagues very productive. The full support of these administrators proved to be essential to the success of this project due to the large DoH organization with employees dispersed across the state. The regional directors or their designees scheduled classrooms and centralized communications for the training sessions in their regional offices. These key administrators also publicized the training sessions and collected the training participants' signed human subject consent forms required by the authors' institutional review board.

INTERVENTIONS
Assessments employed to better understand the target workforce Needs assessment of DoH professionals' informatics training requirements prior to this project was based

Figure 1
Lessons learned Ⅲ The use of focus groups involving both public health faculty and graduate students with substantive public health work experience provided invaluable information that led to revisions in how the authors recruited and trained department of health (DoH) professionals. Ⅲ Recruitment emails for training sessions should include catchy subject lines to capture attention, describe the content as both relevant and useful, and provide concrete skills to be learned. Recruitment efforts also should use key phrases such as ''community-based,'' ''competencies,'' and ''continuing education opportunity.'' Ⅲ The full support of the regional directors or their designees provided essential support for this project, particularly in supplementing communications due to the problematic email system. Ⅲ The authors were impressed on several occasions by the outstanding skills demonstrated by New Mexico DoH professionals while in crisis management mode. Decision making in public health frequently occurs under crisis mode conditions [11,12]. The downside of this mode of operation, however, was that DoH professionals had to handle multiple crises, which sometimes made it difficult for them either to review communications or participate in scheduled training sessions. Ⅲ The use of both email and telephone call reminders increased training session attendance. Ⅲ New Mexico DoH professionals learned best through frequent hands-on exercises during the training sessions. Ⅲ The use of $20 gift cards might have served as a useful preliminary incentive to participate in the project for many enrollees, but the training itself became a primary incentive by the completion of the training sessions, as evidenced by the anonymous follow-up evaluation results. Ⅲ New Mexico DoH professionals reported placing higher value on receiving generic ''Certificates of Completion'' than on continuing education credit from professional associations because the certificates documented training for potential extra compensation in annual performance review processes.

Figure 2
The evidence-based public health process 1. Formulating and refining effective evidence-based public health (EBPH) questions 2. Searching for the best evidence to answer EBPH questions 3. Critically appraising the evidence for relevance and validity in answering the EBPH question 4. Determining if the course of action with the best evidence will be cost effective 5. Evaluating the utility of the best evidence in making this decision on the first author's experiences in co-teaching members of this population enrolled in the UNM MPH program and from his fifteen years' service as a full voting member of the program's curriculum committee. This anecdotal needs assessment was supplemented further during the proposal preparation stage by discussions with high-level DoH administrators. The funded project included two separate focus groups of UNM MPH program faculty members (n ϭ 3) and graduate students with extensive public health work experience (n ϭ 4) conducted by an expert facilitator. The focus groups concentrated on determining the methods for recruiting potential participants, collecting data, and securing participant feedback on the training materials. These two focus groups produced several recommendations (Figure 1): (1) utilize specific terms or phrases when publicizing the training to boost recruitment, (2) issue a generic ''Certificate of Completion'' for the training rather than securing continuing education credit through various professional associations because New Mexico DoH employees value documenting attendance to support possible increased salaries in their annual performance review processes far more than association credentialing, and (3) offer the incentive of a department store gift card to all participants for completion of the training program.

Training
The principal intervention consisted of a three-hour EBPH informatics training session offered to participants at regional centers throughout the state. Each session consisted of several major segments. First, the trainer introduced the overall five-step EBPH process ( Figure 2) lending special emphasis to question formulation and critical appraisal. Next, participants engaged in two exercises involving critical appraisal of informational Websites and statistical Websites. Each session participant then received in-depth PubMed training. The sessions ended with a review of a list of free high-quality public health journals available on the Web.

EVALUATION
As an experienced teacher and instructor, the first author monitored audience real-time feedback during the training sessions and altered the instruction as needed. This training project included two additional formal, structured evaluation activities inspired by the NLM resource, Measuring the Difference [10]. First, participants completed anonymous evaluation forms immediately following their training sessions (Appendix A). Approximately two months later, participants were asked to complete a second anonymous evaluation form, which asked them to reflect on the long-term value of their training session (Appendix B).

Participants
A total of 103 DoH professional-level employees initially enrolled for 1 of the 11 training sessions, but, due to attrition caused primarily by the hurricanes, 87 were actually trained. Many of the participants in the sessions were from remote areas of New Mexico. Fig-Table 2 Summary of key questions on evaluations: collected immediately following the training Score The instructor presented material at a level appropriate for me 4.8* The instructor was able to answer questions clearly and concisely 4.8 The progression of the session was logical and coherent from beginning to end 4.8 I now am more aware of useful statistical and information Websites with relevance to public health 4.7 I am now more proficient in searching for evidence in PubMed 4.7 * Mean level of agreement scale (1 ϭ lowest; 5 ϭ highest).

Table 3
Summary of key questions on evaluations: collected two months following training

Score
Since my training session, I am more confident I will find an answer to these questions (questions raised related to job duties) 4.3 Since my training session, I am now aware of the strengths and weaknesses of different information resources 4.3 I would recommend this training to a colleague 4.5 ure 1 highlights significant lessons learned from the project.

Evaluation results
Results immediately following the first few training sessions clearly recommended increasing the length of the PubMed searching segment of the sessions. Furthermore, DoH professionals were typical adult learners as reflected in their desire to have more practical hands-on experiences during the training. The training materials evolved to reflect the changed emphasis on PubMed and the predominant adult learner pedagogical framework. Table 2 lists highlights from the 83 immediate evaluations, as indicated by mean ratings on a 5-point Likert scale (1 ϭ low agreement to 5 ϭ high agreement). Approximately 2 months after each training session, all participants were sent 2-page follow-up evaluation forms to discover what content from the training sessions had proved most useful. The 68 respondents (Table 3) gave high scores related to their increased confidence in EBPH searching, their more critical approach to information resources, and recommendation of this training to a colleague.

FUTURE DIRECTIONS
During July 2006, the DoH contacted the first author to contract for additional EBPH training sessions covering the identical subject content. A lack of funds since then has delayed final planning and implementation of this new project. Although still under discussion, these trainings probably would occur at eight sites around the state.
The authors have made the final report, the twentytwo-page training handbook, and the evaluation forms available via DSpace [13][14][15] to facilitate rapid diffusion of the knowledge gained from the EBPH project. This project builds on previous exploratory studies by further validating the perceived need among public health practitioners to acquire skills to improve their access to relevant information resources.

ACKNOWLEDGMENTS
The authors thank their colleagues at the UNM Health Sciences Library and Informatics Center (HSLIC) for their support for this project. In particular, they thank Associate Vice President for Knowledge Management Holly Buchanan, AHIP, and HSLIC Deputy Director Janis Teal, AHIP, for their support. They also thank administrative staff members Cathy Brandenburg, Marcia Sletten, MaryLou Seyl, and Natalia Koup for their assistance.
DoH Chief Medical Officer Ron Voorhees provided essential support from his agency. This project could have never enrolled so many prospective trainees and the training would have never proceeded so smoothly without the continuous involvement of the regional directors or their designees: Karen Armitage, Debra Belyeu, Corazon Halasan, Eugene Marciniak, and Margy Wienbar.
Finally, the authors appreciated all of the encouragement provided to this project from National Network of Libraries of Medicine South Central Region Administrators Renée Bougard and Michelle Malizia.