Targeted drug discovery and development, from molecular signaling to the global market: an educational program at New York University, 5-year metrics

Drug discovery and development (DDD) is a collaborative, dynamic process of great interest to researchers, but an area where there is a lack of formal training. The Drug Development Educational Program (DDEP) at New York University was created in 2012 to stimulate an improved, multidisciplinary DDD workforce by educating early stage scientists as well as a variety of other like-minded students. The first course of the program emphasizes post-compounding aspects of DDD; the second course focuses on molecular signaling pathways. In five years, 196 students (candidates for PhD, MD, Master’s degree, and post-doctoral MD/PhD) from different schools (Medicine, Biomedical Sciences, Dentistry, Engineering, Business, and Education) completed the course(s). Pre/post surveys demonstrate knowledge gain across all course topics. 26 students were granted career development awards (73% women, 23% underrepresented minorities). Some graduates of their respective degree-granting/post-doctoral programs embarked on DDD related careers. This program serves as a framework for other academic institutions to develop compatible programs designed to train a more informed DDD workforce.


Introduction
There is a revived optimism despite many challenges in drug discovery and development (DDD) [1][2][3]. DDD educational programs can influence growth and development of novel therapies and market transition [4]. Due to escalating complexity and feasibility barriers, there is a significant need for DDD educational programs. The median cost and time to develop a potential molecule from its discovery to approval is 2.6 billion dollars and 36 years, respectively [5,6]. Only 10 to 20% of health-related research projects reach human trials; only 6% of drugs that enter phase I trials reach approval status [7,8]. The new drug approval rate has remained low [9,10]. Therefore, to promote successful translation in DDD, scientists need skills and training [11]. Few DDD training programs integrate aspects on both discovery and development and expose students to opportunities and experts in the field [12].
In September 2012 with funding from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), we began enrollment for the Drug Development Educational Program (DDEP) at New York University (NYU). We aim to teach graduate and postgraduate students the essentials of DDD and inspire these motivated and talented individuals to get more involved in the translation of scientific discoveries and enter the DDD workforce as trained scientists, engineers, business analysts, and entrepreneurs. This improved, informed DDD workforce will be better equipped to address the nation's biomedical and clinical research needs.
Our educational objectives were met, consistent with reported data from our two-year analysis [13]. Furthermore, we explored why students enrolled in each course, what they learned, and the course's relevance to their career goals. We also traced the career paths of students who received DDEP's career development awards. DDEP aims to teach essential skills and is the only known program of its kind within the Clinical and Translational Science Institutes (CTSIs).

Description of course series
The NYUSOM DDEP was created in 2012 with support from NIDDK. Currently in its sixth year, the DDEP was initially comprised of a two-course series designed to highlight the essential and innovative features of the DDD process. Our goals are: 1) to teach a new generation of multidisciplinary students at the graduate and post-graduate levels about DDD; 2) to potentiate and expedite the translation of therapeutics and other applications, both novel and repurposed; and 3) to bridge the translational gap.
term and focuses on the post-compounding aspects of DDD such as intellectual property, regulatory aspects, and marketing. The spring term course, Molecular Signaling in the Discovery and Development of Therapeutics (MSDDT), expands on molecular signaling pathways as potential sites for drug targets as well as viability testing (Table 1). Each course grants three credits. Classes are two hours in duration, where each begins with a lecture given by an expert in the field followed by an interactive discussion. Students are required to arrive prepared, having completed the assigned reading and coursework.

DDNE (fall) course
The goal of the DDNE course is to provide an overview of the regulatory, economic, ethical, and business aspects of DDD from the preclinical phase through post-approval marketing and monitoring. Students are exposed to a wide range of topics including protocol planning, safety monitoring, cost and pricing analysis, and intellectual property. Students also learn how basic and clinical sciences, statistical analysis, business management, legal, and marketing departments converge in this interdependent, multidisciplinary process. At the end of the term, students are required to complete a final project that entails submission of a "mini" Investigational New Drug Application (IND) [7].

MSDDT (spring) course
In the MSDDT course, the principles of discovering and developing therapeutics in the lab as well as essential signaling pathways such as RAS and AKT are explored. Students learn about entity-specific paradigms that can help predict successful DDD trajectories and how to plan target selection via experimental testing. To ensure students are ready for the different scientific aspects of DDD, students are exposed to topics such as developing receptor and pathway networks, prediction models, new technologies, and challenges of designing animal models and clinical trials. Students are required to submit a final paper on research methods or approaches used to design therapy [8].

Student demographics
Demographic information, training level, and graduate school affiliation was collected from students who enrolled between Fall 2012 and Spring 2017.

Program evaluation
Pre/Post course surveys and analysis-To evaluate students' self-reported contextual knowledge, each student completed a pre course survey on the first day of the course and a post course survey on the last day of the course. The surveys asked students to rate how well they understood each of the topics/knowledge items. The response options were: (1) nothing, (2) almost nothing, (3) some and (4) a great deal. Responses to all knowledge questions were summed to create a total knowledge score. Each survey also included a question about the relevance of the course in terms of career goals. The choices were: (1) not at all relevant, (2) only a little relevant, (3) somewhat relevant and (4) very relevant. IBM SPSS was used to analyze the differences between the pre and post course surveys. Due to small sample size and non-normally distributed data, Wilcoxon Ranked Sum Test was used for all data analyses.
Lecture ratings-On the last day of each course, students received surveys asking them to rate each of the lectures in 4 different domains: content, presentation, relevance, and overall. The options for each response were: (1) poor, (2) lower than expected, (3) satisfactory, (4) above expectations, (5) superior. The response to the question "Was the lecture free of commercial bias?" was also analyzed.
Free-response text-To capture more qualitative information, each survey contained free-response questions. In the pre course survey, students were asked, "What are the main things you hope to get out of this course?" They were also asked to explain their rating for the career relevance question. In the post course survey, students were asked, "What are the main take home points you got out of this course?" IBM SPSS Text Analytics for Surveys was used to analyze the answers to free-response questions. The responses were grouped into categories for query and associated with keywords the students utilized in the response. We manually excluded words that contained components of search terms. Likewise, some responses that were not automatically grouped into the corresponding categories were grouped manually based on content. For the questions "What are the main things you hope to get out of this course" and "What are the main take home points you got out of this course", Chi-square test was used to compare the differences between the pre and post course surveys.
Career development award-Enrolled students were eligible for a career development award. The goal of the award was to provide support for students to pursue activities to advance their careers. Uses included tuition remission, workshops, course/conference attendance, and publication support. Online search and telephone follow-up were used to track recipient's career trajectories.

Student demographics
From September 2012 through June 2017, 139 students completed the DDNE course, and 75 students completed the MSDDT course. A total of 196 students completed at least one course in the DDEP; 18 (9%) students completed both DDNE and MSDDT.

Program evaluation DDNE (fall) course
Pre/Post course surveys: 139 students completed DDNE between September 2012 and June 2017. Of those, 127 (91%) completed the pre course survey, 98 (71%) completed a post course survey, and 89 (64%) completed both pre and post course surveys. Responses to all the individual knowledge questions, the total knowledge score, and the career relevance question were not normally distributed (pre course total knowledge score: Kolmogorov-Smirnov p = 0.006 and Shapiro-Wilk p = 0.114; all others: Kolmogorov-Smirnov p ≤ 0.001 and Shapiro-Wilk p ≤ 0.001). Students rated themselves significantly higher in each of the knowledge areas as well as overall knowledge after taking the course. The mean pre course career relevance (3.59 ± 0.57) was not different from the post course career relevance (3.51 ± 0.62) (Z=−0.480; p = 0.631) ( Figure 2A, Table 2).
Free response text: 113 (89%) of the 127 students who completed the pre course survey explained their career relevance ratings. 66 (58% of the 113 responses with explanations) indicated interest in working in research and DDD related careers. 40 (35%) indicated interest in a career outside academia (Table 3).
In the pre course survey, 94 (74% of the 127 responses) students responded to the question "what they hope to get out of the class." 35 (37% of the 94 responses to this question) indicated they were interested in the business and/or marketing aspects of DDD, 14 (15%) were interested in learning about the regulatory aspects, and 1 (1%) indicated interest in intellectual property (Table 4). In the post course survey, 92 (94% of the 98 responses) students answered, "what were the main takeaways of the class." 31 students (34% of the 92 responses to this question) indicated they learned about the business and marketing aspects of DDD, 20 (22%) said they learned about regulatory aspects, and 13 (14%) learned about intellectual property. In the post course surveys, there is a significant increase (χ 2 p = 0.034) in the number of responses that include keywords in the intellectual property category (Table  4).

MSDDT (spring) course
Pre/Post course surveys: 75 students completed the MSDDT course over five years, 68 (91%) completed the pre course survey, 52 (69%) completed a post course survey, and 46 (61%) completed both pre and post course surveys. Responses to all the individual knowledge questions, the total knowledge score, and the career relevance question were not normally distributed (pre course total knowledge: Kolmogorov-Smirnov p = 0.004 and Shapiro-Wilk p = 0.005; post course total knowledge: Kolmogorov-Smirnov p = 0.004 and Shapiro-Wilk p = 0.226; all the others: Kolmogorov-Smirnov p ≤ 0.001 and Shapiro-Wilk p ≤ 0.001). As was the case in DDNE, students rated themselves significantly higher in each of the knowledge areas as well as total knowledge score after taking the course. Similarly, there was no difference between the pre and post course career relevance ratings (pre course: 3.64 ± 0.55; post course: 3.40 ± 0.85; Z = −1.206, p = 0.228) ( Figure 2B, Table 2). Free response text: 55 (81%) of the 68 students who completed the pre course survey explained their career relevance ratings. A majority of responses (37, 67% of the 55 responses with explanations) indicated interest in working in research and DDD related careers. 17 (31%) students indicated interest in a career outside academia (Table 3).
In the pre course survey, 63 (93% of the 68 responses) responded to the question, "what they hope to get out of the class." 32 (51% of the 63 responses to this question) indicated they were interested in learning how to discover a drug, 9 (14%) were interested in the business and/or marketing aspects of DDD, 2 (3%) were interested in learning about the regulatory aspects (Table 4). In the post course survey, 47 (90% of the 52 responses) students answered "what were the main takeaways of the class." 24 (51% of the 47 responses to this question) students indicated they learned how to discover a drug, 9 (19%) learned about the business and marketing aspects of DDD, 3 (6%) learned about regulatory aspects, and 1 (2%) learned about intellectual property (Table 4).

Where are the students who received career development support after graduation from their primary matriculated program/appointment?
The five graduated medical students who received support are all currently enrolled in US residency programs. Two of the eight PhD students are now graduated; one has a position as a post-doctoral trainee and one is a consultant at a healthcare company. Two Master's degree students graduated; one is a project coordinator at a healthcare company and the other is a clinical trials research coordinator. All four post-doctoral MD students are now assistant professors at academic institutions. One of the seven post-doctoral PhD students is a manager at a pharmaceutical company; the other six continue their post-doctoral training ( Table 5).

Discussion
Our program continues to attract talented and motivated students from a variety of backgrounds at an early career stage. Initially, enrollment included students from Schools of Medicine, Biomedical Sciences, and Arts and Sciences. Currently, the student base has expanded to include the Schools of Dentistry, Engineering, Business, and Education. Moreover, our lecturers have diverse, yet specific, expertise from the various realms of DDD including academia, government/regulatory agencies, industry (large and small), and the business/economic sector. As a result, the lectures and discussions provide a platform for the multidisciplinary students to exchange ideas and share experiences toward a common goal.
The students who received career development awards are arguably our most enthusiastic students. Funds were used most often for conference attendance, which shows that recipients have strong interests in networking and becoming more fluent in their field of interest. Some of these awardees have embarked on careers in competitive medical fields and others are directly involved in DDD. It has been shown that MD/MSCI students apply to more competitive residency programs, and the current institutions and specialties of our MD/ MSCI graduate recipients reflect this tendency [9]. Our program is successful in preparing these students to be key players in DDD. In addition, our career development funding encourages URM students. 23% of our recipients are URMs, while only 8.3% of all science, technology, engineering and mathematics PhDs are URMs [10]. Likewise, women have historically been underrepresented in science and industry but 73% of our career development recipients are women [11].
In the first two years, there were 37 and 23 students in DDNE and MSDDT, respectively [6]. After five years, these numbers have increased to 139 and 75, respectively, reflecting increased annual enrollment in both courses without compromising the quality of lectures and discussions. With a larger sample size, we found consistently high lecture ratings in both courses, and students demonstrated learning in all areas queried. The high post course survey knowledge ratings suggest that our course can give students the knowledge, networking abilities, and confidence needed to successfully participate in DDD projects. In addition to gained knowledge, we can ascertain from the free text responses to "what students got out of the course" that students learned about the regulatory, business, economic, marketing, and intellectual property aspects. Furthermore, in regards to the latter, the number of responses that contained the keyword intellectual property increased significantly in the post-course DDNE survey. This is likely due to the course's highly rated lecture on intellectual property [14][15][16][17][18][19].
Regarding career relevance, the majority of students (58% DDNE, 67% MSDDT) stated a desire to pursue a research and/or DDD related career. Approximately 1/3 of students indicated interest in working outside academia. Current trends in PhD training programs show that only 12.8% of graduates ultimately end up in academic careers [12]. Additional educational programs such the DDEP are important programs to enable students to gain skills and potentiate the DDD workforce.
The authors acknowledge that since there are no final exams (post-test) in our course series (and we did not administer a pre-test), we do not have a quantitative measurement of knowledge gained. The career trajectories of only those students who received the career development awards were examined.

Conclusion
Our DDEP program is successful in teaching DDD to students at an early career stage. Despite the fact that our students are at different levels of training and from different schools/fields, the material is appropriate and relevant. By inviting speakers from different areas of DDD and maintaining a discussion-based forum with multidisciplinary students, our program is able to provide collaborative, yet efficient training. We hope to produce highly trained scientists, engineers, educators, business analysts, and other innovators in DDD to further fuel the process. We will continue to expand the program. The next expansion will include an additional course, Biotechnology Industry: Structure and Strategy. The NYU Drug Development Educational Program serves as a framework for other institutions to develop similar, compatible programs to train early stage researchers in this critical area.   This bar graph displays the number of career development awardees categorized by gender, training level, underrepresented minority status, and purpose of use. Of note is that some students used career development funds for more than one purpose.    Table 3 Free text responses -Career Relevance Question.

Drug Development in a New Era (DDNE)
Total Enrollment 139