A Comparison of Two Methods of Needs Assessment: Implications for Continuing Professional Education

Needs identification is an important component of program planning in continuing professional education. Learners, professional associations, and society all have a stake in ensuring that programs are relevant and focused on important educational needs of professionals. This study compared two different methods of identifying learning needs--perceived needs and knowledge-based needs--for a group of practicing pharmacists (N= 113). The Canadian Consensus Asthma Management Guidelines (1996) provided the framework for the needs assessments and the standard against which pharmacists' knowledge of asthma treatment was assessed. Using data collected via a questionnaire, rank correlation tests showed no relationship between perceived needs and knowledge-based needs. While there was correspondence between the two methods on a few items, overall they did not identify the same needs. This confirmed the results of other research that there are some educational needs of which learners are unaware. Even with the limitations of perceived needs, few continuing professional educators would advocate abandoning this method, although most advocate a combination of methods. The following article discusses the implications of these and other research findings, and current literature on needs assessment in continuing professional education. Many questions remain, however, and there is a need for more research on needs assessment in continuing professional education.


RÉSUMÉ
La détermination des besoins est un élément important dans la planification de programmes en formation professionnelle continue. Les apprenants, les associations professionnelles et la société ont tous intérêt à s'assurer que les programmes soient utiles et qu'ils ciblent les besoins éducatifs importants des professionnels. Cette étude compare deux différentes méthodes de détermination des besoins en matière d'apprentissage-les besoins perçus et les besoins basés sur la connaissancepour un groupe de pharmaciens praticiens (N=113). Les Canadian Consensus Asthma Management Guidelines (1996) ont fourni le cadre pour l'évaluation des besoins ainsi Revue canadienne de l'éducation permanente universitaire Vol. 28, N o 1, printemps 2002 via a questionnaire, rank correlation tests showed no relationship between perceived needs and knowledge-based needs. While there was correspondence between the two methods on a few items, overall they did not identify the same needs. This confirmed the results of other research that there are some educational needs of which learners are unaware. Even with the limitations of perceived needs, few continuing professional educators would advocate abandoning this method, although most advocate a combination of methods. The following article discusses the implications of these and other research findings, and current literature on needs assessment in continuing professional education. Many questions remain, however, and there is a need for more research on needs assessment in continuing professional education.

INTRODUCTION
Continuing professional education is an important educational function of universities. Programs and services may be provided by professional faculties, continuing education/extension units, or partnerships of providers, often including both university and professional associations. Society and professionals have a stake in ensuring the quality, relevance, and appropriateness of continuing professional education (CPE) programs, and the identification of educational needs is generally acknowledged as an important part of planning these programs. Educational needs must be determined with accuracy if learning experiences are to focus on the real educational problems of potential learners.
This article is based on a study that used two different methods to investigate the CPE needs of practising pharmacists and then compared the two sets of needs they identified. The implications of these and other research findings are discussed, as is the current literature on needs assessment in CPE for continuing education providers.

NEEDS AND NEEDS ASSESSMENT IN CONTINUING PROFESSIONAL EDUCATION
Before addressing the concepts of "needs" and "needs assessment," it is useful to situate these concepts within the larger context of the program planning literature. This literature can be seen as falling into two categories: the technical-rational and the political (McLean, 2000;Sork, 2000). In the technical-rational tradition-with its roots in Tyler's (1949) work and its focus on the techniques of planning-needs assessment is a discrete step. In contrast, writers from the political domain, with their base in critical theory, see program planning not as a rational decision-making model, but as a political enterprise in which the negotiation of power and interests is paramount. Cervero and Wilson (1994) are standard-bearers for this approach. One case study included in their 1994 book (chap. 4) describes the way in which the planning actually proceeded for an annual updating seminar for practising pharmacists.
Currently, these two categories are not necessarily seen as "either-or" positions. Authors who base their work in the technical-rational tradition recognize needs assessment as a specific example of a more general phenomenon. For example, Caffarella's program planning model (1994) included "identifying program ideas" as one component, with the identification of educational needs being just one source of program ideas. Similarly, Sork (2000Sork ( , 2001 used the term "justify and focus planning" in his planning framework. Needs assessment is one approach the program planner can use Revue canadienne de l'éducation permanente universitaire Vol. 28, N o 1, printemps 2002 to accomplish this step, and Sork (2001) provided examples of other tools that can also be used for this purpose, including interest inventory, market test, problem analysis, and trend analysis. Three "domains," which he views to be of equal importance, underpin Sork's (2000) planning framework: the technical, the sociopolitical, and the ethical. Pearce's (1998) approach emphasized the importance of professional judgement in program planning, including judging whether the question of conducting a needs assessment is even asked, let alone usually answered in the positive. McLean (2000) used both approaches to analyze his experience of planning a major program, concluding that both are needed. In his view, using only one or the other would lead to an incomplete understanding of the planning process.
Nonetheless, continuing professional educators, particularly in the health professions, usually locate themselves closer to the technical-rational tradition than to the political-power approach as they have considerable interest in identifying the learning needs of their clientele. A common (perhaps the most common) definition of "need" is that of a gap between "what is" and "what should be" (Queeney, 2000;Witkin & Altschuld, 1995). Sork built on that definition to define an educational need as "a gap or discrepancy between a present capability (PC) and a desired capability (DC)" (2001, p. 101, italics in original); he defines capability as "any human quality that can be altered through learning," commonly knowledge, skills and attitudes (p. 101). Thus, according to Sork, an educational needs assessment is the process of "identifying gaps in the capabilities of adult learners" (p. 101). In planning professional development programs for health professionals, this conceptualization of need is an easy fit. (For other discussions of defining the concept of need, see, for example, Beatty, 1981;Monette, 1977.) In the process of defining terms, it is easy to fall into the trap of reifying the concept under discussion. As well, the uncritical use of the technicalrational program planning model can result in an over-emphasis on the techniques of needs assessment. These two pitfalls can lead the programmer to believe that educational needs are objective conditions that actually exist in the world, waiting to be identified. In fact, a need exists only when a subjective act-a value judgement-is made (Sork, 1998(Sork, , 2001. As Sork (2001) states: Whether felt or prescribed, value judgements must be made before a need exists. First, someone must conclude that a present capability is inadequate. Second, someone must describe the desired capability. In both cases these decisions are influenced by  Davidson (1995) presented the case that needs are, in fact, created. He provided a conceptual framework that extends our thinking about the role of the adult educator from that of "needs-meeting" to "needs-making," which he defines as the "processes by which conditions are experienced, expressed, and satisfied as educational needs" (p. 194). He contended that while adult education programmers do not need to discard needs assessment, they do need to engage in it more critically. As well, Collins (1991) and Cervero and Wilson (1994) alerted us to the role that power plays in the process of determining, among other things, whose needs are addressed in adult education programs.
"Needs-making" in CPE occurs in a setting in which "the need to know more," to use Davidson's (1995) term, is highly valued, and where partnerships between universities, professional associations, and, on occasion, regulatory bodies and commercial interests are formed to establish the "what should be's." These "desired" states or capabilities are profoundly influenced (some would say driven) in the health professions by new developments in technology and scientific knowledge, which bring with them new possibilities. In some ways, what is possible becomes what is desirable. Our society places high value on having health professionals who "keep up to date," and it is hard to argue that having current knowledge of treatment products and protocols is not important. And so, this is the context in which continuing professional educators must come to grips with how to determine what professionals "need" to learn.
The method(s) used to identify needs are shaped by, and in turn shape, the way in which the concept of "need" is defined. Perhaps the most frequently repeated statement in the needs assessment literature is that there is no one right way to conduct a needs assessment. In Queeney's (1995) words, "The key to successful assessment is identifying a method appropriate to the issue and to one's goals and resources, and implementing it well" (pp. 5-6).
Various methods are used to identify educational needs (i.e., needs that can be met in whole or in part by learning experiences), ranging from asking people what they perceive their needs to be to extensive and sophisticated assessments in professional practice settings. In continuing pharmacy education, typical needs assessments have focused on "what pharmacists feel they need to learn," usually asked through a questionnaire. Needs identified by such methods are referred to as "perceived needs" or "self-identified needs." A typical example is Richards & Blank's (1997) study to assess the educational needs of practising pharmacists in two areas: drug utilization review and patient counselling. Pharmacists were asked to rate their degree of confidence in their skills and knowledge in each area and to suggest the three most important topics for continuing education programs. Another example is Bilger and Chereson's (1994) study of needs related to a re-entry/career change program for pharmacists, in which respondents were asked to rate their preference for topics and course formats for refresher courses. There are also a few examples of studies in which knowledge tests were designed to identify needs (Johnson, Lemberger, McCormick, Smith, & Balanoff, 1980;Shannon & Weinswig, 1978). Such needs are referred to as "prescribed needs" or "assessed needs." Queeney (1995) discussed the importance of both types of needs, as follows: Perceived needs are those needs that individuals believe they have; assessed needs are identified through a structured assessment process. This pair of needs is the most important to the concept of needs assessment, for if no difference existed between needs individuals perceive they have and those identified through assessment, the role of needs assessment would be limited to asking individuals to identify the discrepancies between their current levels of knowledge, skills, or performance abilities and those they wish to attain. However, most often individuals' perceived needs have been found to differ substantially from needs identified through continuing educators' assessment. (pp. 82-83) Comparisons of needs identified by different methods have not been widely undertaken in continuing pharmacy education. Therefore, this study was designed to compare a commonly used perceived needs assessment and a more objectively derived knowledge-based needs assessment.

Comparing Needs Assessed by Different Methods
Several researchers have examined the relationships among needs identified by different methods with varied results. Misskey, Moss, Lee, and Hill (1985) found no relationships between mothers' felt needs and prescribed needs identified by two methods: mothers' nutrition knowledge and preschoolers' eating habits. Three studies of professionals had similar results. Hiemstra and Long (1974) found no correlation between physical therapists' perceived educational needs expressed on a mailed questionnaire and more objectively derived needs (which they called "real" needs) as evidenced by scores on multiple-choice tests of knowledge in the field. Lockyer, Hanley, Fidler, Toews, and Lysholm-Andrews (1998) studied the learning needs of family physicians in the management of osteoporosis, using three methods of needs assessment: a literature review (to locate needs assessments data), focus groups, and a questionnaire. They determined that although these three methods identified different learning issues, the methods were complementary, concluding that "the focus groups were useful in identifying the content to be examined. The survey determined the relative importance of the concerns identified and the preferred format for the content" (p. 185). A Continuing Professional Educational Development Project conducted at The Pennsylvania State University in the 1980s compared different methods of needs assessment (Queeney, 1995). In this study, representatives of several professions were asked to list the content areas in which they believed they required further education before they were subjected to rigorous practice simulations, so that their strengths and weaknesses could be objectively documented. Comparison of the results of the two activities indicated that the areas of greatest need identified through assessment often were absent from the participants' lists of perceived needs. Perhaps not surprisingly, participants most often believed that their greatest needs were for new knowledge or for refresher courses on aspects of their field with which they seldom worked. But the assessment exercises showed that some of the greatest discrepancies between current and desirable proficiencies were related to functions performed regularly. (Queeney, 1995, pp. 13-14) On the other hand, two studies found that different methods identified similar needs. Macdonald's (1977) study of nurses found a correlation between perceived needs (self-defined needs) and audit-based needs (based on patient outcome and nursing care). Odor (1982) examined the relationships between three indicators of continuing medical education needs of physician assistants: practice needs (i.e., how many patients were seen in a month in emergency or acute care in cardiology, respirology, neurology, etc.), test-derived needs, and self-expressed needs. Significant relationships were found between practice and self-expressed needs; test-derived and self-expressed needs; and practice and test-derived needs.
Thus, the literature does not provide a definitive answer to the question of the validity of self-identified needs, particularly with regard to continuing pharmacy education where very few comparative studies have been undertaken. This study was designed to provide a more complete answer to that question.

THE STUDY CONTEXT
The Continuing Pharmacy Education unit was established in the College of Pharmacy and Nutrition of the University of Saskatchewan (Saskatoon, Saskatchewan, Canada) in 1995 with support from the Saskatchewan Pharmaceutical Association (the regulatory authority for pharmacists in the province). Its mandate is to provide continuing education programming for all Saskatchewan pharmacists (who, at the time of the study, required 15 units [hours] of continuing education annually for re-licensure), and to this end, it offers a range of programs that are designed to meet pharmacists' learning needs. The unit plans and implements these programs in conjunction with external sponsors. Although program ideas come from unit staff, the Advisory Committee, and pharmacists' comments on program evaluation forms, the specific content of programs is usually generated by the presenters (i.e., the "experts"), based on their perceptions about what pharmacists might need to know. Most programs deal with updates of pharmaceutical and/or medical knowledge, or are reviews of new and/or current drug therapies and patient care issues.

Study Sample
The study sample consisted of two groups of pharmacists who attended continuing pharmacy education programs in Regina and Saskatoon, Saskatchewan, in 1996. All 1,030 pharmacists who were then licensed by the Saskatchewan Pharmaceutical Association received the program announcement, and those who attended the one-day program on "Pharmaceutical Care for Asthma Patients" were invited to participate in the study. Therefore, the sample of 113 pharmacists was considered a convenience sample.

Instrumentation
CPE providers frequently use "practice guidelines" as one base for planning programs (Lockyer, 1998;Moore, 1998), particularly when new or revised guidelines or consensus statements are first published. The asthma program was planned and the needs assessment measures for this study were designed using the Canadian Consensus Asthma Management Guidelines (1996), together with those found in other recent literature on asthma treatment. These served as the framework for the knowledge-based and perceived needs assessments and as the standard against which pharmacists' knowledge was assessed. It is in these Guidelines, which became the description of "desired capabilities," that the value judgements inherent in needs identification reside. (The term "consensus" suggests a process involving negotiating power and interests.) The Guidelines were developed in 1995 at the Canadian Asthma Consensus Conference, which was convened by the Asthma Committee of the Canadian Thoracic Society. They were then widely circulated among members of the Canadian Asthma Consensus Group and other interested organizations and medical practitioners in order to achieve agreement on the assessment and treatment of patients with asthma. Consensus guidelines are relatively common in the field of medicine as they represent a "best practice" model for the care of specific groups of patients. However, new guidelines that recommend changes to standards of care often necessitate the development of training opportunities for health professionals, including pharmacists, to ensure their adoption, and this was the case with the revised Asthma Management Guidelines published in 1996. (A further update occurred in 1999. See Boulet, Becker, Bérubé, Beveridge, & Ernst, 1999.) The data reported on in this article were collected via a questionnaire administered at the beginning of the educational program. (For a summary of the use of questionnaires in needs assessment, see Mann, 1998). The questionnaire was made up of three sections: Section I: Pharmacists' perceived needs for information on the topic of pharmaceutical care for asthma patients (21 items). In this section, pharmacists were asked to "indicate to what extent you feel you need information on each of these topics to practice pharmaceutical care for asthma patients." A five-point Likert scale was used for responses, with 1 indicating no need for information and 5 indicating high need. This instrument, originally developed by The Alberta Asthma Centre and The Faculty of Pharmacy and Pharmaceutical Sciences at the University of Alberta in 1994, was adapted with permission of the developers. The approach is based on the assumption that a topic for which respondents indicated a higher need for information represented a perceived greater gap between their present and desired capabilities, that is, a greater educational need.
Section II: Questions designed to measure pharmacists' knowledge of pharmaceutical care for asthma patients (17 true-false items, 22 multiple-choice items). This section was a test of pharmacists' knowledge about asthma and its treatment, and was based on the Guidelines outlined above. A correct response was assigned one point; an incorrect response received no points.
Section III: Demographic information. Participants were asked to provide information on eight demographic variables: 1) Gender, 2) Year of graduation, 3) Employment status, 4) Employment position, 5) Work setting, 6) Percentage of time spent in distributive activities, patient-oriented activities, and management activities, 7) Personal / family history of asthma, and 8) Recent exposure to asthma information.
Two experts in asthma care reviewed the first draft of the questionnaire, with a view to maximizing content validity. The content domain, consisting of recent asthma treatment knowledge (specifically, the Guidelines outlined above), served as the framework for the development of the items and the Revue canadienne de l'éducation permanente universitaire Vol. 28, N o 1, printemps 2002 review. Seven pharmacists reviewed the second draft to provide face validity. These reviews resulted in the clarification of ambiguous items, the validation of the content, the elimination of some items, and the addition of others. The knowledge test was subjected to an item analysis (although it was recognized that a sample of seven was small), which was useful in selecting some items and eliminating others. Three areas from the Guidelines were included in the assessment of perceived needs but not in the knowledge test, in one case because an adequate test would have required excessive time (item 16, How to use a symptom diary), and in two cases because the answers varied depending on the geographic location of the pharmacist (item 20, What asthma support groups are available; item 21, What asthma educational programs are available).

Procedures for Data Collection and Analysis
Participants at the two continuing pharmacy education programs on asthma were informed that a study of pharmacists' educational needs was being conducted (through the Continuing Pharmacy Education unit), the purpose of which was to obtain more information from pharmacists themselves about their needs relating to providing pharmaceutical care to asthma patients. Participants were asked at the beginning of the program to complete the questionnaire, which took approximately 30 minutes. Usable data were obtained from the 113 pharmacists who attended the program, although some data were missing for some items. Complete data were available for 111 respondents.
SPSS 6.1.3. for Windows was used for all statistical analysis, with p<0.05 selected as the level for statistically significant differences. Frequency and percentage distributions, means, and variances were used as descriptive statistics. A mean of perceived needs score was calculated for each item. Knowledge-based needs scores were calculated as a proportion of the number of respondents who answered items correctly. To test the relationship between perceived and knowledge-based needs, both Pearson's productmoment correlation (r) and Spearman's rank correlation coefficient (r s , formerly rho) rank correlation methods were used.

Sample Characteristics
Gender: Women made up slightly more than half of the participants (57.5%).
Year of graduation: This ranged from 1950 to 1995, with the mean year of graduation being 1979 and a standard deviation of 10.7 years. At the time of the study (1996), slightly over 25 percent of the participants were within 9 years of graduation, and another 28 percent were within 10 to 19 years of graduation.

Employment status:
The majority of participants were currently in active practice. Sixty-one percent of respondents worked full-time in a pharmacy-related area, and 26 percent practised pharmacy in a parttime capacity. Data were missing for 7 percent of the respondents.

Employment position:
Nearly 50 percent of the participants listed themselves as staff pharmacists/employees; 23 percent were managers/directors; and 20 percent were owners/partners. Data were missing for 7 percent of the respondents.

Work setting:
The majority (83%) of the participants were employed in a community pharmacy setting; 5 percent in a hospital; and 1 percent in an academic institution.

Time spent in certain activities:
Participants were asked to indicate the extent to which they were involved in three types of activities during a typical day. For distributive activities, that is, filling prescriptions, the mean was 51 percent of time spent. Less than one-third of respondents spent over 60 percent of their time in these activities. For patient-oriented activities (education, monitoring), the mean was 28 percent of time spent, with more than two-thirds of respondents reporting that they spent less than 40 percent of their time in these activities. For management/administrative activities, the mean was 23 percent of time spent, with more than two-thirds of respondents spending less than 40 percent of their time in these activities.
Personal/family history of asthma: Almost one-third of participants had personal experience (self or family) with asthma or chronic obstructive pulmonary diseases.
Recent exposure to asthma information: Over two-thirds of participants had read something on asthma or its treatment in the last six months.
Revue canadienne de l'éducation permanente universitaire Vol. 28, N o 1, printemps 2002 Table 1 compares the perceived needs scores with the knowledge scores. For ease of comparison, the items are shown in rank order according to perceived need. Standard deviations are included to illustrate the range in variability of the scores, which in some cases was very great. These large variances indicate that pharmacists differ considerably among themselves in their perception of their learning needs and in their knowledge. Considerable variation also occurred from item to item. Perceived needs ranged from a mean low of 3.0 to a high of 4.3 on the five-point scale. Knowledge-based needs ranged from a low of 50 percent of the respondents answering the item correctly to a high of 96 percent answering correctly. Thus, the use of the Canadian Consensus Asthma Management Guidelines (1996) as a standard resulted in some learning needs being identified, based on both actual knowledge and learners' perceptions of their needs.

Comparison of Perceived Needs Scores and Knowledge Scores
The two items with the highest perceived needs scores also had the lowest proportion of respondents answering the knowledge test correctly (item 13, The role of new asthma medications, and item 14, What to do if asthma gets worse). Item 15, How to use a peak flow meter, was the fourth highest perceived need and also the fourth lowest knowledge item. Although perceptions and knowledge matched on these items, this was not the case for all items. Some items not perceived as high needs showed up as higher needs on the knowledge tests. For example, item 10, How to take asthma medicines, was not a highly rated perceived need, yet only 58 percent of respondents answered the question correctly. These results are similar to Queeney's (1995) findings that new knowledge areas are likely to be perceived as areas of need, and that objective assessment will identify needs in areas of practice that are regularly performed but not seen by professionals as areas of need.
Two rank correlation methods were used to test the relationship between perceived and knowledge-based needs: Pearson's r and Spearman's r s . The assumptions that should be met for the more powerful test, Pearson's r, are more stringent than those for Spearman's r s , which is a non-parametric test. However, in both cases the coefficient varies between -1 and +1, with correlations near 0 indicating no relationship. Thus, both tests provide information on the strength and direction of relationships.
The resulting coefficients were r = -0.0349 and rho = -0.0544, indicating that, overall, pharmacists' perceived needs scores were not significantly correlated with their knowledge-based needs scores.

DISCUSSION AND IMPLICATIONS
The results of this study demonstrated that no relationship exists between perceived needs and knowledge-based needs on the topic of asthma for a group of pharmacists. Although the two methods corresponded on a few of the items, overall, they did not identify the same needs. These results were similar to those of some studies of other professionals, confirming that there are some educational needs of which learners are unaware (Hiemstra & Long, 1974;Lockyer et al., 1998;Queeney, 1995).

Implications for Continuing Professional Education Practice
Do the findings of this and other studies mean that CPE providers should discontinue the assessment of perceived needs? Not necessarily, although educators certainly need to be aware of the limitations of self-identified needs. The major limitation, of course, is that individual professionals may not always be capable of recognizing their knowledge gaps and practice deficiencies or prepared to name them even if they do recognize them. As Queeney (1995) found, professionals may be willing to identify learning needs related to "new knowledge," but less willing-or able-to identify needs related to "routine" practice behaviour. A study of gerontological registered nurses in South Carolina suggested that learners may underestimate their own learning needs and/or overestimate the learning needs of others (Timms, 1995). The nurses rated either their own personal gerontological continuing education needs or the education needs of their peers, and Timms found that "without exception" nurses saw their own learning needs as not as great as those of their peers. She noted that "using the felt needs approach alone is inadequate, given that learners may not have sufficient resources, adequate vocabulary, or conceptual ability to make decisions about their learning needs" (p. 87).
Even with these deficiencies, many educators see value in obtaining data on perceived needs. Timms (1995) noted support in the literature for basing programs at least partly on felt needs because these programs are perceived to be more relevant by potential participants. Sork (2001) identified the motivational advantage of learner-identified needs, stating that: programs based on needs identified or acknowledged by learners themselves are typically more attractive to those learners than programs based on needs identified by others. . . .unless [adults] recognize a gap between their present and desired capabilities that the program addresses, they are unlikely to be enthusiastic, engaged participants. (pp. 101-102) Queeney (1995) also saw value in the use of perceived needs assessment: because people who believe that their knowledge, skills, or performance abilities are weak in certain areas may lack the confidence to perform well in those areas. Individuals' perceptions that they have specific needs may motivate them to participate in continuing education. (p. 83) Identifying felt needs is also seen as a useful first step in planning programs; once general areas where programs may be needed are identified, more detailed assessments can be carried out to determine specific program content.
With the complex and fast-changing knowledge base of most professional practices today, it is reasonable to expect that professionals will need assistance in assessing and identifying their learning needs. Queeney (1995) suggested that some of the limitations of using self-identified needs can be overcome by careful wording of questions, and provided several examples in which the structuring of questions can help respondents recognize areas of deficiency. She advised that: a number of specific questions can guide respondents to consider relevant factors rather than simply offer quick answers without much thought. Often, unless particular areas are pointed out to them, people simply do not think of them. Specific questions lead them to consider those areas and elicit responses that are closely related to discrepancies in respondents' proficiencies and to their educational needs. (p. 119) However, it is widely recognized that the limitations of self-perceived needs are such that they must be supplemented by other methods of needs assessment. Lockyer (1998) concluded that because practice in the health professions is increasingly "evidence-based," planning committee members want more than a list of "favorite topics" when deciding on programs (p. 191).
Knowledge tests are one way to supplement self-identified needs, although this approach adds to the time required to identify needs and produces test anxiety in some learners. The study discussed in this article demonstrated that the knowledge test identified some needs that would not have been identified using the self-perceived instrument alone. A related but different approach is "self-assessment," a process that uses "a selfadministered testing process that provides confidential, personal information to participants based on external, profession-defined criteria" (Klevans, Smutz, Shuman, & Bershad, 1992, p. 17).
Current writing in CPE recognizes that, while knowledge tests are useful, methods that go beyond that approach are needed to identify the strengths and deficiencies of professionals in actually applying their knowledge and skill in their practice settings. Queeney (2000) noted that such methods do not necessarily have to be time-consuming and expensive, citing focus groups and supervisor reports as examples of less expensive approaches than more costly live simulations and practice observations and audits.

Implications for Research
Many questions remain, however, and more research on needs assessment in CPE is clearly needed. Timms (1995) saw the need for research "to refine methods of collecting needs assessment data to improve accuracy and relevance in identifying learning needs" (pp. 87-88). Lockyer (1998) identified research into multiple methods to determine which methods are best combined and under which circumstances they should be combined. She suggested that "additional research is needed to refine the tools so that they are more easily implemented in a routine way in program planning and development" (p. 192). This suggestion is well-taken: continuing professional educators cannot afford to do a major research study each time they are faced with planning a program. Research to help determine the conditions under which it is wise to invest more substantially in needs assessment would be welcome, as would research into how critical indicators of educational needs can be more easily identified.

CONCLUSION
Needs assessment has major implications for program development and delivery. In particular, if needs assessment were to focus to a greater extent on applied practice needs, then programs would have to be developed to address the gaps identified. Queeney (2000) called for more emphasis on programs that enhance professionals' abilities to apply new learnings to solve problems in their practice. Such programs must go beyond the provision of information to assist professionals to apply their knowledge and skills. This "practice-oriented CPE" will be more complex and certainly more expensive to design and deliver (p. 379). Queeney described the challenges facing continuing professional educators in this way: They will need new capabilities, including those related to collaboration, needs assessment, practice-oriented instructional design and delivery, performance-based evaluation, interprofessional education, and distance education. No longer simply program providers, they will become performance consultants to the professionals they serve, their employers, and the professions themselves. At the same time, they are being asked to balance good education principles against the increasingly entrepreneurial demands of their organizations and institutions, forcing them to adopt cost-effective strategies for designing, developing, and delivering CPE. (pp. 379-380) Canadian Journal of University Continuing Education Vol. 28, No. 1, Spring 2002 Other recent writings challenge us to revision our practice by moving beyond "providing programs" to focus on professional learning and how it can be facilitated (Mott & Daley, 2000). Mott (2000) reviewed models of how professionals learn (including the mental schema model, the skill acquisition model, and Schön's reflective practitioner model) and discussed some implications for CPE. Daley (2000) described how professionals construct knowledge in the context of their practice and provided not only an expanded model of learning in CPE but also its implications for the provision of CPE. To meet these challenges, it is clear that partnerships will be needed between universities, professional associations, employers, regulatory agencies, and professionals themselves.
Needs assessment will continue to be a key component of effective CPE practice into the future. As program planners, we will be challenged to incorporate needs assessment strategies that both reflect the realities and complexities of professional practice and develop professionals' capacitiesand motivation-to assess their own learning needs.