Canadian Journal of Learning and Technology Volume 31 ( 1 ) Winter / hiver 2005 A Review of What Instructional Designers Do : Questions Answered and Questions Not Asked

Abstract: The purpose of this literature review was to determine what evidence there is that instructional designers apply ID Models, as well as to establish what other activities and processes they might use in their professional activities. Only ten articles were located that directly pertained to this topic: seven reporting on empirical research and three case descriptions recounting development experiences. All ten papers pertained to process-based ID models. Results showed that, while instructional designers apparently do make use of process-based ID models, they do not spend the majority of their time working with them nor do they follow them in a rigid fashion. They also engage in a wide variety of other tasks that are not reflected in ID models. The purpose of this literature review was to determine what evidence there is that instructional designers apply ID Models, as well as to establish what other activities and processes they might use in their professional activities. Only ten articles were located that directly pertained to this topic: seven reporting on empirical research and three case descriptions recounting development experiences. All ten papers pertained to process-based ID models. Results showed that, while instructional designers apparently do make use of process-based ID models, they do not spend the majority of their time working with them nor do they follow them in a rigid fashion. They also engage in a wide variety of other tasks that are not reflected in ID models. Résumé: L’objet de cette revue de la littérature consistait à identifier les preuves selon lesquelles les concepteurs de matériel pédagogique appliquent les modèles de conception pédagogique et à identifier les autres activités et processus qu’ils pourraient utiliser dans le cadre de leurs activités professionnelles. Nous n’avons trouvé que dix articles qui traitent directement de ce sujet : sept portant sur des recherches empiriques et trois sur des descriptions de cas retraçant des expériences de conception. Les dix articles traitent des modèles de conception pédagogique axés sur les processus. Les résultats ont démontré que bien que les concepteurs de matériel pédagogique utilisent manifestement les modèles de conception pédagogique basés sur les processus, ils ne les utilisent pas dans la majorité des cas et ne les suivent pas à la lettre d’une manière rigide. Ils abordent aussi une diversité d’autres tâches qui ne figurent pas dans les modèles de conception pédagogique.


Introduction
The research literature pertaining to Instructional Design (ID) or Instructional Systems Design (ISD) theories is extensive and extends at least as far back as the 1970's (Gustafson & Branch, 2002;Shrock, 1991).In fact, the roots of ID can be traced back to the seminal work of Robert Gagné (1965) on the conditions of learning and early attempts to apply general systems theory and systems analysis (Banathy, 1987).The 1970's saw a proliferation of published ID models all based on the core of the ADDIE model of analysis, design, development, implementation and evaluation (Gustafson & Branch, 2002) and, by 1980, as many as 60 such models were identified (Andrews & Goodsen, 1980/1991).The majority of these models (e.g., Dick, Carey, L., & Carey, J.O., 2005;Morrison, Ross, & Kemp, 2004;Smith & Ragan, 2005) could be said to be process-based, that is, they stipulate the processes and procedures that instructional designers should follow in their practice.Some models, such as those of Gagné and Briggs (Gagné, Briggs, & Wager, 1988), Merrill (1983), and Reigeluth and Stein (1983), were more theory-based and developed on the basis first of behavioural learning theory and later cognitive theories of learning that have dominated the field for over 25 years (Willis, 1998).Regardless, these models described an expressly linear, systematic, prescriptive approach to instructional design (Andrews & Goodsen, 1991;Braden, 1996;Reigeluth & Stein, 1983;Wedman & Tessmer, 1993) and were strongly objectivist in nature (Jonassen, 1999).Although they have moved away from strict linearity and are less explicitly prescriptive, such models continue to thrive in various portrayals (e.g., Morrison, Ross, & Kemp, 2004;Seels & Glasgow, 1998;Smith & Ragan, 2005) and have been taught to thousands of graduate students (Willis, 1998).But in practice, we suggest that models of instructional design, while implicitly prescriptive, are in fact conceptual frameworks for practice.ID models are useful to designers and inform practice, but few if any designers actually use models to confine their practice.
Relatively recently, the field has experienced the strong influence of constructivist learning theory and a shift from teacher-controlled to learner-centred instruction (Reigeluth, 1996;1999).This movement led to the emergence of a number of ID models based on constructivist learning principles (e.g., Cennamo, Abell, & Chung, 1996;Hannafin, Land, & Oliver, 1999;Jonassen, 1999;Mayer, 1999;Shambaugh & Magliaro, 2001, in press;Willis, 2000) and has, in turn, stirred a vigorous response from advocates of more traditional models (Dick, 1996;Merrill, 1996).All in all, it is safe to say there is no lack of advice on how to do instructional design.But is the advice of either side heeded?Do instructional designers actually use ID models?Does the utility and adaptability of ID Models meet practitioner needs?Are instructional design models grounded in practice?The purpose of this literature review was first to determine what evidence there is that instructional designers are applying ID Models in their work, as well as to establish what other activities and processes they might use in their professional activities.After considering these issues, we turn our attention to what questions are missed when we restrict our view of instructional design to the processes, skills and practices of instructional designers.Ultimately, we suggest that a social-constructivist view of instructional design begs different types of questions, not based on discrete competencies, but rather on the meaning instructional design has for society, institutions, designers and users.

Review of Current Literature
The goal of this review was to determine what research there was describing how instructional designers, especially those working in higher education (including instructional designers, course developers and faculty members who do instructional design or course development), practice and what they are doing when they do instructional design.While there have been a plethora of position papers, opinion pieces and theoretical papers presenting instructional design advice or models, only ten articles were located that directly pertained to this topic.Of these, seven papers reported on empirical research -six survey studies and one qualitative interview design study -which are reported in the first section below.The remaining three articles were case descriptions in which authors recounted their development experiences with instructional design teams and provided advice or practice on that basis.All ten papers pertained to process-based ID models.None were found that considered the use of theory-based models.

Empirical Research on the Activities of Instructional Designers
Over a decade ago, Wedman and Tessmer (1993) conducted a survey study of the design practice of 73 instructional designers developing training for business and industry.40% of their respondents were from the same training and development group in the Midwestern United States and the remaining 60% from a variety of business and government contexts.The purpose of the survey was to determine if a) these instructional designers strictly followed the prescriptions of design models or b) if they used the models selectively, what factors guided their decisions.Respondents were asked to rate the frequency with which they completed 11 common ID activities derived from the Andrews andGoodsen (1980 / 1991) review and the 1990 iteration of the Dick and Carey ID model: • Conduct a needs analysis.
• Determine if need can be solved by training.
• Write learning objectives.
• Identify the types of learning outcomes.
• Assess trainees' entry skills and characteristics.
• Develop test items.
• Select instructional strategies for training.
• Select media formats for the training.
• Pilot test instruction before completion.
• Do follow up evaluation of training.
Respondents were then to select reasons for excluding a specific activity.Most (> 50%) reported using common ID activities on a regular basis but omitted one or more activities on every project.A sizable minority (at least 25% on 7 of 11 activities) acknowledged omitting many of these activities from their practice.The highest frequency responses for excluding an activity were: a) decision already made, b) not enough time, and, c) considered unnecessary.Wedman and Tessmer (1993, p. 53) noted that most ID models are based on three major assumptions: a) all activities prescribed by the model will be completed, b) each activity will be completed before proceeding to a subsequent activity, and c) each activity will be completed to the same degree of precision.They concluded that such ID models were not compatible with ID practice at the time in that they did not allow for selective completion of activities and were not sensitive to the factors that influence designers' decisions.Rather, they postulated that designers instead created multiple layers of instructional design activities based on the specific design situation as proposed in their Layers of Necessity Model (Tessmer and Wedman, 1990).To verify the Layers of Necessity Model, Winer and Vásquez-Abad (1995) replicated the Wedman and Tessmer study with members of a local National Society for Performance and Instructional (NSPI) chapter in Montreal, Canada.From 246 members sent surveys, 66 valid questionnaires were received for a 28% response rate.Of these, 60% reported having design or development tasks as a part of their job responsibilities at the time.The majority (80%) of respondents agreed that the 11 steps were part of the ID process and accepted the process as valid.The results confirmed the Wedman and Tessmer (1993) finding that instructional designers do not systematically perform all the steps in the model.Only seven participants (10%) reported always performing at least the first 10 steps (except follow-up evaluation).The highest frequency responses for excluding an activity were: a) decision already made, b) not enough time, c) considered unnecessary, and d) client would not support.These were nearly identical to the previous study.
Again, a lack of expertise was the least frequently cited reason.Winer and Vásquez-Abad concluded that there was a parallel between instructional design steps seen as most necessary and those performed most frequently, thus supporting the Layers of Necessity model.More recently, Cox (Cox, 2003;Cox & Osguthorpe, 2003) carried out a survey of 142 instructional designers employed full time in both academic and corporate settings.
It is important to note two important delimitations of this study.First, only alumni of the major American Instructional Technology programs (those teaching Instructional Design) were recruited.Second, a relatively narrow definition of instructional designer was used, i.e., "an individual whose primary responsibility is to determine instructional needs and to select or create appropriate interventions in a systematic way based on an understanding of human learning" (Cox, 2003, p. 7).The respondents who participated in the study consisted of 64 full time instructional designers (56%), 25 faculty, 24 administrators and managers, 10 developers, 6 teachers/trainers, 4 project managers and 4 students.Respondents indicated that, on average, 53% of their professional time is spent in organizational tasks, while 47% is spent performing instructional design tasks.
Instructional design tasks were defined as the stages of the ADDIE Model (analysis, design, development, implementation, or evaluation), while organizational tasks included such activities as project management, supervising personnel, professional meetings, academic research, marketing/sales, and professional development.When asked to apportion their instructional design time by task category, respondents indicated that development took 29% of their time and design 21%, followed by analysis (20%), implementation (17%), and evaluation (14%).Project management (28%) and professional meetings (24%) consumed the majority of respondents' organizational time, with supervising personnel (17%) and academic research (13%) next.The most informative result of this study is that, rather than working on traditional instructional design tasks (as defined by ID models), instructional designers spend over half their professional time engaged in the sorts of tasks listed in Table 1.Two survey studies from Australia reported on the perceptions of academic authors (faculty) concerning the role of the instructional designer in creating distance education courses in higher education.While these studies did not attempt to ascertain what activities instructional designers actually engage in, they do highlight what clients (course authors) believed they should do.Table 1.Instructional Designers' Time Spent on Organization.
Roberts, Jackson, Osborne, and Somers Vine (1994) surveyed 40 coursewriters (a response rate of 70%) at the University of Tasmania and asked them to rate seven instructional design roles on a three point scale (Table 2).Examination of the weighted totals in the two studies indicates that course authors in both settings valued the editorial roles of their instructional designers much more than traditional instructional design functions although those roles were appreciated as well.As evidenced in the other three studies, it seems clear that the expectations by authors of instructional designers -especially those working in higher education -ranges far beyond the application of design models and certainly includes editing and project management in at least equal amounts.Perhaps the most thorough study to date on the roles and activities of instructional designers was conducted by Allen (1996) in Australia.Allen surveyed 140 people either working as instructional designers or who had stated an interest in instructional design, and 99 usable questionnaires were returned for a response rate of 70%.Of these, 66% were employed as instructional designers, while the remaining 34% either worked as instructional designers, but had a different title, or instructional design was not their main job responsibility.
Fifty-one of those with the title of instructional designer worked at universities or Technical and Further Education (TAFE) institutes and 53 respondents had some qualifications in instructional design (mainly graduate level courses).As in the Wedman and Tessmer (1993) study, respondents were asked to rate the frequency with which they completed common ID activities, but were given a more extensive list of 29 items encompassing all of the activities in the earlier studies by Roberts et al. (1994) and MacPherson and Smith (1998), as well as items reflecting editing and project management functions.Items were rated by respondents and the 11 highest rated items are listed in Table 3. Six of the most frequent items were traditional instructional design activities such as "determining instructional strategies" and "designing goals and objectives."Five of these most frequent choices, however, including three of the top five, such as "designing the layout and appearance of materials" (graphic design), "editing," and "project managing the development of materials," fit what Allen describes as the "technician role of an editor."Allen concludes that the survey results highlight a dichotomy between definitions (models) of instructional design and the activities in which many instructional designers engage.Table 3. Rankings of Common Instructional Designers' Activities.• Working on the design, i.e., to translate the client's needs into a plan that will be used to produce a product that meets the client's needs.
• Working with other members in a team, i.e., designer must be a team player and a true collaborator.
The respondents saw several challenges related to these responsibilities.The first challenge was working effectively with clients, guiding them through the design process, describing the problem to be solved, and helping them to make the right decisions based on the project's needs.The second challenge was balancing multiple roles.Over half those interviewed reported also being project managers or performing project management duties in addition to being an instructional designer.They also recounted engaging in such activities as script writing (for audio and visual segments), programming, creating animation and graphics, writing technical documents and training others.The third challenge was adapting to technological change.Respondents noted that rapid technological advances were continuously introducing new requirements to the field.They felt the pressure to produce educational products using new technological tools and emphasized the need to stay current.

On the basis of these results, the authors delineated four competencies for instructional designers:
• Communication skills: Instructional designers should be able to communicate effectively with clients, subject matter experts, and other team members both verbally and in writing.5 compares the findings of studies reported in this paper to those competencies and suggests a relatively close match.
The IBSTPI competencies are based on a number of assumptions that reflect both the nature of the field of instructional design and designers themselves (Richey et al., 2000).First is the understanding that instructional designers are persons who demonstrate design competencies on the job regardless of job title or training and that ID competencies pertain to persons working in a wide range of job settings.Second, instructional design is a process most commonly guided by systematic design models and principles, an assumption that does have some support in the above studies.Third, ID competencies are assumed to be generic and amenable to customization and should be meaningful and useful to designers worldwide.This last point appears to be strongly supported by the research reported here.Finally, the authors of the IBSTPI competencies assume that they define the manner in which design should be practiced [emphasis added] (Richey, et al., 2000, p. 41).This is based on the notion that instructional design processes and procedures retain much of the early influence of systems and behavioral learning theory, but also reflect cognitive and performance improvement theory.But do they reflect the design process itself?

How Designers Approach Design
The issue of how designers approach design has been addressed in the literature from a number of angles.As outlined previously, ID models (or, at least, process-based models) have been viewed as incompatible with ID practice in that they do not encourage selective completion of activities and are not sensitive to factors that influence designers' decisions (Liu et al., 2002;Wedman & Tessmer, 1993;Winer & Vásquez-Abad, 1995).Gibbons (2003) expressed a similar point of view and proposed that, as instructional designers learn their practice, they move through a series of phases in which their designs have a specific focus.The first is what Gibbons termed media-centrism, where designers construct their designs using the vocabulary of the medium (usually the latest emerging technology), rather than to see it as a malleable medium for developing learning interactions.The second phase, message-centrism design, places primary importance on media constructs rather than the demands of the message itself.The third stage is strategy-centrism, in which designers use rules to govern the delivery of compartmentalized information and interaction components.It is only in the fourth phase, model-centrism, that designers are able to think in terms of system and model constructs that support problem-solving.Based on this analysis, Gibbons (2003) contended that instructional design consists of multiple layers of decision-making, each with its own set of design constructs and processes.These layers include a) model/content, b) strategy, c) control, d) message, e) representation, e) media-logic and f) management.
One of the main competencies that emerged in the study by Liu et al. (2002) was that instructional designers should develop strong problem-solving / decision-making skills.In this regard, Rowland (1992) carried out a comparative study of eight novice and expert instructional designers engaged in a design project to better describe what designers actually do when engaged in the design process.The design process was clearly separated into two phases: problem-understanding and solution generation and key differences emerged between novice and expert designers.During the problem-understanding phase, experts tended to link the given information in the design situation to experiences with similar problems and develop a preliminary concept of what the problem was.They then retrieved a mental model of the types of information they would need to obtain and used it to guide further inquiry and analysis of the problem situation.They typically interpreted the problem as poorly defined by the given information.Novices, on the other hand, tended to interpret the problem as well defined by the given information and did little elaboration.Rowland (1992) reported his results to be congruent with the research on expertise and indicated that expert instructional designers clearly employ a definable problem solving and decision-making process.He suggested that ID tools, unlike procedural design models, should foster a deep understanding of the system of concern and should include such characteristics as flexibility of structures and processes, a workspace for construction of problem representation, and mechanisms for making multiple links between problems and solutions.
Rowland suggested that, rather than to be taught procedures or even problem-solving heuristics, novices need to develop experience in the design process and that a case-based method of teaching, providing involvement with real or realistic situations, might be the most appropriate way for new instructional designers to learn the design process.
Beyond Process: Looking Under the Rock Most of the discussion in this paper, and indeed in the literature of instructional design, concentrates on the discrete skills, competencies and activities involved in the practice of instructional design.Even where the literature identifies non-traditional elements of ID, the focus is still on discrete roles and functions (see Table 5).While these are important elements, we suggest that by focusing so closely on functional elements, existing literature largely overlooks important and emerging questions about instructional design.These questions focus not so much on what instructional designers do when they carry out a project, but rather on what it means to be an instructional designer, and participate in the culture of instructional design.How do instructional designers construct and enact their professional identities?How do they describe the importance of what they do?Where do instructional designers find communities of practice?What satisfactions do instructional designers draw from their work, and how do they see their contributions in the larger context of learning and society?Do instructional designers see themselves as agents of change, as leaders in important cultural shifts?
We know precious little about these important issues and we suggest that research should focus clearly on socio-cultural issues in instructional design, not solely the technical aspects of how instructional designers perform the rudimentary functions of ID.For example, Nelson and Stolterman (2003) encourage the development of a design culture.They see design the ability to imagine "that-which-does not-yet-exist" and make it appear in a concrete form as purposeful new addition to the real world.In their view, the process of design is about developing an appreciative judgment about what is to be considered and the exploration of possibilities, leading to a compositional interpretation, a process that relies on various types of judgments, including appreciative judgments, instrumental judgments (e.g., the application of design models) and framing (reflection in action).
Design is always about making judgments about design situations that are complex, rich and replete with tensions and contradictions.This is compatible with the multi-layer view of the design process advocated by Gibbons (2003).We would add that these views of design also include the larger influences of social, institutional and personal change.Those who engage in design must accept that their role is one of leadership in the process, and emphasize their contributions as change agents within complex social systems.
These perspectives point to fresh productive avenues for research into the practice of instructional design.In order to truly understand what instructional designers do and how to help them develop more effective practice, we not only need to further study their actual practice, but also to help them more fully understand the roles they play as leaders in the enterprise of learning.
These roles were selected by the researchers based on their experience working with instructional designers and piloted with three experienced course authors.MacPherson and Smith (1998) later conducted a follow up on the study by Roberts et al. at Central Queensland University.They surveyed 54 authors (82% response rate) using the same questions about the role of instructional designers and the same rating system.
The final report (Liu, Gibby, Quiros, & Demps, 2002), a qualitative study of 11 practicing instructional designers working for new media development companies in Texas, was to determine instructional designers views of (a) their responsibilities, (b) challenges faced, (c) how they meet those challenges, and (d) what skills are important for being an effective instructional designer.In the view of the respondents, instructional designers could be said to have four major sets of

•
Knowledge of ID models: Instructional designers should be well-versed in several instructional design models and strategies from which to choose a case specific process [emphasis added] and keep up with new education or training theories and research.• Problem-solving / decision-making skills: Instructional designers should be able to perform multiple responsibilities, step into new roles when necessary, and overcome obstacles under a deadline.• Technology skills:Instructional designers should have a basic knowledge of important software tools used in the field and be aware of newly advanced tools as they become available.Case Descriptions of Instructional Development Project Reports Bichelmeyer, Misanchuk and Malopinsky (2001) reported a case analysis of adapting an online Master's course on Instructional Design and Development from a residential course to a web course.The design team used three distinct bodies of research literature during the design process: a) that specifying appropriate features for Web-base instructional products, b) that on instructional design processes, and, c) that describing the experiences of website designers.Their instructional design processes included confirming goals and objectives, learner analysis and designing instructional experiences according to the ID model of Gagné, Briggs and Wager (1988), as well as usability testing (formative evaluation).Based on their analysis, the authors suggest that for Web-based projects instructional designers should be concerned with the following tasks: • Use as many human resources as possible for adaptation, design and development, that is, engage in team development.• Confirm the capacity and appropriateness of technology to address instructional needs.• Set minimum technology standards for students.• Use formative evaluation in the form of iterative feasibility testing.• Provide detailed technology training for learners.• Develop policies for ownership of materials.Only two of these tasks, technology to address instructional needs and formative evaluation, could be classified as standard instructional design tasks.Based on the experience of these authors, the ID process extends far beyond the existing models to include such items as team building, training learners in the use of technology, and developing administrative policy.Rowley, Bunker and Cole (2002) presented a case study of the development of a large scale blended military training course."Instructional design -integrated product teams (ID-IPT)" were formed to apply a systems-engineering style design approach and consisted of subject matter experts from several military systems acquisitions disciplines, a media production representative, a project management representative and one or more instructional designers.The course design emphasized motivational and practical concerns and the transfer of training into practice and was based on situated learning and problem-based learning theories.The article concludes that the development of effective blended instruction is a complex process requiring extensive interactions among a team of instructional product designers and developers.As indicated in the previous study, in addition to traditional activities such as lesson design, formative and summative evaluation, the instructional design process would include such activities as team-building, project management, and story scripting (writing) Finally, Glacken and Baylen (2001) report on a case study of how a faculty member and an instructional designer worked together to develop an online undergraduate health education course that embraced a problem-based learning model and used emerging technologies to support the online learning activities.The authors described a number of lessons they learned about the process of designing, developing, delivering and evaluating the course.Four of these lessons were directly related to the instructional design process per se: The faculty member, instructional designer, and other technology staff need to work together as collaborators during the online course development process.The pedagogy -not the technology -must drive course delivery.Student interaction is critical in online learning environments.Faculty need to be prepared for changes in their role in an online learning environment.The second and third recommendations could be seen as traditional instructional design activities (selecting and designing learning strategies), but the first focuses on team building and project management, while the fourth encompasses faculty development (teaching).Discussion The purpose of this literature review was to determine what evidence there is that instructional designers are applying ID Models in their work, as well as to establish what other activities and processes they might use in their professional activities.What do these studies tell us about the day-to-day practice of instructional design?While instructional designers apparently do make use of the techniques delineated by traditional, process-based models, it is clear that they do not spend the majority of their time working with them nor do they follow them in a rigid fashion (e.g., Liu et al., 2002; Tessmer & Wedman, 1990; Wedman & Tessmer, 1993; Winer & Vásquez-Abad, 1995).They also engage in a wide variety of other tasks that are not reflected in ID models (see Table 4).Table 4. Non-traditional Instructional Design Skills Required of Instructional Designers.How do these compare to the instructional design literature?It appears that the expert advice from the field matches what practicing instructional designers say they are doing.The International Board of Standards for Training, Performance and Instruction (IBSTPI) published an extensive set of instructional design competencies based on the literature and on practice (Richey, Fields, & Foxon, 2000).Table

Table 2 .
Course Authors' Perceptions of the Importance of Instructional Design Roles.