Can Instructional Design Deliver on the Promise of the Web?

After 40 years, technology-based learning is an overnight sensation. It has a new, trendy name: e-Learning, which usually connotes delivery using computers and the Internet. Suddenly, it seems that many CEOs, superintendents, college presidents, and even many state and national political leaders, are supporting the implementation of large-scale technology-based learning solutions. The services of e-Learning specialists are in demand as never before.

After 40 years of toiling in the wilderness of underfunded “demonstration” projects and short-lived companies, why now? The reason, of course, is the ubiquity of high-powered technology due to the explosive growth of the Internet. But the dynamics of the Internet are playing out differently in training, academic education, and non-traditional learning environments such as the home and community-based organizations. In each case, there is the potential for basic transformation of the way in which learning services are provided, the roles of learners and instructors, the economics and the organizational structures needed. Each context places its own unique demands on the field of instructional design (ID), including design theory, methodology, tools, and professional roles needed. And in each context, practitioners of sound methodology, regardless of training or theoretical orientation, are so much in the minority that the ultimate effectiveness and credibility of e-Learning may be jeopardized.

The challenge for ID professionals, therefore, is not only to evolve the field, but to assure that the products of sound practice are recognized and valued. This article will briefly examine major trends for change in each sector, and then describe how each is affecting (or ought to affect) the development and dissemination of instructional design theory and methodology. For simplicity’s sake, the assertions here are offered as opinions, with few supporting references. The reader is encouraged to take these assertions as a practitioner’s opinion and to challenge them critically. The hope is that the thoughts presented here will be not be the last word, but the beginning of many dialogs on the future of ID.

Corporate training appears to be leading the transformation to e-Learning. Major e-Learning initiatives are now common in large (Fortune 1000) companies. According to the annual Training magazine survey, e-Learning expenditures are growing to as much as 30% of the training budget in the leading companies. This compares with a level of spending in the 10% range throughout the 1990s (Rosenberg, 2001).

The motivating factors for e-Learning in the enterprise are predominantly economic. Over the past 20 years, businesses of all kinds have greatly increased their productivity by re-engineering their business processes to exploit information technology. Many business functions that are now judged strategic literally could not be performed any other way. The result has been transformational, not just incremental, change.

Enterprise managers now demand transformational change from the training function. The most direct rationale is based on reductions in training delivery costs. Information technology infrastructure is a sunk cost, so the training function does not have to cost-justify it. “Letting the electrons do the travel” offsets the costs of travel to the training center, the overhead of the classroom, and much lost trainee time. Furthermore, e-Learning is instantly available, so the advantages of timely, on-demand learning provide access impossible in a training center. Finally, e-Learning is scalable: once developed, tens of thousands of employees or customers can use it immediately. When the training supports a change management or technology dissemination initiative, this capability can be crucial. These benefits far outweigh the relatively high up-front cost of training development using e-Learning.

A more subtle benefit in many corporations is that e-Learning is often outsourced. Thus, many of the fixed costs of a platform training infrastructure are eliminated, and replaced with variable costs that are more easily charged to operational units and projects. Furthermore, management of the e-Learning function is often decentralized; it is in the hands of the line managers who contract for it.

Unfortunately, many corporate e-Learning implementations naively assume effectiveness. Evaluation of any kind beyond the “smile sheet” is not planned, and often there is no provision for measurement of learning outcomes, or even utilization. Some more cynical practitioners observe that it often seems that managers or customers care only about the appearance of training, not its effect. These practitioners argue that managers like e-Learning only because it is relatively cheap to deliver, and it absolves them from the obligation to provide training, at the least total cost.

While this view probably does not accurately reflect the views of the majority of managers, there is a grain of substance within the cynicism: current models of evaluation for business settings (often called Return on Investment, or ROI, models) are rarely used to assess e-Learning. The lack of evaluation is often taken as evidence of its low priority for managers. However, when a procedure is honored more in the breach than the practice, it is often wise to question the utility of the model. Therefore, it may be that current models are not adequate to the requirements of e-Learning.

The result, as many ID professionals have observed, is that most Web-based training products violate the most basic principles of instructional design. It is not only possible, but likely, that users of e-Learning will never encounter a product built according to sound ID principles, regardless of what theoretical framework one uses to define those principles.

We are inclined to underestimate the importance of this trend. If we agree that sound ID (by any definition) makes a difference in effectiveness—and perhaps even in utility and learner satisfaction—then we must also agree that most users of e-Learning will find their experience wanting. Furthermore, these learners will not discriminate between well-designed and poorly-designed e-Learning, nor will they recognize the relative value of one school of learning theory over another; they will most probably assume that all e-Learning is like their experience, and thus all e-Learning is equally weak. Poor quality hurts even the reputations of those who do not purvey it.

The trends discussed above have six potentially profound impacts on the field of ID. The six effects concern (1) e-Learning; (2) Needs assessment, ROI and measurement of outcomes; (3) the influence of intellectual capital and knowledge management; (4) the fusion of training, performance support, and knowledge management; (5) the need for better instructional systems design methodologies; and (6) the multiple career paths for entry into the field. Taken together, these effects will dramatically affect the evolution of ID as a field, and may ultimately have a transformative effect on the field.

The demand for e-Learning development far outstrips the available supply of people with competency to develop it—regardless of whether the competency is gained from an academic program of any theoretical orientation, a commercial workshop, or even gifted intuitive practice. We may legitimately call for more training programs of all types to meet the demand. In the meantime, however, the shortage of trained people means that most e-Learning is being built by people who lack the expertise to build effective products. Naïve managers and subject matter experts may not even recognize that there are any specialized e-Learning design and development skills.

The end result is that most e-Learning is being developed by people who lack instructional design competence, using general-purpose tools and methodologies which are unlikely to result in sound instruction. This situation has the potential of stimulating general disillusionment with e-Learning. If such a counterreaction occurs, then ID practitioners will be tarred with the same brush unless they can differentiate their work from the mediocrity that may rapidly become the norm. This will happen regardless of how competent they are and without regard to the theoretical persuasion of their training.

These issues—the need for improved, accelerated and large-scale professional training, the need for change in methodology and tools, and the poor quality of much (perhaps most) e-Learning—make a compelling case for renewed emphasis on asserting a strong professional identity for the field. A first step is development of competencies (such as those produced by IBSTPI, 2000). Ultimately, professional certification is likely to play a critical role as well, as a means for both producers and consumers of e-Learning to label high quality.

Changes in the business environment also affect the way in which training development projects are defined and ultimately assessed: needs assessment and evaluation (usually referred to as return on investment, as discussed above). Let’s examine them separately. Conventional approaches to needs assessment are based on gap analysis (Rossett, 1987). Central to an effective gap analysis is adequate definition (and, ultimately, measurement) of criterial performance. This is difficult, costly, and time-consuming enough when the cycle time is relatively long. But when product development and business process cycle times are short (measured in weeks or months), anticipating the probable performance problems and measuring them cost-effectively is much more difficult. Almost equally difficult is performance measurement in situations of high uncertainty. If one’s enterprise is in an extremely volatile business environment (a familiar situation for many technology companies, especially those based on the Internet), it may be very difficult to adequately describe desired performance, much less measure it, more than a few weeks or months in advance. In such an environment, the thoughtful ID practitioner is placed in the situation of having to train a workforce so they will be ready to execute innovations which have not yet been identified.

Faced with such challenges, a common reaction is to forego sound needs assessment, and to rely on Kirkpatrick’s (1994) Level 1 (smile sheet) or Level 2 (knowledge test) measurements. Or, almost equally bad, a needs assessment is done with a focus on performance problems, but the problems defined and measured are those that are easily measurable—often well away from the “cutting edge” of the enterprise. With either approach to needs assessment and performance measurement, the almost inevitable result is that the training becomes trivialized or irrelevant. Conventional approaches to needs assessment are subject to pitfalls of either type.

Conventional ROI assessment, the sibling of needs assessment, presents similar limitations. Specifically, Watkins, Foshay, and Kaufman (1998) argued that Kirkpatrick’s (1994) familiar 4-level model for evaluation of training effectiveness has three limitations. First, the model’s highest level of evaluation is impact on the businesses’ bottom line. Yet connecting training to the bottom line is often tenuous, except in cases where the training results in a cost savings. Cases based on value added by the training are often much harder to measure, because the value-added chain in most enterprises is itself not well measured. Many value-added characteristics of a company’s outputs are difficult to relate to the bottom line, because they do not directly result in more sales. For example, product innovations or service qualities which are viewed as competitive necessities, or which are important to customer satisfaction, safety, or the reliability and effectiveness of a product or service have a bottom line impact measured only by a rather indirect opportunity cost argument. Training via e-Learning may be one of the things it is mandatory to do in order to grow a line of business, but it is often difficult to isolate an effect due to training which is separate from the many other things a company must do in order to succeed in a line of business.

Second, the Kirkpatrick/Phillips ROI model stops short of measuring value to clients and society in general. This is important, because many of the kinds of value-added provided by many business practices (including training) can be measured more directly by customers than they can by the provider’s bottom line. Consistent with Kaufman’s argument, measurement of value to the customer is ultimately more important—and can be more direct. In such cases, looking to a change in the provider’s bottom line can be difficult or even misleading. Again, this is particularly important to training, because a too-limited view of the impact of training can lead trainers to focus on the wrong things.

Third, the Kirkpatrick/Phillips ROI model does not deal adequately with intellectual capital. Stewart (1999) has championed the recognition of knowledge as an asset in our increasingly knowledge-based economy. He argues that the generally recognized accounting practices governing business globally recognize only the costs associated with knowledge, not the assets. In a knowledge industry business, this means that the accounting picture of the business, with its focus on the bottom line, is profoundly misleading. While the current fad for knowledge management technologies may not survive, Stewart argues that the importance of intellectual capital to the enterprise is basic and will become increasingly apparent. If he is right, then the implications for building a sound ROI case for training are profound. It may mean that an ROI case based on bottom-line impact (and usually based on cost savings) grossly underestimates the benefits of training.

For an example, let us examine the case of distance education. The justification for distance education is often based on direct cost savings of avoiding platform training, a common approach to Kirkpatrick’s Level 4 for both needs assessment and evaluation. Yet the cost savings are probably the least important reason to do distance education. Advantages such as the ability to propagate knowledge quickly throughout the enterprise will result in benefits such as faster, more complete innovation, better business judgments based on a superior understanding of the business environment, and more value added for the customer. In an evaluation, such benefits are very difficult to project to the bottom line, however, unless one can build an opportunity-cost argument (for example, by claiming after the fact that a particular contract would have been lost had it not been for a particular innovation which resulted in some way from the distance education initiative among other initiatives intended to foster innovation). In a needs assessment, such opportunity-cost arguments rest on indirect inference; thus, building an opportunity-cost argument in a forward-looking needs assessment is much more difficult. When one adds to the mix a business environment that is volatile and has short cycle times, a sound needs assessment becomes almost impossible. Without such an argument, the training manager who falls back on cost savings to demonstrate bottom-line impact is likely to be greatly misled. The bottom line-driven needs assessment will lead to authorization of distance education projects that save the greatest cost, rather than those that are most effective and that address the most important business needs. Thus, a case such as this is one way to make even well-intentioned trainers vulnerable to the common complaint that they aren’t focused on what’s important to the company, or that they seem to focus more on delivery of training (by distance or other means), rather than its purpose or effectiveness.

The alternative, Watkins et al. argued, is to focus on important benefits to the client and the client’s client (and ultimately society as a whole), whether or not they can be cleanly projected to the bottom line. To work in this way, one builds needs assessment and “return on investment” evaluations based on accomplishments which are meaningful beyond the boundaries of the company. Often these performances and their measures cannot easily be expressed in bottom line terms, but can be measured by other means (such as customer loyalty; customer satisfaction– of the customer’s customer; environmental impact, adaptability to changing global economic conditions and thus create demand, rate of innovation, positive impact on brand integrity, etc.)

The limitations of conventional needs analysis and evaluation/ROI models that focus on the bottom line have an even more fundamental weakness: the definition of bottom line itself. Stewart (1999) first published the now-familiar argument that in the knowledge economy, the principal asset of any company is its collective knowledge of its business and its environment. Yet, this asset does not show on a conventional balance sheet, and there is no offset to the costs associated with the asset (such as research, training, internal communication, etc.). Thus, the bottom line is a profoundly misleading picture of a company’s balance of assets and liabilities, and the financial models associated with it often lead senior managers to make decisions which damage their principal asset (knowledge) in order to reduce costs or protect less important assets which do show on the balance sheet. The same problem leads “pure” knowledge-based companies (such as software companies) to have undervalued assets, making it difficult to attract capital.

The generally accepted accounting practices—the global “rule book” of accounting— is only beginning to change to accommodate intellectual capital. However, good progress is being made in methodologies for rigorous definition of intellectual capital (Lev & Mintz, 1999). As these methods make their way into the enterprise, it is critical for training managers to be among the first to use them as a basis of needs assessment and summative evaluations of return. This will be the basis of the next generation of needs assessment and ROI methodologies for training, and will take us well past the Kirkpatrick model.

A major result of research using the framework of cognitive psychology has been a vastly improved understanding of the nature of knowledge structures and their importance to performance. Learning models such as Anderson’s ACT* (Anderson, 1999) have been used as the basis of greatly improved accounts of the nature of expert and novice problem-solving, the role and structure of knowledge (schemata), and new syntheses of instructional strategies and performance support strategies such as Van Merrienboer’s (1997).

The same theoretical framework can serve as a basis of cognitively-oriented electronic performance support systems (EPSS) which focus on delivery of knowledge (especially facts and well-structured procedural knowledge) “just in time,” as it is used. The benefits of performance support have been well documented (Villachica & Stone, 1999).

But an even more important application of this theoretical framework is in the emerging field of knowledge management (KM) (Allee, 1997). In effect, KM extends the intent of EPSS to include ongoing capture of the collective knowledge of an enterprise. Ideally, the collaborative workgroup technologies of KM make knowledge capture and dissemination an inherent part of the fabric of work in knowledge-based organizations.

For both EPSS and KM, however, a major challenge is determining how to structure the knowledge for maximum usefulness. Tools of cognitive task analysis (Jonassen, Tessmer, & Hannum, 1999), which are based on cognitive models such as ACT*, have the potential of making a tremendous contribution. Going one step further, these same analyses of knowledge structures can be used as a basis for defining training. Thus, we have the possibility of using a single, unified analysis of knowledge structures as a basis of the two most powerful strategies in the enterprise for management and dissemination of intellectual capital: KM/ EPSS and training. Strategies that combine knowledge delivery and cognitive strategy training, such as van Merrienboer’s model, are an important result.

Furthermore, these same techniques for defining knowledge structures can serve as the basis of the definition of intellectual capital, and thus may be used as the basis for a new generation of ROI justification based on treating knowledge as an asset.

To make all this happen, however, new methodologies that integrate development of EPSS/KM and training components, and that take the new approach to needs assessment and ROI evaluation discussed above, are urgently needed. This is an excellent challenge for the field of instructional design, and may well transform the scope, definition and even the name of the field over the next decade or so.

The current business climate places a premium on speed. Speed of development, delivery, and dissemination is critical, even at some sacrifice of quality, and even if there is some elevation of cost. Furthermore, the uncertainty of the current climate challenges a basic assumption of conventional, top-down (linear or “waterfall”) ISD models: that it is possible to fully define the training need before designing the solution. In conventional ISD, changes in scope greatly increase project risks and costs, and are to be avoided if at all possible. Yet such changes are often unavoidable and so common that one must ultimately wonder if the problem is with the client or with the methodology itself.

Tennyson and Foshay (2000) are among those who argue for a “fourth generation” of ISD methodology based on the principles of rapid prototyping, iterative design and development, incorporating frequent learner trials in a learner-centered approach to design and development. The argument in favor of such methodologies is precisely analogous to those used in engineering fields such as software development and manufacturing engineering.

While Merrill’s ID2 project, the AIDA project, and a handful of commercial efforts (such as Designer’s Edge) have made some progress in scaffolding and automating the front-end design process, much remains to be done. In software development and a variety of engineering fields, tools for rapid protoyping and iterative, user-centered development are widely available. In ID, we are not so lucky.

Some large-scale private-sector training developers (such as NetG, Inc., and PLATO Learning, Inc.) have made some progress, but ISD remains a “black art” advocated by some, and truly practiced by few, in relatively unusual circumstances. The net result is that most e-Learning does not reflect current “best practice” in ISD. This state of affairs has led some to question the future of ISD itself (Gordon & Zemke, 2000).

As a field, therefore, ID is at a crossroads. If the field undertakes the development of true “fourth generation” tools and ISD methodologies, then the “ISD model” will successfully evolve. If not, practitioners will continue to honor it more in the breach than the practice and, ultimately, the ISD methodology will become irrelevant to the practice of ID.

Most e-Learning developers are not graduates of academic degree programs in ID. It is probably true that most e-Learning developers have acquired whatever skills they have through a combination of on-the-job training and experience, whatever they have read, and perhaps the odd three- to five-day workshop. A large variety of academic programs produce graduates who sell their skills in e-Learning development. For example, programs in cognition, psychometrics and evaluation, software (especially in fields such as EPSS, interface design and knowledge management), technical writing, media production, advertising and journalism, as well as classroom teaching and all the various content areas, all produce graduates who compete and collaborate with instructional designers. Yet the demand is so strong, and consumers so uncritical, that the market seems to be capable of absorbing the outputs of everyone who claims expertise in e-Learning and knows how to put together a Web site. The danger to the perception of e-Learning—and the ID field—have been discussed above.

The classic marketing solution to this problem is to establish a brand, and make quality claims surrounding it. We are used to companies doing this in highly competitive consumer goods markets (Coca-Cola, Microsoft, Bose, Mercedes, etc.), and there are some brands in educational service markets (the M.D. degree, the Ivy League universities, Educational Testing Service, etc.). But there are few established brands in e-Learning (PLATO® being one of the very few).

Establishing a brand is difficult, time-consuming, and costly. It requires clear definition of features, absolutely consistent adherence to the defining standards of the brand, and unrelenting investment in teaching the meaning of the brand both to the providers and the consumers.

If instructional design is to achieve recognition as adding value in e-Learning, it can do so by establishing an “ID Brand.” This is often done by development of standards, awards, and certification programs, for practitioners and for products. ID has the advantage of a good start in this area (IBSTPI, 1998), but a profession-wide, global, coherent effort is needed, which cuts across the ideological barriers that have grown up in American academic settings over the past decade or so. Not coincidentally, such an effort might have the salutary side effect of focusing all participants on definition of a single, cumulative body of instructional design knowledge, and counteract some of the needless fragmentation of the field that has characterized the past decade or so. It might be possible to obtain support for such an effort from government agencies, industry, and professional associations.

The next section examines the potentially transforming effects of the Web on education, and the implications of these transformations for future development of the instructional design field.

The transformation of education by the Web is happening more slowly than in training, and the dynamics are different at the post-secondary, K-12 school and nonschool areas. Therefore, the discussion of trends and issues needs to address each area separately.

There can be little question at this point that the explosive growth of distance education is rapidly transforming post-secondary education. As in the training arena, the primary drivers are economics and access. Distance learning is rapidly becoming a popular choice for continuing professional education, mid-career degree programs, and lifelong learning of all kinds. As so-called “non-traditional” students become an increasingly large segment of the student body at the post-secondary level, campus-based programs, whether or not they are residential, are leveling off in enrollment. So, colleges and universities see distance education as a way of sustaining growth. For some institutions, even a modest distance education program could mean the difference between a budgetary surplus and a loss, especially for tuition-driven instructional programs. Thus, distance education has an importance much greater than the enrollment figures may suggest.

For many institutions, a principal motivator is the relatively unfamiliar force of competition. For-profit, private-sector competitors are breaking the monopoly of conventional institutions, provoking a fast-moving entrepreneurial response by some schools while leaving others at a standstill. Even traditional institutions are in competition, because distance education programs can serve remote learners as well as they serve local ones. The result often seems to be an ill-considered “land rush” mentality: “we’d better stake out our claim before all the good territory is gone; we’ll worry about effectively mining it later.”

But these trends probably are not the most significant ones for post-secondary education, in the long run. Three additional trends are beginning to have an effect, and may ultimately lead to even more profound changes.

Alternatives to conventional degrees. We are seeing a growing number of alternatives to conventional academic degrees. For example, professional certification programs provided by software vendors such as Novell and Microsoft carry substantial value in the labor market. A wide range of professional organizations have certification programs centered on examinations of knowledge and performance, which provide a means to establish credentials in their field without an academic degree. Many of the training programs associated with these alternatives lend themselves well to distance education. The result is that an academic degree is no longer the sole means of entry into the professional work force. This trend may ultimately mean that certain kinds of academic degrees (such as the undergraduate liberal arts degree) may become expensive luxuries earned only by the few. It might even be that this trend eventually leads to a decline in the proportion of the workforce with academic degrees, while the proportion of the workforce with advanced training and professional certification increases.

Traditional barriers to economies of scale are falling rapidly. It is in some ways ironic that the most common model for distance education centers on the online textbook as an alternative to lectures, supplemented with discussion by way of chat groups, e-mail and the like, and accompanied by various written projects. Faculty are rapidly discovering the level of effort in producing even this instructionally weak form of e-Learning, and they are also discovering the level of effort involved in delivering the e-Learning to a group of students. Economies of scale are possible in this model, but only for the technology-based components. Thus, it is possible to predict that a market for electronic textbooks (or components of them in the form of learning objects) will emerge, and will occupy a position roughly equivalent to that of the current college textbook market. As with college textbooks, it seems likely that for every commonly offered course (or topic, using a learning object model), a handful of dominant prepackaged courses (or objects) will emerge to become the international standard for use in distance learning. Faculty members will adopt whichever of the standard packaged courses or objects appeal to them, and use the flexibility of the technology to customize the courses or objects by editing and adding their own material as they see fit. It may be that we will see more standardization of distance learning content across institutions than is now the case in the classroom. There is no reason why the local community college couldn’t offer the same introductory economics course electronic components as an Ivy League school, if only the local faculty and students are up to the challenge.

Note also the significance of the absence of economies of scale for the labor-intensive faculty support components of the distance learning model described above. Ultimately, this limitation will serve to limit declines in cost and price of distance education by this model. It may also stimulate renewed interest in more machine-intensive models of instruction such as conventional tutorial computer-based instruction.

Quality is largely undefined. Traditionally, quality of instruction at the post-secondary level is entirely the affair of the individual faculty member. Accreditation typically focuses on resources and offerings, not program performance. In distance education, however, we are beginning to see alternatives to this model emerge. The spread of performance-based testing and the concern with quality of distance education is leading a number of providers to examine models of quality that are based on learner performance (University of Illinois, 1999). These same standards may eventually begin to apply to traditional academic programs as well, probably as demand increases from prospective employers and learners. Thus, distance education could eventually be the point of leverage to develop and propagate performance-based quality standards throughout post-secondary education.

However, this trend is certainly the least sure of the three. For the most part, post-secondary curriculum standards and tests are rare even at the department level. Neither faculty nor students make much of a distinction between information and instruction, or even between factual knowledge and skill. It is too early to predict with much confidence that meaningful quality standards for instruction will emerge from the present confusion over quality in distance education. For now, at least, quality in distance education is largely undefined.

As in training, most of the development work in distance education is being done by faculty with no formal training in teaching of any kind, not to mention training in ID or any of the related e-Learning fields. Sophisticated distance education programs use instructional designers and other specialists in a supporting role to develop distance education, but this is still relatively rare. Whether development is by individual faculty member or a team, however, the need is for simple, highly templated instructional models and tools for building learning objects and entire courses. These tools must be cost-effective to use, given the relatively limited economies of scale that are likely to be possible in many distance education contexts. Current tools are little more than specialized electronic textbook creators (really simple Web site authoring tools). They embody no real instructional knowledge in the sense that the first generation of automated CBT design tools (mentioned previously) do. This observation leads to the conclusion that, as in training, ID should be at the forefront of creating cost-effective models and tools for distance education. Ideally, such models should probably have templated learning objects based on the full range of available options for computer-based instruction. They would go well beyond the now-dominant “online textbook” models for distance education, and would probably emphasize meaningful interaction to the extent that can be cost-justified. The models and tools would foster an economy of learning objects to achieve economies of scale where possible. For example, these models and tools could:

• Use classical CBT tutorial and simulation techniques only for high-volume or high-criticality course components, due to the high cost of development of these systems.

• Use case/problem-based (“constructivist”) techniques only for low-volume, high-criticality integration of learning and moderate- to ill-structured problem solving, due to the moderately high cost of development coupled with labor-intensive implementation.

• Use knowledge management as much as possible for factual knowledge, and for low-volume and low-criticality declarative knowledge and well-structured procedures.

• Integrate knowledge management, community, and tutorial techniques within the context of an entire curriculum developed in support of a certification or degree.

• Vary degree and kind of interaction and feedback depending on learning needs and types of learners.

• Use performance-based assessment which is simple to develop, and only moderately costly to use.

Fully developed models of this sort might mix on-line and campus-based learning events of various kinds, and perhaps even some conventional classroom-based seminars. However, such “traditional” program components would come to be viewed by learners as high-cost (because of the cost to them of coming to campus). Because of this, these components would tend to be reserved for high-value learning events such as face-to-face project team or community-building and work sessions, or lectures on topics too new or too ephemeral to be presented online.

It is unclear how (or if) a market will emerge for learning objects (or whatever the scalable technology-based components of distance education courses turn out to be). If a market does emerge, along the lines of today’s college textbook market or otherwise (can you imagine a Napster-like economy of learning objects?), it is unclear how (or if) such a market will value what ID can add to a development effort. Thus, a major challenge for the ID field is to get faculty and learners to recognize the value of sound instructional design (regardless of the theoretical framework used to define it). In the absence of meaningful performance-based quality standards for distance learning, this may not be possible.

The conclusion, therefore, is that faculty, the distance education organizations, the learners and their employers, and the ID field all have a common interest in performance-based definitions of quality. In the short term, ID faculty should lead the effort to shape institutional policies on distance education quality standards and accreditation, and should actively engage in faculty consultation and development. The time for such initiatives is now. To abuse a cliché, “the train is leaving the station, but it’s not clear that ID, as a field, will be on board.”

The same rationale leads to the conclusion that ID faculty should work to influence the structure of the emerging learning object economy. XML-based metatag standards such as the IMS project are a first important step. ID faculty should work to steer these efforts in the direction of creating a market of courseware components with easily-understood instructional attributes as well as content attributes. To do so, we need a universal and easily understood language to describe instructional attributes regardless of their origins in various models of instruction and learning. Such a language, used as a basis for metatags, can facilitate the economies of scale which make possible high-quality ID work.

The driving forces for e-Learning in the training and higher education sectors are cost reduction and convenience of access, leading to tremendous emphasis on distance education. Thus far, however, this has not been the case at the K-12 level. The reason is simple: the costs of K-12 usually are non-discretionary and fixed by state regulations and tradition. Thus, except for the occasional experiment at a charter or alternative school (see next section), technology does not offer cost savings; it is usually an incremental cost. The claims of 30 years ago that software was “teacher-proof” and the fears that technology would be a “teacher replacement” have (thankfully) been shown to be largely unwarranted. Instead, productivity gains expected from technology are mostly focused on improvements in measured learning outcomes.

Four drivers for technology in K-12 schools are apparent:

The primary driving force for technology, especially at the secondary level, is the national movement toward curriculum standards and accountability as means of education reform. The heavy emphasis by ESEA Title I on assistance to underprivileged elementary children essentially created the elementary educational software market (including integrated learning systems) and structured that market for its first 20 years. Although ESEA has been expanded to K-12, it is still the main source of funds for elementary software.

Another traditional source of spending on software has been the desire to improve curriculum quality through supplemental software such as “edutainment,” drill and practice, references, tools, and online supplemental materials and events such as “electronic field trips.” Additional emerging forces surround improved access and convenience, such as distance education efforts for special-needs students and for small, poor and rural schools.

All of these factors have been important in the development of the PLATO Learning System^®, PLATO stand-alone CD-ROM products, and the PLATO Web Learning Network^®. Virtually all market-driven suppliers of educational software for schools are responding to one or more of these four drivers of demand.

We have observed that “best practices” in improving learning outcomes through use of technology seem to use the following four strategies.

1. Supplemental review and reinforcement, whether in the classroom or in out-of-class supplemental programs based in after-school or even online homework programs. Such programs often use drill-and-practice software (including most “edutainment” games), and some use PLATO-style tutorials. When implemented on a large enough scale, relatively modest increments in learning seem to be the result.

2. Complementary/enrichment: use of technology to add to the curriculum high-quality learning experiences which deepen understanding and promote higher-order thinking skills. This includes many problem- and project-centered software applications created using “constructivist” models, such as tools, resources, case problems and simulations. When done on a large enough scale, significant gains in understanding and transfer have been observed.

3. Teacher leverage/ individualization: use of technology to implement sub-stantial individualization. Often includes complementary/enrichment activities, but also includes direct computer-based assessment, tutorial instruction, and instructional management. May use a mastery learning model. This model is often used in programs designed to increase access. When done well, substantial improvements in learning result.

4. A potential strategy, yet rare in software, is use of technology to provide a teacher scaffold. As poor teacher training and the shortage of well-trained teachers is recognized as a major barrier to school improvement (especially in math and science at all levels, and in reading at the secondary level), technology is beginning to be called upon to scaffold the teacher as well as the learner.

School spending on technology has been growing dramatically. School software spending is now in the realm of $1 billion annually (Moe & Blodgett, 2000). However, even this level of spending constitutes only a fraction of 1% of the $375 billion budget of schools, clearly not a strategic level of expenditure. By contrast, textbook spending is typically around 2% of budget; all program-related expenditures other than salaries and building-related costs total typically in the 10% range. By this measure, technology in schools still has a long way to go before it reaches the level of systemic transformation comparable to what is happening in higher education and training.

K-12 education uses a wide variety of software types, including tutorial, drill-and-practice, games and simulations, information, tools and assessment. Regardless of the software type, the greatest need is for software that:

• Is comprehensive in coverage of curriculum standards

• Is modular, customizable, and designed for large-scale individualization

• Includes built-in assessment

• Requires a minimum of teacher training and effort to integrate into a curriculum and use.

• Supports administrative requirements for accountability and change management.

In addition, an emerging requirement was identified above for software to be designed for “teacher leverage” and “teacher scaffolding” as well as the learner. If these are the needs, there is ample demand for instructional design research and theory in a number of areas for which there has been little or no focus in recent years. Specifically,

• Classic instructional design research has established well-understood and well-validated guidelines for directly teaching and measuring declarative knowledge and well-structured procedural knowledge via tutorial CBT.

• While there has been considerable progress in establishing design principles for moderate- to ill-structured problem solving (using case problems, games, and simulations), there is still much work to be done on strategies using computer-supported collaborative learning (CSCL), instructional simulation design, and computer-based measurement of higher-order learning outcomes.

• Integration of instruction, knowledge management, and performance support, a new development in training, has yet to make a meaningful appearance in educational software.

• Very little research has been done on computer-based teacher training and embedded teacher scaffolding. Research on professional communities of practice as a means of improving teacher skills is in its infancy, and it remains to be seen if such communities will be effective with the majority of teachers and contexts.

• The greatest promise of technology is still for universal individualization. There is an urgent need to update research on issues such as instructional management, learner and system control—and even learning styles—to reflect current cognitive and constructivist learning theory.

• There is a continuing need to study the process of change management in education systems, especially since society as a whole is placing a large bet on the ability of technology to drive education reform. Technology-based innovations are of little significance unless they are cost-effectively scalable to thousands of sites and millions of students.

• The task of integrating technology into the curriculum overwhelms all but the most dedicated and talented teachers. Research is needed on ways to make this task much less labor-intensive. For example, a standardized language for describing the features and benefits of educational software (and relating them to curriculum standards) would allow dependable automated databases of software resources to be created on a national basis. Currently, a teacher knows more about the food products he or she buys on Saturday (due to uniform nutrition and ingredient labeling regulations) than the same teacher can know about the software he or she picks from a catalog or search engine on Monday. Under present circumstances, only by trial and error can a teacher select and integrate software effectively.

Looking farther ahead, it is difficult to predict how the economies of scale afforded by technology will propagate in the K-12 educational system. As in post-secondary education, the economies of scale of the technology-based components of the solution will create substantial pressure (in the form of price and availability differences) toward national standards. Compared to software useful nationally, curriculum components of purely local or specialized interest will carry a higher price and/ or lower quality, if they are available at all. In time, pressures will emerge to minimize local differences and special needs, or to support them by means of publicly-funded software development. It is unrealistic to expect teachers to develop their own instructional materials in large quantity, given the state of the art (though they may develop instructionally useful resources, lesson plans, and other kinds of information).

It also is unclear how (or if) technology’s economies of scale will reach learners if schools do not exploit the potential. One possibility is already emerging: direct delivery of educational services to homes via the Internet and broadcast media (especially the “back channels” of high-definition television). We already have a model of the power of such services to change education, in the form of “Sesame Street” and its effect on the K-1 curriculum. It is easy to project even greater transformations via the digital media—at least for middle-class parents with the disposable income to spend on such services.

If the economies of scale possible with technology are able to change the cost structure of primary and secondary schools, ID research should lead the way. For example, research is needed on how technology can differentiate roles in the educational system. Similarly, research is needed on highly individualized school structures with no grade levels and no classes of 25. In another example, there is a need for a sophisticated economy of courseware components (similar to the learning object economy that may be emerging in the post-secondary and training distance education fields). ID researchers could lead the way in adaptation of the existing IMS standards for schools. In a fourth example, current “constructivist” models generally aren’t scalable; they require too much high-quality teacher input to be generalizable across most schools and most teachers, given realistic teacher training expenditures. ID researchers can create technology-facilitated constructivist models that place more modest and feasible demands on teacher skill level and time.

Finally, the instructional design field is well-positioned to educate teachers, administrators, and policy makers on issues of implementation of technology in the schools. It is common for teachers, administrators, and policy makers to have limited vision for what technology really can accomplish in schools, coupled with naïve expectations of dramatic results and little or no meaningful implementation planning. Even points as basic as the difference between information and instruction are not widely understood, making informed selection of educational software and integration into the curriculum a rarity. Policy decisions on issues such as optimum student-to-computer ratio are made without reference to the desired instructional models, and often without benefit of clearly articulated goals for improvement of teaching and learning through technology.

Meanwhile, principles for diffusion and adoption of innovation, widely studied in the 1960s and 1970s, have been almost forgotten today (except in business, as “change management.”), and are rarely taught even within instructional technology curricula. There have been examples of ID-led change management efforts (such as Florida’s School Year 2000); there needs to be many more.

These trends do not mean that there is no interest in technology as a means of reducing (or reallocating) the cost of schools and increasing convenience and access, parallel to the trends in higher education and training. Thus far, however, this kind of technology use has been limited to some relatively small but important “non-traditional” educational niches. These include certain kinds of charter schools and alternative schools that are experimenting with a variety of alternative ways of structuring school (and including some non-school-based systems), some “virtual schools” aimed at isolated, mobile, special needs, rural, and homebound students, and—perhaps most notably—home schools. It remains to be seen if these niches will eventually grow to constitute a serious challenge to the hegemony of the conventional school model. While they serve only a small percentage of K-12 students, their rate of growth is impressive (Moe & Blodgett, 2000).

It is possible to speculate from the experience of these programs concerning the nature of teaching and the skills required in future technology-based educational systems. In our experience, a common thread across many non-traditional approaches is a radical change in the role of the instructor. In many programs, an instructor with content expertise is in the loop indirectly, if at all. Much of the primary work of instruction is carried by technology, while teachers focus individually on the learners and their cognitive and social development, performing the role of instructional manager and “guide on the side” rather than “sage on the stage.” We have observed informally that the teachers who thrive in non-traditional settings often are more interested in their students’ development than the curriculum content. They often feel no compelling need to develop a high degree of expertise in the content they teach; they are more interested in working individually with the learners. If this pattern turns out to be generally true, it might mean that in the ideal high-tech educational program, the most important teacher skills will be of the “high touch” (learning process) variety, with content expertise given secondary importance. Differentiated staffing patterns could be one result. Another result could be a radical change in the emphasis of teacher training programs and the selection processes for the profession.

So much is happening in non-traditional settings, and so little research has been completed, that there is an enormous need for ID research in this area. For example, it may well be that today’s non-traditional models, which often individualize heavily using instructional technology, suggest the path to the future of “mainstream” education. Often these programs use differentiated staffing models, and often they are community-based, do not use large school buildings with “egg crate” classroom designs, and often they follow non-traditional schedules and calendars. These may be the strategies by which the schools in general can use technology to change its cost structure. However, the requirements of non-traditional settings have received only a small amount of research and are not well understood. The systems thinking of an ID researcher can be invaluable in investigation of these models.

Similarly, distance education research is still in its infancy, especially for school-aged learner populations. Again, the ID perspective is particularly useful here.

Already, however, it is clear that issues of quality and assessment are as critical here as in traditional forms of education, but non-traditional programs often must “prove” their worth in ways not expected of “mainstream” schools. The ID perspective can inform evaluation strategies which often ask only naïve questions about technology.

Because the cost-benefit characteristics of non-traditional programs are very different from the traditional conventional educational systems, we can predict that there will be substantial implications for what kinds of technology solutions will be feasible and cost-effective. Again, ID research on feasibility of technology in these contexts is needed.

The overwhelming demand of the training field for people with ID backgrounds has had the effect of diverting much attention of the field away from education. Now that technology is finally entering the educational system in substantial quantity, we can hope that the trend reverses.

The explosion of opportunities outlined in this chapter means that we can evolve and grow the theory of instructional design in many directions. At the same time, however, researchers and practitioners are facing such a demand for their talents that getting support and time for disciplined research and theory building is often extremely difficult. We live in “interesting times,” but it may be a bit like the ancient Chinese curse.

For instructional design and technology, this is “stand and deliver” time. Professionals in the field are finally getting their chance to make good on the visions of learning transformed by technology. But we have neither unlimited time nor unlimited resources to prove our worth to the leaders who are now advocating use of technology in training and education. If the expected results aren’t realized, it won’t matter which learning theory, design methodology, academic program or software company did or did not succeed. No matter who we think the “winners” and “losers” are, if the public and policy makers do not see their expectations fulfilled, the credibility of technology as a transformative force will be damaged. It is incumbent upon all professionals with a commitment to the potential of technology in education and training, no matter what their theoretical or ideological bent, to collaborate and advance the common vision. As much as our understanding of technology in education and training has developed over the past 40 years, we still understand only a small fraction of what we need to transform the craft of instructional technology into an engineering discipline (or science, if you prefer). Given the challenges we face, practitioners in the field should have no use for ideological bickering among various theoretical positions. No single line of research can possibly lay a unique claim to wisdom and understanding. There is much to be accomplished, and little time to do it. Let us proceed, together, with the hard work of building a cumulative and unified base of knowledge for e-Learning.