4. Conclusions and Discussion

The evidence is overwhelming that ALN tends to be as effective or more effective than traditional modes of course delivery, at the university level. There really is no need for more studies to explore this question. What we need is more research that will enable us to make it even more effective, especially as new technologies proliferate. For example, most early ALN's were "text-only" discussion; many now include multi-media web based tutorials instead or in addition to a textbook, and possibly audio and/or video interaction with the instructor. How important is multi-media in student teacher and student-student interaction, and how may it best be used? With very small and pervasive devices coming into use (e.g., wireless and pocket PCs), how can these devices best be used to further improve convenience of access to ALN's? What forms of collaborative learning vs. individual assignments are most appropriate for different kinds of courses or class sizes? As Alavi and Leidner(2010) point out in their call for "greater depth and breadth of research" on technology-mediated learning, research in this area must look at how technology influences learning, which involves an "explicit consideration of relationships among technology capabilities, instructional strategy, psychological processes, and contextual factors involved in learning" (p. 1). We need to develop both more sophisticated and more comprehensive theoretical frameworks, and also more valid methods and instruments that those which have characterized a majority of studies to date.

In conclusion, it is time to start looking at the evaluation of ALN beyond the single course and consider the full range of evaluation variables. Consider the following top down structure of evaluation variables that might form an overall model for the study of ALN effectiveness and impacts:

• Amount of learning
• Time to Degree
• Satisfaction
• Motivation
• Enjoyment
• Participation

• Effort to Learn
• Time to Learn
• Time to educate
• Cost of resources
• Effort to access learning
• Availablility of course as needed
• Convenience of courses

• Accessibility for new populations
• Reduce or eliminate distinction between distance and regular students
• Make direct use of human resources not normally available in a course
• Introduce new programs
• Enjoyment
• Participation

• Change the nature of courses
• Change the nature of degree programs
• Change the nature of educational institutions
• Change the nature of departments
• Change the nature of teaching
• Change the evaluation criteria of faculty

• Impact on individual (job, earnings, advanced study. etc.)
• Reputation of Program
• Reputation of Institution
• Introduce new programs
• Accrediation of Program

• Type of Students (objectives, part time, learning abilities, gender, age, experience, and other demographics)
• Type of Course (Laboratories, skill, subject area, etc.)
• Teaching Methodologies employed (e.g., is discussion graded, are individual or group assignments and activities used)
• Size of course
• Information on courses available to potential students
• Technology employed (media mix, usability, fuctionality)

From the above and the preliminary correlations based upon courses as well as our understanding of higher education, it is time to begin formulating testable models that involve many of the key variables, and to begin to use meaningful historical data to verify or improve our models.

For all the talk of university research, as with most commercial organizations, universities are reluctant to expose the critical historical data that they have, for fear of negative publicity. The few times that this has succeeded in studying industry it has tended to be the result of an independent and non competitive organization established that would guarantee the anonymity of detailed data supplied to the organization when it came to the publication of the resulting analysis reports. The incentive for the organizations participating was to be able to compare their performance with others and in so doing, have a better idea of how to make decisions to improve their situation.

It might be that the formulation of such an independent depository of historical information is one of the next important steps in furthering the evaluation of ALN. While we have done a reasonable job, as an academic community, in proving the viability of ALN, our form of distance learning is probably still a minority in terms of the methods whereby distance learning is being offered. ALN researchers tend to believe that those approaches that do not include extensive communication and interaction among students and faculty are inferior to ALN approaches as well as face to face education, but this message has not "gotten through" to a lot of distance course designers.We need to raise our evaluation and research dissemination efforts to a new plateau and use the opportunity presented to provide a deeper understanding of higher education effectiveness than exists to date.