Table 2: Summary for eight studies with largely positive ALN findings

For these eight studies in Table 2, the preponderance of the evidence is that ALN is more effective than traditional courses.. All of the results indicate ALN to be better, or else some results show ALN as better and others show "no difference." The studies that are judged to be in this category include those by Alavi; Andriole; Benbunan-Fich et.al. (2010), Hiltz; Hsu et.al.; Hiltz & Wellman; Thoennenssen et. al; and Turoff and Hiltz.

For the remaining 12 studies, the preponderance of the evidence shows "no significant difference" between ALN and the traditional courses or sections or experiences that are compared. Either all the results show "no significant difference," or there are mixed results with results better for ALN for some courses or measures, and worse on others (e.g., for the SCALE projects at Illinois [Arvan, 1998], there are some conflicting results related to course and/or experience of the instructor.).

There are no qualifying empirical studies for which ALN is clearly shown to be less effective than traditional modes of course delivery, on the whole.

The first important point to note is that the "No Difference" cases really indicate that ALN is just as effective as face to face and when this is added to the positive results for ALN there is a four to one ratio of positive results to negative results in these 19 studies.Furthermore, if we realize there are only four instances of objective measures that are negative and none of them are direct measures of learning, we find the results rather overwhelming support the hypothesis that ALN is a meaningful alternative to the classical face to face class, which tends to be as effective or more effective, depending on the circumstances of the particular implementation and the measure used.

In terms of some of the negative results, the course completion or drop out rate is probably higher than it should be because of mistaken expectations by students, either because they are new to the use of ALN or because they do not have the student network to warn them away from particular offerings where the course might not live up to the description. In fact, many studies of ALN show that the role of the instructor and his or her ability to deal with this new mode of learning is a principal factor in ALN success. Teaching ability is important, as well as experience in the mode of delivery. A lot of face-to-face courses may be taught by less than perfect instructors but the face to face environment can tolerate a wider range of instructor abilities. We must evolve a mechanism to specify in evaluation work the competence of instructors or it will be quite clear that the results can easily be confounded.

Most universities have a standardized survey for measuring student reactions to an instructor at the end of the course. It is only this year that the union at NJIT agreed to a slightly modified version of a teaching evaluation course to be put online for all ALN courses, so all on line courses will receive the same survey and those teaching online sections can be directly compared with respect to their face to face sections and with others teaching the given course. It is this sort of data that should be assessed longitudinally to determine how the performance of the instructor evolves over time, and which instructors are the important ones to carefully debrief in order to determine relative success factors in the teaching of a given course.

Table 3: Summary for eleven studies with largely mixed or No Difference" ALN findings

Some of the variables that are related to the degree of effectiveness of a particular course implementation are suggested by the various correlations with learning effectiveness reported in the studies included in this analysis. These are shown in Table 4. There are not enough replications testing specific relationships to reach any firm conclusions on these relationships at this point, but they do suggest hypotheses to be tested in future studies.

_____________________________________________________________________________

Table 4: Correlations or Interactions Reported for the positive studies.

• Perceived collaboration correlates with perceived learning
• Activity by individuals correlates with Perceived learning
• Perceived collaboration correlates with motivation
• Higher objective Grades in Computer Science correlates with the use of ALN
• SAT Scores and/or GPA are the dominant correlation with course grades (independent of ALN or face-to-face)
• Initial subjective expectations correlates with later actual participation
• Course type and mode of delivery interacts.
• Degree of collaboration correlates with satisfaction with course

Correlations or Interactions Reports (unique to one study) for the Mixed or No Difference Studies

• Use of quizzes (for feedback only) improves learning perceptions by instructors and students
• Collaboration falters when there is not sufficient critical mass.
• Laboratory use and collaboration are correlated in a Chemistry course.
• Collaborative groups online have higher satisfaction than individuals online.
• Group Communications with Learning Game led to higher learning levels than with either alone.
• No benefit found for use of telephone communications
• Dense Hyper-links interfere with ease of learning.
• More efficient management of distance students possible
• Larger courses in distabce posibble

Technology problems correlate with dissatisfaction by students.

____________________________________________________________________________

With respect to student feedback it is not surprising to find from a number of studies that face to face students often think that the quality of their work or the quality of discussion is better face to face, when in fact the opposite may be true when expert judges are used. This is similar to when an employee changes from using a manual approach to using a computer approach to handle their tasks. The way one is used to doing work is usually felt to be a better way than having to learn a new one. It often takes a long interval of time for the employee or anyone else to get used to a new way of solving problem or learning and to begin to actually realize they are doing well.

It is also time to capitalize on the records that exist for students to begin to conduct some longitudinal analyses on students.We need to relate a student's subjective opinions about an ALN course to how many such courses they have taken.We must view their grades in relation to how they have preformed in the prerequisite course and their overall grade average, in order to help to control for possible effects of self-selection of mode of delivery.In this manner we can begin to get a handle on the true long-term impacts of ALN. One would suspect that a longitudinal analysis will begin to show that those who actually desire to use the ALN alternative (as compared to those who are basically "forced" to take an ALN section because there are no available alternatives) do better than those in comparable face to face classes