Cost-effective computer tutorials
Alan L Tyree

A new form of computer tutorial system was tried in International Law at the University of Sydney in the first semester of 1992. The form and theory of the tutorials are described and the outcome of the trial.

International Law is a compulsory course usually taken in the 4th year by Sydney University Combined Law students or in the 2nd year of the Graduate stream. It is a "two unit" course which means that it normally meets twice a week. Each meeting is two hours in length, and the course has traditionally been taught using lecture methods supplemented by eight one-hour tutorial periods. The total duration of the course is one semester.

The course covers both private and public international law. There has been an attempt in recent years to teach the two topics in an "integrated" fashion. The tutorials have generally not reflected this integrated approach. There have been four tutorials primarily devoted to private international law topics and four devoted to public international law.

The total enrollment in the course is approximately 260 students. There are three different groups, and in accordance with Faculty policy at the University of Sydney each group may be examined separately if the teachers so desire. The aim of this policy is to permit and encourage diversity of approach to subject matter.

In the first semester of 1992, two of the three groups were taught jointly. The teachers of these two groups agreed to a pilot study which would use the CRES computer examination system (see below) as a vehicle for computer based tutorials in the subject.

Students in these two groups were offered a choice of ordinary tutorials or computer based tutorials. They were given a comprehensive demonstration of the computer system, so that their choice should have been informed.

Of the 154 students present at the time when the choices were made, 70 chose the computer tutorials. 166 students sat the first examination in the course, 73 of them having participated in the computer tutorials.

The computer tutorials were structured so as to parallel as closely as possible the existing tutorial structure. Each computer tutorial was on the same topic as the corresponding traditional tutorial, and the design of the computer tutorials attempted to ensure that the time required was approximately the same as for the corresponding traditional tutorial.

CRES tutorials (described in detail below) do not follow the traditional style of computer based teaching (CBT). That traditional style might be called "pseudo-Socratic". A certain amount of information is provided to the student and a question usually multiple choice) is then asked. On the basis of the response to the question, the system then presents the next unit of information.

The problem is that to be effective, the builder of the traditional system must anticipate all possible responses that the user might give. Not only that, but the builder must anticipate the reason for the response since the system must "branch" on the basis of the response.

This traditional approach to CBT is very demanding. Research in the area repeatedly shows that 100 - 400 hours of work is required to build a one hour tutorial. Further, the construction methods are such that modification is nearly as expensive as the original construction, a devastating prospect if we are considering building good tutorial systems for dynamic areas of law.

By contrast, a good CRES question requires about one and a half hours to build. A one hour tutorial requires seven or eight CRES questions. The tutorials are "modular" in the sense that a change in the law will usually require change only to a few questions in the tutorial, and these changes may often be minor ones which can be accommodated with a few minutes work.

The tutorials are based on ideas developed by the educational theorist B S Bloom. He and his colleagues at the University of Chicago developed teaching methods which were the first to yield measurable improvements in student performance. At about the same time F S Keller was developing his "personalised system of instruction" which also showed dramatic improvements in student performance. The story begins in 1968.

In order to understand the details of the CRES tutorials, it is valuable to make a short digression into the theory of educational testing.

The International Law CRES tutorial system consists of eight CRES examinations. Each examination contains between six and ten CRES questions. Students may use the system at any time when the Faculty Computer Laboratory is supervised. During the course of the experiment, the Laboratory was supervised four hours a day, five days a week.

Private International Law was examined separately in the two sections which participated in the CRES experiments. The examination was conducted immediately before the Easter recess and after the students had completed four tutorial session.

The CRES students did slightly better in the examination, although the results were not statistically significant. The average mark for the "standard" students was 63.39 with a standard deviation of 12.65. The average mark for the CRES students was 65.36 with a standard deviation of 11.70.

The results of the final examination were not available at the time of writing. A student evaluation of the CRES system is planned, but again the results are not available at the time of writing.

These results need to be treated with a great deal of care. Both the "human" and the CRES tutorials were optional, and detailed participation records were not kept. A student was included in the CRES group if he or she had used the system at least once.

Assuming that the groups are statistically indistinguishable, it must further be remembered that both groups are self-selected. There is no evidence to jump to the conclusion that CRES tutorials are as good for everyone.

What kind of student selected the CRES tutorials and why? We do not know at this time. It may be that the student evaluation will throw some light on this. Informal discussions with the CRES students provided some clues: "timetable freedom"; "Some students dominate the normal tutorial"; "I am able to sit in an ordinary tutorial without doing any work"; "Comfortable with computers"; "Just thought I would like to try something different". Colleagues opposed to the idea of computer assisted learning have other explanations: "did not like to get involved"; "poor verbal skills"; "too lazy to prepare for tutorials".

We judge the CRES tutorials to be a success. There is a certain amount of Faculty suspicion and some outright opposition. There is also some opposition from the student politicians which appears to be based on a fear that we are planning on replacing human tutors with computers.

Much of the suspicion and opposition appears to be based on the informal comparison of computer tutorials (of any kind) with human tutorials. We are prepared to accept that human tutors are better than computers, although we note that there is not the slightest shred of evidence that this is so if the tutorial groups are of the size that we have at the University of Sydney.

In our view, the argument based on this comparison simply misses the point. The choice is not between human tutors and computers. It is between computer tutorials and no tutorials at all. It is this latter situation which is the norm in most of our classes at the University of Sydney.

What our efforts have shown is that the computer CRES tutorials are useful and that we can afford them. This means that we can now proceed to introduce them in classes where we presently cannot afford to provide tutorials in any other form. In those classes where we do provide human tutorials, we will encourage CRES tutorials to be offered as an option to students who prefer them. Such an option has the dual advantage of offering students a greater choice and reducing the size of the "human" tutorials.

The Keller Plan Project and the development of the CRES and SAGES software have been supported by generous grants from the Law Foundation of New South Wales. The Keller Plan Project is managed by Alan Tyree and Shirley Rawson.

The International Law tutorial option is partially funded by a Law School/Law Foundation grant to the following applicants: Hutchinson, Musgrave, Opeskin, Rawson, Rothwell, Skapinker, Tyree