Tuesday, November 13, 2012

The Difference Between a Reference Point and a Criterion

The Difference Between a Reference Point and a Criterion

Edward Renner

Donald Eastman III, the President of Eckert College, wrote an op-ed piece in the Tampa Bay Times about the limits of online learning: “…what works for most students…is a small classroom…where a respected authority…is a spellbinding revealer of mysteries – not simply because he or she knows things we don’t, but because a gifted teacher reads the audience the way an actor reads the room…”

On July 17, the University of Toronto announced that it had joined Coursera.  In response, Clifford Orwin, a professor of political science at the U of T, wrote in The Globe and Mail that “the classroom experience is at the heart of education…The electricity that crackles through a successful classroom can’t be transmitted electronically.”

Pamela Hieronymi in her essay in the Chronicle declared that the capacities of online technology “should not be confused with the training provided by one mind interacting with another.”

In short, the chorus of critics is that online and virtual is a shoddy imitation of the real thing. Such declarations miss the point. They are assertions that the ideal traditional classroom is the real criterion against which online should be compared, rather than serving as a reference point for comparison with other alternatives.

The issue of whether the new technologies are consistent with a hypothetical ideal appropriate for the specific circumstance of lecturing to a captive audience at a fixed time and place is a meaningless theoretical exercise. The essential exercise is comparing this particular circumstance with other circumstances using an objective external standard.

The standard at one extreme is a situation in which hardly anyone learns anything. At the other extreme is one in which almost everyone learns everything. These two limiting distributions can be plotted on a graph in which the X-Axis is the proportion of the material learned and Y-Axis is the proportion of the class.

In practice, of course, both limits can only be approached. Every class results in an actual distribution defined by the standard deviation around the average amount learned. The distribution for any class can be plotted on the same graph as the two limiting cases. This simple graphic provides an objective external standard for comparisons between different circumstances and different teaching methods. The only question is, on the average, how close does any particular effort approach a limit, and what is the spread between the students who are most and least successful?

The classic example this kind of research has been carried out over the past several decades on the teaching of large enrollment introductory physics classes. Typically, students in these classes could calculate answers to problems using formulas, but they were unable to apply the concepts to answer simple basic questions.

Harvard Professor Eric Mazur found that after a semester of lecturing, on the average, students understood at best about 30% of the material. However, 60% understood the material when it was presented online, and the classroom was “flipped” to practice applying the concepts in small discussion groups.

Professor Carl Wieman of the Science Education Initiative at the University of British Columbia has carried out controlled experimental studies on this method. In a recent study published in Science he found that the online presentation of the material followed by peer group discussions in the classroom more than doubles the average amount of material mastered. In addition, 90% of the students reported enjoying the interactive teaching techniques more than traditional lectures; while only 1% disagreed (8% were indifferent). In addition, levels of student engagement and attendance were significantly higher with the flipped classroom.

This is the type of information that needs to be informing policy discussions over the relative effectiveness of different circumstances and methods of teaching, not declarative statements comparing the new digital communication techniques with a theoretical classroom.

What is of theoretical importance is identifying the variety of dimension that account for the means and standard deviations of the distributions of actual students, under different specific circumstances. Like all such comparisons, there are large individual differences. The result for different groups provides comparative empirical reference points; none of which are an ultimate criterion.

We might suspect, much like a flipped classroom of today, that back when experienced professors interacted with students personally known to them in small classes, that the average amount learned was relatively larger compared to classes today. Currently, many large lecture classes are often taught by overworked adjunct professors who often do not even have on-campus offices. Given budget constraints that trend is likely to continue.

Also, we might reasonably assume that a technologically challenged professor would do much worse trying to teach on line, than doing so face-to-face, no matter how large the live class. Just as the newly appointed Millennial professor might do much better using the new technologies rather than trying to teach using face-to-face lectures. A class of adult learners may very well respond differently to the two modes of teaching relative to a homogenous age cohort of Millennials.

Such individual differences as these are of great theoretical importance. They can be empirically identified and dealt with strategically by doing the best job possible with the resources we have. These differences will not be addressed, however, by refusing to accept responsibility for change ourselves in light of the many new circumstances and teaching methods now available.

In the fall of 2007, after a 15 years absence from undergraduate teaching, I became an Adjunct Professor in the Honors College. I figuring a small class for me to enchant would enhance my retirement. The course met three times a week and had three required full length textbooks. Now, in 2012, there are no textbooks. The Monday and Friday classes are virtual, there are no (zero) classroom lectures. All substantive material is delivered online. Students write, comment and challenge each other throughout the semester, meeting on Wednesday for a moderated exchange of ideas.

My biggest surprise was how much easier it is now, with 21st Century digital technologies, than it was before to have even higher levels of student engagement with each other, the material and the professor. The technologies are more respectful; they allow students to do their work in the time and space that best fits their life and their circumstances – which for many includes a job. Socially, they are more collaborative and participatory. Technically, they allow efficient access to material that is more comprehensive, engaging and up to date.

Each year as the class became more online with less lecturing, the student evaluations and level of performance went up. The quantitative evaluations are now exclusively positive and “strongly agree” the most frequent response to all items. Having gone from three to one formal class each week has raised the sobering possibility that zero might be even better. I expect for some, perhaps even the majority, that that might be the case.

However, I quite enjoy the weekly meeting, and rather than face that possibility my current scholarly effort is focused on creating a metric for social science and humanities courses that, like the concept test in physics, can be used to measure changes in the level of cognitive complexity and critical thinking that takes place over the term. My subjective evaluation alone is not sufficient.

We need to recognize that the art and science of teaching and learning in the 21st Century is now different. Our challenge is how to systematically go about using the new technologies to enhance teaching and learning without making declarative statements bases on our beliefs, as if they were something more than just that. We have the capacity to reflectively apply the science and critical thinking we teach our students to what we ourselves are actually doing. Our own teaching is the ideal place to demonstrate the power of scientific inquiry and critical thinking that we claim to be our non-replaceable purpose as teachers.

Edward Renner teaches in the Honors College at the University of South Florida

No comments:

Post a Comment