Tuesday, November 13, 2012

The Difference Between a Reference Point and a Criterion

The Difference Between a Reference Point and a Criterion

Edward Renner

Donald Eastman III, the President of Eckert College, wrote an op-ed piece in the Tampa Bay Times about the limits of online learning: “…what works for most students…is a small classroom…where a respected authority…is a spellbinding revealer of mysteries – not simply because he or she knows things we don’t, but because a gifted teacher reads the audience the way an actor reads the room…”

On July 17, the University of Toronto announced that it had joined Coursera.  In response, Clifford Orwin, a professor of political science at the U of T, wrote in The Globe and Mail that “the classroom experience is at the heart of education…The electricity that crackles through a successful classroom can’t be transmitted electronically.”

Pamela Hieronymi in her essay in the Chronicle declared that the capacities of online technology “should not be confused with the training provided by one mind interacting with another.”

In short, the chorus of critics is that online and virtual is a shoddy imitation of the real thing. Such declarations miss the point. They are assertions that the ideal traditional classroom is the real criterion against which online should be compared, rather than serving as a reference point for comparison with other alternatives.

The issue of whether the new technologies are consistent with a hypothetical ideal appropriate for the specific circumstance of lecturing to a captive audience at a fixed time and place is a meaningless theoretical exercise. The essential exercise is comparing this particular circumstance with other circumstances using an objective external standard.

The standard at one extreme is a situation in which hardly anyone learns anything. At the other extreme is one in which almost everyone learns everything. These two limiting distributions can be plotted on a graph in which the X-Axis is the proportion of the material learned and Y-Axis is the proportion of the class.

In practice, of course, both limits can only be approached. Every class results in an actual distribution defined by the standard deviation around the average amount learned. The distribution for any class can be plotted on the same graph as the two limiting cases. This simple graphic provides an objective external standard for comparisons between different circumstances and different teaching methods. The only question is, on the average, how close does any particular effort approach a limit, and what is the spread between the students who are most and least successful?

The classic example this kind of research has been carried out over the past several decades on the teaching of large enrollment introductory physics classes. Typically, students in these classes could calculate answers to problems using formulas, but they were unable to apply the concepts to answer simple basic questions.

Harvard Professor Eric Mazur found that after a semester of lecturing, on the average, students understood at best about 30% of the material. However, 60% understood the material when it was presented online, and the classroom was “flipped” to practice applying the concepts in small discussion groups.

Professor Carl Wieman of the Science Education Initiative at the University of British Columbia has carried out controlled experimental studies on this method. In a recent study published in Science he found that the online presentation of the material followed by peer group discussions in the classroom more than doubles the average amount of material mastered. In addition, 90% of the students reported enjoying the interactive teaching techniques more than traditional lectures; while only 1% disagreed (8% were indifferent). In addition, levels of student engagement and attendance were significantly higher with the flipped classroom.

This is the type of information that needs to be informing policy discussions over the relative effectiveness of different circumstances and methods of teaching, not declarative statements comparing the new digital communication techniques with a theoretical classroom.

What is of theoretical importance is identifying the variety of dimension that account for the means and standard deviations of the distributions of actual students, under different specific circumstances. Like all such comparisons, there are large individual differences. The result for different groups provides comparative empirical reference points; none of which are an ultimate criterion.

We might suspect, much like a flipped classroom of today, that back when experienced professors interacted with students personally known to them in small classes, that the average amount learned was relatively larger compared to classes today. Currently, many large lecture classes are often taught by overworked adjunct professors who often do not even have on-campus offices. Given budget constraints that trend is likely to continue.

Also, we might reasonably assume that a technologically challenged professor would do much worse trying to teach on line, than doing so face-to-face, no matter how large the live class. Just as the newly appointed Millennial professor might do much better using the new technologies rather than trying to teach using face-to-face lectures. A class of adult learners may very well respond differently to the two modes of teaching relative to a homogenous age cohort of Millennials.

Such individual differences as these are of great theoretical importance. They can be empirically identified and dealt with strategically by doing the best job possible with the resources we have. These differences will not be addressed, however, by refusing to accept responsibility for change ourselves in light of the many new circumstances and teaching methods now available.

In the fall of 2007, after a 15 years absence from undergraduate teaching, I became an Adjunct Professor in the Honors College. I figuring a small class for me to enchant would enhance my retirement. The course met three times a week and had three required full length textbooks. Now, in 2012, there are no textbooks. The Monday and Friday classes are virtual, there are no (zero) classroom lectures. All substantive material is delivered online. Students write, comment and challenge each other throughout the semester, meeting on Wednesday for a moderated exchange of ideas.

My biggest surprise was how much easier it is now, with 21st Century digital technologies, than it was before to have even higher levels of student engagement with each other, the material and the professor. The technologies are more respectful; they allow students to do their work in the time and space that best fits their life and their circumstances – which for many includes a job. Socially, they are more collaborative and participatory. Technically, they allow efficient access to material that is more comprehensive, engaging and up to date.

Each year as the class became more online with less lecturing, the student evaluations and level of performance went up. The quantitative evaluations are now exclusively positive and “strongly agree” the most frequent response to all items. Having gone from three to one formal class each week has raised the sobering possibility that zero might be even better. I expect for some, perhaps even the majority, that that might be the case.

However, I quite enjoy the weekly meeting, and rather than face that possibility my current scholarly effort is focused on creating a metric for social science and humanities courses that, like the concept test in physics, can be used to measure changes in the level of cognitive complexity and critical thinking that takes place over the term. My subjective evaluation alone is not sufficient.

We need to recognize that the art and science of teaching and learning in the 21st Century is now different. Our challenge is how to systematically go about using the new technologies to enhance teaching and learning without making declarative statements bases on our beliefs, as if they were something more than just that. We have the capacity to reflectively apply the science and critical thinking we teach our students to what we ourselves are actually doing. Our own teaching is the ideal place to demonstrate the power of scientific inquiry and critical thinking that we claim to be our non-replaceable purpose as teachers.

Edward Renner teaches in the Honors College at the University of South Florida

Occupy Learning

Occupying Knowledge and Learning

Edward Renner
The communication technologies of the 21st Century have threatened both the time-honored ways of delivering education and its social and cultural purposes.

The debate over delivery is whether the digital technologies and online applications are actually a means for enlightenment. Many do not embrace the new technology because they believe them to be a “shoddy imitation of the class room experience.” Or, that it is the millennial mind that needs to be fixed, certainly not their teachers.
The debate over purpose is whether online is primarily a financial tool to create new revenue streams by video recording lectures to reach distance and nontraditional students, or an opportunity to systemically restructure the substance and nature of higher education.
The Educational Divide
These internal debates over delivery and purpose have created an educational divide that rests on false either/or distinctions between live classes and online material, rather than the complementary aspects of how to most effectively use the technologies for teaching and learning. These debates go to the core of how, not whether, the roles, functions and responsibilities of higher education have changed as a result of the digital revolution.
The new communication technologies are neither a second-rate educational experience nor a cheaper commodity. To view them as such is to diminish their value. Failure to embrace and use their potential is to cling to the dead idea of a 500 year old concept that lectures and books are still the primary currency for teaching and learning. Both have been replaced by the new communication technologies, binding delivery and purpose together into a new 21st Century entity.
The divide is paralyzing change, while higher education is failing to come to terms with unsustainable increases in tuition and the need for wider and more successful access.
A Void Waiting to Implode
The public debate over financing higher education grows more urgent every day. When the student loan bubble bursts – as it surely will – higher education will be required to reposition itself, if it has not heeded the warning and done so proactively.

On the nontraditional side there is no such confusion. The commercial on-line and for-profits have both a marketing advantage and a clear strategy. Their products are practical, job centered, and non-critical. They are not part of the liberal elite, and their negotiable content better serves the preservation of traditional social values.  Education as a commodity fits the bill of state legislators who are feeling the political heat of rising costs of public education as austerity measures causes state revenue to be replaced by tuition increases.
The competition for educational dollars will only continue to grow in the face of continued financial constraints. This transfer of title to greater standardization, less physical structures and lower cost is well underway.
However, contrary to its manifest appearance, the real story is not about the healthy democratic process of government oversight finding a balance between consumer protection and corporate profits. It is about the role and function of teaching and learning in the 21st Century as the instrument of human progress. This is no small issue. The ownership of both knowledge and learning has replaced economic growth as the gatekeeper of the human condition.

There are only two ways the merger of delivery and purpose can go: Either there will be a further commodification of knowledge with the for-profits competing with public institutions for the educational marketplace, or traditional institutions of higher education will re-invent themselves to actually serve the dual role of centers of public knowledge and to provide massive open online learning opportunities.

Owning Knowledge and Learning

The Occupy movement provides a conceptual context for the unification of delivery and purpose into a new 21st Century entity in which lectures and books are replaced by the power of the new digital communication technologies. Massive open online courses (MOOCs) can be widely inclusive of the general public and still personally responsive and individually evaluative of registered students.
One of the purposes of my MOOC “Forums for a Future” is to expand the physical, social and temporal boundaries of the class to be able to include the parents, extended family and social network of the students so they may share their educational experience with significant people in their personal life. This simply inflates and enriches the reach of the course at little extra cost. In addition, because the course is online and fully public and self-paced, anyone is free to drop-in, or to fully participate. Finally, any existing public interest group – such as church discussion group -- can create their own section by physically meeting together at a set time or in virtual space. If they wish, they can invite university students to drop in, or even to participate with them; this is something students who have completed my course may do for credit as facilitators, or as paid participant observers for evaluation. In every variation, there are interpersonal interactions among self-selective groups who are able to use the opportunity provided by an open door classroom to tailor their participation to suit their own unique needs as learners.
The new technologies have given us the opportunity to restructure both how (the delivery) and what (the purpose) we teach as the counterforce to education becoming simply another commodity. It is time to close the false distinctions of the educational divide and to occupy both knowledge and learning as the new role and function of public higher education in the 21st Century -- as the essential instrument for the enhancement of the human condition.

Good education is both disruptive and essential for democracy. The content and who controls it does matter.


Edward Renner teaches in the Honors College at the University of South Florida