The Data-Driven Classroom

   by Stephen Smith

Like many of her students, Professor Anna Dollár waits until the last moment to prepare for class. But Dollar's not procrastinating. She's calibrating.

Engineering Professor Anna Dollár teaching at Miami University in Ohio. (Photo: Stephen Smith)

Two hours before class time, Dollár punches up a "learning dashboard" on her computer. She sees a set of screens and graphs detailing how well each of the 30 students in her engineering statics class understood their homework, which was due at midnight last night. Dollár then changes her plan for class; today's discussion will focus on just the material that has the most students struggling.

"I have to have a bag of tricks, if you will, and make the decision in the very last moment which tricks I'm going to use," Dollár says.

Welcome to the data-driven classroom, an approach proponents say promises deeper learning in less time than the standard textbook-and-lecture format. Dollár's class is part of the Open Learning Initiative (OLI) championed by Carnegie Mellon University in Pittsburgh, a school known for its research into the cognitive science of learning.

Anna Dollár is a professor of mechanical and manufacturing engineering at Miami University of Ohio. She co-authored the statics course -- a fundamental engineering class on how objects and forces behave -- with Carnegie Mellon professor Paul Steif.

There are 12 current OLI classes online. Most are introductory science and engineering courses, but there's also logic and French. Much of the courseware is free. A fee is charged for some classes no longer supported by grant funding. OLI has been adopted by faculty at 133 other public universities and community colleges.

An exercise from Anna Dollár's Engineering Statics course. (Credit: Carnegie Mellon)

Dollár's students do much of their coursework online before the class sessions. They get no printed textbook. They read the lessons on their laptops and watch videos of physics demonstrations. Students work through homework problems -- called "learn by doing" -- and call on virtual tutors whenever they want, by clicking a green "hint" button in the corner of the exercise. Software records each step of a student's work and delivers the homework data to the learning dashboard Dollár opened minutes ago.

The process is called adaptive learning. The software adapts the online material to zero in on concepts or problems where individual students need the most help, rather than offering a boilerplate set of lectures and homework to everyone regardless of how they're doing in class. The professor, in turn, adapts the work in the classroom based on data in the learning dashboard.

This just-in-time style of teaching is more time-intensive for the instructor than old-style lecturing, Dollár says. "But on the other hand," she adds, "[it's] so much more rewarding. I don't lose my students after the first ten minutes. Let's face it: after ten minutes of lecture, a lot of students start thinking about something completely different."

New Students, New Tools

The OLI at Carnegie Mellon University (CMU) has the ambitious goal of fundamentally transforming higher education. OLI Director Candace Thille says most college classrooms still work they way they have for generations: students read a textbook, attend lectures or discussions, takes tests and write papers and wait for days or weeks to find out if they're learning what they need to in order to pass. Meanwhile, Thille says, more American freshmen show up on college campuses each year with a wider range of disparities in academic preparation and abilities.

"So we're being asked to serve not only a greater number of students but this greater diversity of students," Thille says. "The tools and methods that we had developed to serve a much smaller, much more homogeneous population don't fit for the challenge before us."

Marsha Lovett (Photo: Stephen Smith)

Because many American colleges and universities face acute financial strains, administrators are hungry for more productive classrooms -- meaning ones filled with students who pass their courses with fewer faculty hours spent on the effort. Potential technological solutions like the OLI are watched eagerly.

The roots of the Open Learning Initiative date back to at least the early 1980s, when researchers at CMU began developing computer-based tutoring systems that helped students learn faster and more effectively than they did on their own. Just after the turn of the 21st century, MIT and other American colleges began experiments that put their courseware online for anyone in the world to use -- for free. That launched an open-access movement that's now moving at a furious clip. As years progressed, the courseware grew increasingly interactive. Course developers added more interactive lessons, hands-on examples and instant quizzes. Meanwhile, CMU professors started using the OLI courses in so-called blended classrooms, where the instruction takes place partly online and partly in the bricks-and-mortar world.

Research on the effectiveness of OLI courses is relatively limited. CMU psychology professor Marsha Lovett and others ran tests showing that blended classes at CMU offered promising efficiencies. "An online course taught in hybrid mode, with an instructor meeting with students a few times -- not as often as in a regular course -- and with students using online material as most of their original learning and practice, the students could learn a full semester's worth of material in half the time with equal or better learning gains," Lovett says.

Professor Anna Dollár is encouraged by what seems to be deeper and faster learning taking place in her statics classroom. Being a scientist, she's reluctant to claim victory until more studies are conducted on the OLI model. Early results suggest there are two key reasons for the accelerated success in OLI classes, Lovett says. Students have to work through the online material to move forward in the hybrid class; in a regular class many students don't bother to read the textbook or are less engaged when they do.

Perhaps more importantly, the online lessons and quizzes give immediate feedback and instant help for students who might otherwise flounder for days with a concept before their next contact with a teacher. With the software always watching, it's also harder for struggling students to simply slump into their chairs and fail.

Practice and Confidence

On the couch in his off-campus apartment, Onye Mora leans in to his laptop screen to study an equation he's supposed to solve. Mora is in Anna Dollár's statics class. He's reviewing a section of the course on friction by drilling through the interactive exercises. "If I get a lot of answers right I might skip to the next topic and work through that," Mora explains. "I scroll down until I find something that challenges me."

Miami University student Onye Mora does online engineering homework in his apartment. (Photo: Stephen Smith)

Mora is a junior engineering student at Miami University, a school of 15,000 students in the small town Ohio town of Oxford, about an hour's drive north of Cincinnati.

If Mora gets stuck on a problem, the virtual tutor provides a series of hints to help him discover the key concepts he needs to solve the question. The built-in quizzes tally how well he's mastering the material. "This is one of the classes where I study less going into exams but perform better," Mora says. "Even if you don't realize it, you are seeing the same concepts multiple times before you enter the testing. It's gaining confidence over time in the materials, and just the fact that you are getting so much practice."

Practice and confidence are keys to successful learning, says Marsha Lovett. The OLI courseware is built to discourage students from plowing through the material at the last minute. "Cramming is not very good for long-term retention," Lovett says. "If you spread [the material] in smallish chunks, regularly spaced, you'll have longer retention than if you cram it into one little time just before the test."

The latest academic research into how people teach and learn helps the OLI team build and refine what they call the "scaffolding" of their online courseware. Ken Koedinger teaches human-computer interaction at CMU and is one of the guiding forces behind the OLI. He says cognitive scientists like to explore the difference between how experts in a given field and novices approach new information and solve problems. Experts, he says, can often just see an answer, while the rest of us grope in the weeds.

"What's important and surprising about expertise development is how our brain soaks up patterns without our even realizing it," Koedinger says. "That's how people learn a first language. It's not like people tell us how to speak English. Babies listen to language and their brains pick up on those patterns."

Deep expertise in a field is obviously a critical asset that colleges look for in their professors. Koedinger says it can also be an obstacle to teaching. He says experts often have a blind spot that blocks them from perceiving a problem from a student's point of view. "If you're a chess expert, the chessboard looks different" than it does to a beginner, he says. Experts grasp patterns and relationships by second nature. In essence, the expert can't understand what the student can't understand.

Ken Koedinger (Photo: Stephen Smith)

In designing the OLI courseware, Koedinger and the Carnegie Mellon team work with experts to overcome the "blind spots" and break each piece of knowledge into its building blocks. They deconstruct the patterns so students can rehearse putting the pieces back together. And because the OLI software captures every click of every student's mouse, massive pools of "clickstream" data help the OLI team tease out what kinds of lessons and exercises lead students to master their subjects most effectively.

"One of the big surprises in this data set for me is when you look at the learning process from task to task, it takes a long time for kids to get better at specific content, whether it's math or science or language," Koedinger says. "The progress we see is steady, but it's slow. That's one of those things about learning we often forget -- how much repetition and practice is critical to becoming an expert."

Learning Dashboard

It's 2:15 on a Thursday afternoon -- showtime for Professor Dollár and the statics class. There were four "learning objectives" in today's online module, and 76 percent of the class passed the quiz. The dashboard showed which questions and exercises the class grappled with the most. Some of the students sent Dollár private "feedback messages" through the courseware asking for help. She typically sends about six emails before each class to assist students who might otherwise use up valuable class time on problems most of the others understand. Students can also see her for help during office hours.

Miami University Professor Anna Dollár prepares for class by reviewing the learning dashboard. (Photo: Stephen Smith)

There are a couple of concepts in today's module on friction that a majority of the class struggled to grasp. So Dollár has tailored some PowerPoint slides and some lecture notes to zero in on the trickiest points. "The learning dashboard gave me a very clear picture of what I need to emphasize," Dollár says.

In class, Dollár projects selected problems on a whiteboard, sketching equations in colorful markers. She prods and teases her students with affectionate, dry humor. Freshman Danielle Oliver says this was the first college course she's ever taken where the work is online first and in the classroom second. "At first it was hard, but then it made sense," Oliver says.

Onye Mora was initially skeptical. He'd taken an online class at a different university and hated it. It was composed just of online lectures and static material, not a carefully constructed course with exercises, tutors, quizzes, and the classroom component. "This is refreshing," Mora says. "It's a different and better way."

Sophomore Jason Ina is ambivalent about the OLI approach. He finds the class sessions too repetitive of things he's already mastered in advance online. "We have to show up to class because attendance is taken, but you could easily get by doing the homework assignments and turning them in. The lectures in class aren't that much different than what's talked about online," Ina says.

For Ina, part of the problem may be that Ina learned much of the material in high school, especially the physics. Review is usually boring, he concedes. But Ina says that the OLI system is more effective than standard textbooks for people who are completely new to the subject.

"This online software kind of substitutes for a textbook," Ina says. "It kind of helps you manage your time throughout the course instead of cramming at the end. Which a majority of students do."

Students and Steel

In an academic article on how OLI can help transform higher education, Candace Thille compares producing college graduates to manufacturing rolls of steel. The analogy makes some sense, given that Carnegie Mellon is in Pittsburgh—a former steel capital—and takes one of its names from the steel tycoon Andrew Carnegie. "Higher education can no longer run away from the P word: productivity," Thille and her co-author write. Faculty sometimes flinch at words like productivity and efficiency, Thille says, because increased productivity can come at the cost of lower quality. And, she adds, "They think we want to replace professors with computers. Not true."

Students in Dollár's class vote on a question by holding up colored cards. (Photo: Stephen Smith)

What OLI designers want to replace is guesswork. Thille says most instructors in a typical classroom use intuition—not data—to determine how well students are really learning. Thille points out that modern steel manufacturing uses a relentless stream of fine-grained data to improve the product. That's what the learning dashboard does for a college professor, she says. And computers are great at handling dull, repetitive tasks like grading quizzes.

Thille says OLI courses place an even higher premium on faculty expertise and creativity than the textbook-and-lecture system. As computer-driven courses expand across American campuses in coming years, there may be less demand for graduate students to grade assignments or run discussion groups. And each full-time faculty member may be able to instruct more students in the same number of hours. Such efficiencies may help contain the spiraling costs of higher education. "Manufacturers use data at every nanosecond to adjust their processes," Thille says. "That's the piece that higher education can learn from."

And, presumably, avoid a long slow fade into oblivion, like American steel.

American Public Media © |   Terms and Conditions   |   Privacy Policy


Support American RadioWorks with your purchases