Feeling over-objectified

Plot table from A Taxonomy for Learning, Teaching, and Assessing

I believe wholeheartedly in the importance of designing courses around meaningful learning objectives.

Online educators, though, often seem to be obsessed with enforcing a very behavioristic system of measurable objectives and sub-objectives using Bloom’s taxonomy. (This comes up in the otherwise laudable standards of Quality Matters. They are extremely specific about how to use learning objectives, even though the matter isn’t at all settled or clear in the research they rely on.)

Before Christmas I borrowed A Taxonomy for Learning, Teaching, and Assessing from the library. This book presents the well-known “revised” Bloom’s taxonomy.

What I found was a detailed guide that seems like it would be very useful for institutions or school districts who are trying to calibrate curriculum assessment at a wide scale. I found less there for individual course designers or for students.

In the absence of a strict institutional assessment regime, I remain convinced that straightforward, general, vernacular-English learning objectives are better than Bloom’s ones, for a few reasons.

We can get a false impression of rigor from the six levels

If every objective was calibrated to have identical scope and depth, you might reasonably say that “Design…” was more rigorous than “Clarify…” But what if the design task is fairly mechanical and the clarify task is extremely thorny and subtle?

To some extent, the levels abstractly make sense on a scale of rigor (remembering is lower than applying, which is lower than creating a brand-new structure). Two caveats, though:

  1. That scale has no basis in research on learning or cognition (which isn’t all that settled anyway), and
  2. The levels don’t relate in any sort of sequence, even if it seems like they do (someone can apply something without understanding it, certainly without recalling it directly, and which of those is more difficult is a judgment call in a given situation).

Tinkering with verbs doesn’t make something fundamentally more measurable or meaningful

The Taxonomy book goes on at great length about the subtleties of placing a task at the appropriate level, and I don’t find too much fault with their suggestions—it’s all fairly consistent. But I don’t think that most instructors or instructional designers want to take the time to plot objectives according to the book’s levels, sub-levels, and sub-sub-levels. More importantly, too, what’s the use of assigning that subtle meaning when a layperson (a student, for example) couldn’t possibly perceive the distinctions?

One of the primary tasks in the Applying the Quality Matters Rubric course I took last fall was to judge whether course objectives were measurable or not. My classmates would pounce on the fictitious sample instructor’s objectives:

“‘Understand strategies for overcoming public speaking anxiety’??? This instructor doesn’t know anything about objectives! We can’t even review this course because the objectives are completely opaque and unmeasurable…

“But we can change the verb to describe, and then everything’s fixed.”

Seriously? First, if you know exactly what the measurable version should be, was there really a problem in the first place? That makes it purely semantic and nothing to do with the meaning. Second, to a student or a sane instructor, isn’t the first version fairly clear? No, it’s not 100% clear what understanding entails, but isn’t describe just as vague? Can’t you describe in different ways, at different depths?

It’s all too neat

The way we use objectives suggests that learning is easily planned, sequential, neatly packaged, and identical from student to student. Certainly the objectives provide direction—and an aimless, directionless college course is probably a bad thing. Has anyone ever learned anything in careful order like that, though? And has anyone ever had a great college course that never changed gears or veered into unexpected areas?

I think Bloom’s over-promises and suggests that we understand learning better than we do. Learning is a complex system, differing from person to person, and it includes all sorts of non-cognitive elements (motivation, prior knowledge, and so on).

Saying that a student met a learning objective is a big claim.

  • It requires that your objective, materials, and assessment were perfectly planned and aligned.
  • It requires that the proxy measurement of that assessment was highly correlated to the behavior described in the objective. (Are all assessments so authentic and transferrable to the real world?)
  • It requires that each learning objective is evaluated individually, objectively, without interference from grading scales, curves, and so on. (Does a D mean you met the objective? If you got a penalty for turning in the project late, does that mean you met the objective to a lesser extent? How often do graduate students get Fs?)

What, then?

Let’s create our courses around clear, meaningful learning outcomes. Let’s focus on providing an overabundance of resources and practice for reaching those outcomes—not just a carefully prescribed sequence that hews exactly to our objectives. Let’s share these outcomes with students in a transparent, easy-to-grasp way, without bombarding them with micro-objectives.

If we give students a few clear, strong course outcomes, they’ll begin to understand the connection between the outcomes and the activities in the course. We’ll also be leaving room for the course to take different paths, for students to learn in the messy, serendipitous way that people really learn.

And let’s judge their learning based on those outcomes, but with the understanding that the data will give us no more than hints. And let’s use that data to try to make the next time go better.

Information-thick worlds

Ack Stacks, 2008 (Borges meets Indiana Jones?)

We thrive in information-thick worlds because of our marvelous and everyday capacities to select, edit, single out, structure, highlight, group, pair, merge, harmonize, synthesize, focus, organize, condense, reduce, boil down, choose, categorize, catalogue, classify, list, abstract, scan, look into, idealize, isolate, discriminate, distinguish, screen, pigeonhole, pick over, sort, integrate, blend, inspect, filter, lump, skip, smooth, chunk, average, approximate, cluster, aggregate, outline, summarize, itemize, review, dip into, flip through, browse, glance into, leaf through, skim, refine, enumerate, glean, synopsize, winnow the wheat from the chaff, and separate the sheep from the goats.

—Edward Tufte, Envisioning Information

A beautiful, hopeful riff on information literacy. (To me, a reminder of the importance of liberal arts in any education.)

Book: Creating Significant Learning Experiences

Fink’s book, now out in an expanded second edition

I read L. Dee Fink’s Creating Significant Learning Experiences while I was working with faculty on some first-year humanities courses, and his framework for thinking about course outcomes changed my entire approach.

His course design process isn’t notably different from other models, but his call for broad, beyond-the-content learning outcomes is unique. It’s congruent with what I feel were the strengths of my own liberal arts college experience.

Instead of approaching learning outcomes by looking at the content, Fink wants you to consider a broader picture:

  • Foundational knowledge
  • Application (to real situations)
  • Integration (with other courses, personal life, civic life)
  • Human dimensions (learning about themselves and others, about the profession, about working in groups)
  • Caring (interest in the subject, interest in other areas, values)
  • Learning how to learn (in the subject, in college, in the workplace, in life)

Most course-design models hit those first two elements but leave the other four unexplored (or at least unenumerated).

But can those be measurable course outcomes?

Fink argues, and I agree, that the cognitive domain of Bloom’s taxonomy limits us: “Individuals and organizations… in higher education are expressing a need for important kinds of learning that do not emerge easily from the Bloom taxonomy” he says (p. 34). If we have other outcomes in mind for students for a course, then why are we restricting ourselves? We end up valuing what we can measure instead of trying to measure what we value.

Fink does still want us to assess those last four categories and to be transparent about expectations, but he puts less emphasis on mechanically linking the wording of the outcomes to specific cognitive areas or levels of rigor. For him, depth and enthusiasm during the class matter more than a rigor level written in the syllabus.

To me this is wholly more persuasive than much behaviorist and constructivist micro-design of learning. There’s no doubt, though, that assessing those non-content areas is challenging—especially online—and subjective.

More human feedback and interaction than is the norm for an online course? Good!

Curmudgeonly question: Is “passive” learning always so passive?

Laurentius de Voltolina; Liber ethicorum des Henricus de Alemannia
Sage on the stage, slackers in the back (by Laurentius de Voltolina, from Wikimedia)

Most instructional design makes assumptions about the inferiority of traditional “passive” learning—specifically, the lecture class.

Is it passive learning, though, or is the learning simply off the radar?

Motivated students take notes, download PowerPoints, read, and get together for group study sessions; hardly a passive experience. Activity and engagement are just the reverse of a flipped classroom, at least for a modestly motivated student. Granted, these may not be authentic tasks… unless your goals are for students to learn to read, write, synthesize, and learn better. And to be self-motivated.

Why are we so anti-lecture, then?

First, behaviorism and constructivism don’t like learning activity to be variable and unknown; they demand a prescribed process, or at least careful planning and oversight. Second, bad lectures abound; they’re too quick and easy to prepare. Creating a problem-based learning plan for a course requires a lot of thought and work. (Would lectures with that amount of thought of work still be inferior, though?)

On the other side, all I can think is this: I genuinely dislike going to a workshop or conference session and being asked to “share my thoughts with a neighbor” or “brainstorm one side of the issue with my group,” or even “explore on my own and then share.” As a motivated and (fairly) skilled student, I usually feel more comfortable—and perceive more value in—listening to smart or experienced people talk. I’d wager many motivated students feel the same way.

Whether this is an argument for lectures or against cookie-cutter “active” class activities, I’m not sure. I think there are lessons for online course design, though, especially for those of us who knee-jerkedly dismiss the idea of including lectures in our materials. Do we do ourselves a disservice by dismissing what students want or expect and discounting how they’ve already learned to learn?

Maybe the lecture class is the ultimate form of student-centered learning? We provide the exposition, and they control all the work and the learning. Surely it’s more student-centered and active than web tutorials and quizzes?

Instead of being anti-lecture, maybe we need to become lecture experts and help people to lecture better, to expand their lectures into something new and creative—and to include meaningful, rigorous activities to follow the lectures.

Toward a new instructional design

Railroad tracks
Should this be the end of the line for the old cookie-cutter ways of doing things?

If online education is front and center now, and institutions like ours are moving forward with new enthusiasm, then I think we have reached a good moment to re-examine what online learning can be or should be.

From University of Phoenix to Harvard, institutions of every stripe are offering things online—and with a fair amount of homogeneity in terms of design and student experience. Instructional design has arrived in mainstream higher education, and online classes are no longer a novelty or a peripheral option. I think we now have an obligation to grow and mature as a discipline, to leave no assumption unexamined.

One of my blogging projects for the next year will be to explore the standard practices of instructional design through a critical lens, especially through self-reflection. Here are some of the topics I’m eager to think about:

  • How people learn: How do we break out of the mindset of behaviorism, which gives us the comforting illusion that learning is easily broken down into components and can be laid out in a line? How do we create and assess meaningful learning outcomes while respecting learning as a complex system?
  • Mindfulness: How do we get ourselves and our students out of the habit of thinking things are clear cut, consistent, and knowable? How can we create student experiences that are ill defined, realistic, and meaningful? How do we bring serendipity, nuance, and doubt to an online course?
  • Experimentation and play: Related to mindfulness, how do we make ourselves comfortable with experimenting, with taking risks to find potentially better ways of doing things? How do we keep play, enthusiasm, and passion in our work, and how do we balance that with the need for consistent ways of doing things?
  • Pedagogical traditions: Although few would argue that traditional classroom experiences can’t stand to be improved, shouldn’t we give more deference and consideration to the ways people have learned in various disciplines? Instead of pulling all online courses into a consistent (bland) model, why don’t we do more to improve on each discipline’s pedagogical strengths?
  • Faculty- and student-centered course design: How do we make sure online learning is still human—computer mediated but not computerized? How can we empower instructors to work in this new space with a sense of confidence, ownership, and freedom? And how student-centered can a course be when it was designed and constructed from top to bottom before they even enrolled?
  • Technology serving learning: How do we separate valuable uses of technology from the flashy sales pitches of the many industries that push all these devices, apps, gadgets, and services? How can we use real-life technology instead of spending time and money teaching students to use “education technology” that they won’t use after college? How do we teach digital literacy in our online courses?

I’m hoping to explore these topics with fresh eyes and with healthy skepticism. I hope we can throw out some of our preconceptions and build up a more nuanced, fluid, thoughtful approach to instructional design.