Teaching and Performance 22

Teaching is hard.  Some people are naturally good at it, and others are naturally quick at developing good teaching skills.  For most of us, though, figuring out how to teach effectively is a real challenge.

As a professor, teaching is a central aspect of what I do for a living.  I’m deeply committed to teaching well.  And today, social and technological developments are changing the context of teaching and learning in significant ways.  So we have an obligation to rethink what we do in light of the new environment.  Yet I find it hard to talk about teaching with others, at least in the university context that I’m in.

So I’m trying something new.  I’ll post some thoughts in the hope of generating a conversation.  I welcome your thoughts and input.

The starting point for me is that teaching and performance are connected, but distinct. Giving a spellbinding lecture is performance.  It may be good teaching or bad teaching.  It has the advantage (most likely) of getting students to show up, pay attention, and perhaps remember something of the experience for a long time.  All of those are correlated with good teaching.  They aren’t themselves teaching. Think of a brilliant actor on stage or a great episode of a TV show.  You wouldn’t call that teaching. It’s performance.  You might learn something by watching, but the quality of the performance doesn’t necessarily indicate the quality of the learning. (And yes, I know that movies and games and other things can be designed to teach — I’m a gamification expert, after all! I’m talking about the normal ones designed primarily to entertain.)

Performance is an input; teaching is an output.  You can measure performance at the source, but you can only measure teaching at the receiver, which in this case is the student. As a consequence, performance is much easier to see than teaching. Any observer can see it. You can even see it by watching a recording.  It exists, at least in part, outside of the course.  Teaching doesn’t.

This distinction seems obvious to me, and yet most people I talk to seem to confuse performance and teaching.  And that’s a problem.  If we reward and encourage performance, we may not actually be encouraging good teaching.  In fact, we may redirect effort away from it, toward things that don’t necessarily advance learning.  This isn’t a new problem, or an unfamiliar one.  Basically every text on teaching in higher education published in recent decades criticizes the lecture and the content-delivery emphasis it promotes.  But the emphasis on performance is a particular problem in this age of MOOCs.

MOOCs make performance even more visible, while hiding teaching further.  Now the performances are available to anyone in the world through online video, and directly viewed by tens of thousands of eyeballs in the course.  It’s easy to compare good performers and bad performers side-by-side.  And the pre-recorded lecture structure (with the dominant xMOOC approach) removes the interactivity that’s present even in most in-person “lecture” courses, as well as the distraction of sitting in a room with other students.  The MOOC experience is just you the student, and the instructor on stage.  And students can vote with their clicks to a greater extent than in a university setting, rewarding the brilliant performers with larger numbers.

None of this is necessarily a bad thing.  In many ways, it’s a very good thing.  To be clear, I’m a proponent of MOOCs.  I think they can provide tremendous potential benefits for education, in many contexts. And I’ve taught a large MOOC, twice. In many ways that makes it easier for me to see the dangers they also create.  We need to view MOOCs as opportunities for experimentation, not magical solutions.  And among other things, we need to push back on our natural tendency to view their pedagogical effects purely from the standpoint of performance.

So here’s my personal problem.  I feel pretty confident that I’m good at performance.  I do a great deal of public speaking. I get plenty of positive feedback on my ability to convey concepts in an engaging way.  And my Gamification MOOC attracted a high number of students, very strong engagement/completion numbers, and great anecdotal responses.  All well and good.  In fact, those are things I’m proud of.  Yet they tell me little or nothing about how to teach effectively.  And the support resources I’ve found for improving teaching are really geared more toward improving performance.  I suspect that most university professors, having gone through PhD programs, are less comfortable with performance than teaching.  I’m the opposite.

Most of my teaching in my “day job” as a professor is in relatively small (20-70) classes, on topics that don’t emphasize mastery of formal concepts.  The students need to interact, and they need to apply knowledge.  I can’t get away with just performing.

So here’s the question I keep coming back to: how do I judge success?  How can I figure out what’s good teaching, not just good performance?  Because until I can answer that question, I find it immensely difficult to feel any sense of satisfaction in what I’m doing.

Good teaching means that students are learning.  But that’s just rephrasing the question.  How do I know if students are learning?  I can see how it might be possible to answer that question in subjects where there’s a well-developed, structured set of concepts that students need to master, generations of students have mastered, and instructors have seen students master time and again.  I don’t teach anything like that.  Any bar I set seems arbitrary: I’m just making up a target for students to hit.  It doesn’t tell me if I’m doing well or poorly as a teacher.

I think if I can develop a better answer to this question, it will make me more effective as both a classroom teacher and a MOOC instructor.  And it will help me understand where I can use the MOOC to enhance my in-person course on the same subject.  This is the benefit and the danger of the “flipped classroom” notion: it separates teaching and performance to a large extent.  If the students watch the performance on video at home the night before, what they experience the next day in the classroom had better be teaching.  That’s a scary thought.

Am I looking at this the right way?  For those of you who teach, how do you decide what constitutes success in a course? What resources were helpful to you in answering that question?  

22 thoughts on “Teaching and Performance

  1. Brian Harris Aug 2,2013 4:40 pm

    I’m not an academic and my response is informed by teaching high school for two years right out of college with the Peace Corps, so this might not be entirely germane to your question.

    Halfway through my second year a student came to me from the first year. He’d gone on to the capital city to continue his schooling. He sought me out on vacation to thank me for being such a demanding teacher, and that he felt very prepared and comfortable in the big city school. At that point I felt like I may have had some success in teaching.

    When you wrote “Performance is an input; teaching is an output” that struck me as being not quite the right formulation. One reason for this is that labeling it an output posits that it’s quantifiable, and I don’t think good teaching is very quantifiable. Learning results from an alchemy between good teaching and what the student brings to it. I think any good teacher is looking for enthusiasm, curiosity and engagement from their students. When a teacher sparks that in a student, that seems to be the highest success.

    • kwerb Aug 3,2013 8:48 am

      I agree, it’s heartening when I hear from students that they found my courses valuable. The trouble is: (1) I don’t know how the larger number of students who I don’t hear from feel, and (2) I don’t know whether that means they enjoyed the course or truly learned. A few students saying I’m great is terrific way to convince my colleagues I’m a good teacher, because they generalize from the tiny data set. It just doesn’t convince me.

      You raise a good question about quantifiability. I’m not sure saying that teaching is an output — i.e. it’s something observable in the students, not the teacher — necessarily means that. One could observe those effects in digital form, assigning them quantifiable numbers, or one could measure it in analog form, and generally say the student has made significant learning gains.

      You’re dead on that the “output” of teaching is a synthesis of what comes from the teacher and what comes from the student. So things like curiosity, pre-existing skills, as well as race, gender, socio-economic status, etc. have an impact. I don’t think that changes anything I said. As a teacher, there’s one variable you control (you) and another you don’t (students), at least once you’re in a particular classroom. There’s better teaching and worse teaching possible in any context.

  2. Cynthia Clark Aug 2,2013 5:02 pm

    I have never read or heard lectures referred to in this way, it is very interesting. I did take your gamification course and I agree you are a good performer!

    I used to teach high school mathematics, physics, and chemistry. As a doc student, I now teach educational technology and science methods. I consider my class a success when students can apply what we have covered to their actual lives. For example, when a student explains to his seat mate the motion his falling pen is exhibiting using correct physics concepts and lingo, without realizing I am standing right behind listening. Or when they intuitively figure out what is coming up next.

    For my educational technology class, it is when a student uses technology in a new way to enhance effective learning in her subject area. This tells me the student understands what makes technology effective as a teaching tool. Basically it is when students apply the concepts discussed in new situations, situations never covered in class.

    It is not an easy road, but I do think teaching plays an important part. Creating an environment where students actually care enough about the content to want to reflect on the ideas outside class to me is part of teaching. Helping them to develop faith in themselves, in their ability to access and assess information, and to realize that failure is just information (seems to be the theme on Twitter today, I take no credit) are probably my most important jobs.

    Given this, I would say you are a teacher, even within an xMOOC context. You showed enthusiasm which motivated me to investigate the subject further. You also provided pros and cons and materials which guided me to dig further in order to judge for myself. I would have preferred to receive feedback from you on the assignments; to know if I had failed and if so, have you provide good information! But given the numbers I understand that was impossible (which is my main issue with MOOCs).

    I look forward to reading other concepts about teaching, especially as it is my role to teach teachers!

    • kwerb Aug 3,2013 9:08 am

      Thanks for your thoughts, and your nice words about my MOOC. Your comments seem to point toward a more inquiry-oriented method, especially when it comes to assessments. Having students do a test doesn’t really show their ability to apply the concepts in new ways.

      My struggle is that the MOOC felt far easier to teach than my small in-person courses. (I’ve never had to teach a class for more than 75 students.) It took more time, especially up front, because I had to learn new ways of doing things and develop all the materials, but it didn’t seem hard to engage students. Perhaps that was because the non-engaged ones just invisibly tuned out or dropped out. If so, it’s OK — it’s a non-credit MOOC, and many students clearly did get something from the experience.

      In two years of actively experimenting with my in-person courses, my efforts to empower students for real learning have consistently failed, and the results have been getting worse. The students like the course less, and their performance in the project-type activities I give them is poorer. That’s what I’m struggling with. It’s why I’ve circled back to asking what my goal really is, and how I might measure it.

      • Cynthia Clark Aug 3,2013 10:23 am

        Your last sentence caught my eye “I’ve circled back to asking what my goal really is”. Perhaps the question should be “what is the goal of a student taking the course”. Why do they need your course? Is it part of the curriculum for a particular degree? Will it provide them with the skills to pursue future jobs, careers, interests? Perhaps starting the class with the question “what do you hope to get out of the class?”, or “why did you enroll in this course?” could help focus the content a little more. Give the students some say in their education.

        However, if it is a mandated part of a curriculum path then perhaps the curriculum needs to be re-evaluated. Universities are very slow to change, more so than K-12 schools, and the curriculum may be out of date. Obviously take this with a grain of salt as I have no idea what a business degree comprises!

        • kwerb Aug 3,2013 10:36 am

          That’s actually the subject of my next post, which I’m working on. But the short version is a combination of “no clue” and “a bunch of different reasons.” When I ask students why they took the course, the modal response is “the subject sounded interesting.” I’ve yet to get a good answer that way.

          The good news and bad news for me is that we don’t have a curriculum, per se. I have pretty much total freedom to do whatever I want in the classes I teach. Some are required and some are electives, but in both cases there’s no directive on what that course is supposed to cover or achieve. That’s the challenge: it’s entirely up to me to decide.

  3. Nat A Lee D. (@MoodleMuse) Aug 3,2013 3:42 am

    I have struggled for years to be confident that I am truly assessing ‘this’ learner on ‘this’ learning outcome. My ability as a teacher (as opposed to a performer) and the successfulness of my teaching today, to this learner, are not directly-related questions. So many factors come into play. The simpler the learning outcome the easier it is to confidently assess a learner’s change in knowledge, awareness or behaviour. At a certain point it becomes impossible, or perhaps ‘unscientific’ is a better phrase.

    I believe that the answer is you can’t assess how effective your ‘teaching’ was. Perhaps the question is: should we be assessing how effective our teaching is? The person who is in the best position to answer this question is the learner, and perhaps they can only do this consciously on their death bed, and perhaps they don’t want to, or can’t, tell you. Subconsciously each learner is affected by their ‘teachers’, in both the short and long term, but can not articulate that affect – hence the term subconsciously.

    Let me switch to the perspective of a learner. I came to your class for my own reasons and will learn what I need at the right time. Perhaps you will know this, but more likely you won’t. I may not even know what it is I want to learn. What I ‘learn’ is my concern – not yours. Last week I googled a Professor, Mike Morwood, who I met during a Distance Education Course with brief residential schools I undertook 15 years ago. I have googled him a few times over the years. I was very sad to see that he had recently passed away at great loss to both the academic community and wider community. Mike would not remember me as one of hundreds of learners who sat in a packed lecture theatre. He was not particularly strong on ‘Performance” – Unlike a lecturer from English 101 – a great performer – Charismatic, Intelligent and Arrogant. I had intended to major in Education and English, but very soon realised that the English Department could not teach me anything I wanted to learn. The Archaeology Department, on the other hand, were staff I respected. Humble Intelligent people who taught me how much we don’t know. So I switched majors.

    In hindsight, I realise that my distance education contributed to me finding the courage to leave a religious cult that discouraged Education and instilled a fear of people outside of the religion. My exposure to people, like Professor Mike, subconsciously challenged my values and choices. Sure I passed Archaeology 101, but I learnt a whole lot more than that – and he will never know.

    Now I am a teacher (Despite not majoring in English or Education 😉 Each learner comes to my classroom for their own reasons. A mature aged student once told me that our college gave her the strength to leave an abusive drug-dependant partner. From our college statistics she may not be ’employed’ or even ’employable’ but Learner Analytics miss these types of learning outcomes.

    Are you looking at this the right way? I agree that performance and teaching (better described as Learning) are separate.

    So how do I decide what constitutes success in a course? I find all benchmarks arbitrary and unsatisfactory. As a teacher, I respectfully leave that to the learner to assess, and accept that it may, or may not, be a reflection on me.

    • kwerb Aug 3,2013 9:19 am

      Absolutely true. But there still has to be a way to decide what I did well or poorly in teaching a course. Otherwise I’m left just telling myself I did a great job, which feels phony.

      Admittedly, this is bound up to some degree with external forms of evaluation. In my university teaching context, we don’t have the state-mandated tests that are becoming so overly influential to evaluate teaching in K-12. What we have are student evaluations. Virtually all the teaching reward and recognition systems are keyed to them.

      Clearly, students giving the course a high score is not identical to successful teaching. But if the students don’t score me well, I need something else to feel comfortable telling myself I can discount their perceptions. That’s where I’m coming from in looking for some way to evaluate whether I’m doing my job effectively.

  4. kwerb Aug 3,2013 7:54 am

    OK, I think I finally fixed the WordPress problem that was preventing comments from showing up here. Sorry about that!

  5. Daniel Falcón Aug 3,2013 11:42 am

    Hi Kevin
    For me the KPIs that I use are:
    1. Students improvement and advance at the educational goals I propose.
    2. The interest of students in the class and activities.
    3. My interest. I realize when I have moments I loose interest in teaching to one group is because I´m not getting what I want the students acomplish (sounds a little silly but It´s real)

    • kwerb Aug 3,2013 12:15 pm

      Daniel, how do you come up with those educational goals? Because they can’t just be arbitrary, or we’re back at the same spot.

      • Daniel Falcón Aug 3,2013 10:45 pm

        What I do like the first task for them is to put in a video (1 minute and a half) what they expect to learn in the course and why. They upload the video in a private channel between the first and the second class. Then I check them and I work to develop that skills in the course. After that, I made a control if they feel they get what they want and also a project they develop in all the course to see if they can do what they need.
        I can do that because normally I don´t have more that 30 students and some are group tasks.

  6. Curt Frye Aug 3,2013 12:31 pm

    I’m a technical writer who’s writes for Microsoft Press and also create online courses for a leading company in the field. When you teach outside an established body of knowledge, the only way to measure success is student performance post-class. In other words, how they do in the real world applying what you taught them. Unfortunately, that means you have to wait for the feedback loop to start. I wouldn’t be too concerned about not knowing right away, but stay in touch with your gamified grads to see how your contributions helped them and what you might change.

  7. Anonymous Sep 17,2013 1:51 am

    Hi Kevin!

    Great post. I think the course might serve two purposes: 1. expose people to gamification. 2. Train people in how to gamify. My majority of people taking the course might not want as a career gamification design but want a general notion of it, so that they can identify circumstances where it is useful and hire a gamification design expert to do it. To evaluate them, your current quizzes and exam might be sufficient. For the people taking the course to become gamification design experts, the assignments seem great. You might want to ask gamification firms how they train their designers and compare/incorporate that. Also, taking a random sample of participants’ assignments and comparing their answers to those given by design experts might tell you how much they’ve learned and how much it will transfer when they go work at gamification firms.

    There’s also the others things to look at that could improve interest, retention, etc.

    There are two Coursera courses that might interest you:
    https://www.coursera.org/course/virtualinstruction

    https://www.coursera.org/course/bigdata-edu

    Yours is a wonderful course! Good luck.

  8. Maura Pfeifer Sep 30,2013 4:28 am

    Hello from Japan!
    The topic of this post actually is something that comes up often in the Training and Development field. Have you looked at the Kirkpatrick Model? (http://www.kirkpatrickpartners.com/) It raises the question in my mind of what kinds of behaviors/skills do you want your students to leave your class with? What do you want them to be able to do on the job? A Wharton MBA is quite an investment. What’s the return on investment? It’s been an interesting shift for me as I’ve transitioned from teaching younger learners skills that are largely easily testable to working in a corporate training and development context.

    Frankly, as an adult, in a flipped classroom, what I want is more facilitation of learning rather than traditional teaching. As a trainer, that’s what I provide. My trainees are capable adults, I’m the subject matter expert on English and business communications.

    On a different, but related note, taking your Gamification class last spring allowed me to partner with my husband Brian, who also took the class, and create a gamified element of our corporate EFL class focusing on increasing the amount of time trainees spend using English outside of class by doing a combination of self-study and more fun quests. We’re also co-presenting on the topic at the two largest regional conferences next month in Japan and Korea.

    So if we go back to the Kirkpatrick Model:
    Level 1 – I enjoyed the class.
    Level 2- I got a 95% in the class, so I learned something.
    Level 3- I brought it back and actually implemented a gamified system on the job.
    Level 4 – Well, this one’s really hard to measure, as all our evidence is anecdotal – but the trainees seem to be doing more homework (no control group, and self-reporting on course feedback forms is highly unreliable). It may also lead to higher course satisfaction. But that’s yet to be seen.

    Personally, this has been a fun project, and I know I’d love to have a chance to implement other gamification projects professionally in the future as well. Hmm… anyone stateside want to hire me?

    -Maura

    • kwerb Sep 30,2013 8:33 am

      Thanks for the comments, Maura. I don’t know anything about training and development, but one takeaway from my MOOC teaching experience is that there’s too sharp a line between that world and university-based higher education.

      That being said, I agree that if there is a clear instrumental goal for a course, it’s relatively easy to define success criteria. And indeed, most students took my MOOC as a training course for gamification development. Or at least, I think they did, because they were businesspeople rather than in school.

      In teaching at the university, however, I don’t see my goal as training. Even when I’m teaching the exact same subject, most students aren’t taking as a prerequisite for a job in the gamification field. Whenever I ask myself questions like, “what kinds of behaviors/skills do I want my students to leave my class with,” I’m stumped. I literally stare at a blank screen for a while and then give up. That’s the hurdle I’m trying to overcome.

  9. Pingback: Teaching as Learning | Werblog

Comments are closed.