January 19, 2015

Assessing, Measuring Connected Learning Outcomes

Categories: Connected Learning, Digital Learning
jaimie hoffman quote open courses allow for important connections

Although the preponderance of publicity about Massive Open Online Courses has focused on well-funded enterprises such as Coursera and Udacity, and superstars of what is mostly an online version of a lecture course, such as Sebastian Thrun, those of us who have been excited about the potential of MOOCs for a more student-centric, discursive, networked, peer-driven kind of course look to pioneers such as Jim Groom, Jonathan Worth, Anne Balsamo, Michael Wesch and Gardner Campbell. These pioneers don’t have the mainstream fame that the venture-backed edupreneurs have attracted, but educators in the thick of connected learning, ds106, phonar, and FemTechNet represent the real future of online-enhanced learning.

So, these pioneers, obviously, had to be part of Connected Courses, the free online class we developed and offered last semester for fellow college and university professors to learn to teach their own open courses.

Jaimie Hoffman, who teaches at California State University, Channel Islands, was one of the instructors. She started out as a (very helpful) skeptic about measurable learning outcomes of connected learning. In order to plan our open, connected course, a group of practitioners was assembled for most of a week in June on the campus of UC Irvine. Mimi Ito gave me a dream opportunity to select the all-star team to plan the course.

We also invited a few educators who were interested in connected courses but who had not yet drunk the Kool-Aid. That’s where Dr. Hoffman came into our planning.

During our conversations at the Irvine workshop, Hoffman raised concerns about assessment and measurement of learning outcomes, one of her specialties. Now that she has conducted three separate connected courses, I asked her about her concerns regarding assessment. She replied:

“I realized that creating a connected course does not mean I suddenly lose ‘control’ of the learning experience of my students. I still get to decide what I want students to learn through the course, what strategies or digital tools best support this learning, and I still play the role in the learning (coaching, feedback and assessment) process that makes sense for me. External individuals can drop into the course, formally (by actually participating in the learning activities) or informally (just viewing the course site) but, I can control how and if that happens. For instance, there were some pages with learning activities on them that I made password protected or only editable by people with specific email addresses. Assessing learning in a connected course has turned out not to be all that different than in the face-to-face classroom; in fact, if anything, it has been easier because I have a significant amount of evidence by which to judge performance (e.g. student-produced videos, blog posts, participation in discussion on VoiceThreads, etc.).  This is useful because we know multiple touch points on student work is a better measure of student learning than perhaps, one exam and a few papers.”

We all found it useful during the workshop to have a skeptical practitioner among us, and after her first, tentative queries were welcomed with lively dialogue about the issues she raised, she became an important part of our working process.

Hoffman did what she encourages her students to do — ask critical questions and more: to reflect on what they learn by pursuing these questions. Hoffman later blogged:

 “As I started making sense of the WHAT, I started questioning the WHY…

  • Why would you go to all the effort of connecting your courses? What are the benefits of having students at one University connect with another?
  • Why do this in the first place — how do we KNOW that students value this experience? What evidence exists that shows students benefit from an open connected course versus a traditional face-to-face or online course?
  • Why would we want “drop in” students?
  • Why is open better than BlackBoard?

I couldn’t really answer these questions myself that first day of the workshop (and did not really want these experts and amazing people to know I didn’t know what they were talking about), I went back to the hotel thinking “well… I guess that’s a ‘cool thing’ for others to do.”  I also had a quick conversation that night with my colleague Jaime Hannans from CI who shared some ideas of how she might be able to use the idea of open courses with her nursing students.

Then it happened.  I started seeing the light…. the ideas came…. oohhh… if my classes were open I could connect my students with other students internationally so they can develop international perspectives from people ACTUALLY living in other countries.  Then I thought… hmmm… I could connect my students with students in the K-12 system where they (many are future teachers) can practice using digital tools to teach children and mentor children about college along the way.  I also realized the power this could have for keeping leaders in higher education trained beyond the completion of their degrees; for instance, they can pop into my assessment class to “brush” up on their skills.  So many possibilities!!” 

Fortunately, Mike Wesch was able to help us out a lot on why to start with the “why.”  Hoffman continued to work as a member of our planning team and faculty for what we came to call “#ccourses.”

Hoffman’s own courses produced an abundance of evidence for others who are curious, even skeptical about or reluctant to jump into open, connected learning. Her students’ work speaks for itself:

“UNIV 349 — Diversity in Higher Education (face-to-face course) — blogs that serve as a reflection of the previous class session and capture thoughts of the current week’s reading.

COMM 220 — Group Communication (face-to-face course) –

    • Student blogs that ask students to connect class concepts with the show Survivor.
    • Online module completed with Ritsumeikan Asia Pacific University in Japan.
    • Survivor Country Analysis project that required students to research and collaborate to create multi-media presentation about how groups function in an assigned country. A few groups actually created their own WordPress page to complete this task.

LS 490 — Liberal Studies Capstone (mostly online course)

  • These students have created their own WordPress page to serve as an eportfolio.  They have been working on their sites this semester but they should be updated by 12/12 so I’ll be able to provide you with a sample product of this assignment.
  • Last… I will share a module that WAS PLANNED to happen with the local high school but I had to cancel it because I just felt that it was not going to be as successful as I needed it to be.  This is an example of a sort of risk that did not turn out great, yet worth sharing as I’ll be revisiting it for the future.

Hoffman and I spoke about all this and more in the video below. She gives advice on how to frame and scaffold reflection on the open web, and words of encouragement for those who are contemplating jumping into connected learning.