Synthesising MOOC completion rates

Via a combination of thinking about ‘what makes a successful MOOC?’, and looking for a topic for my final project on the Infographics MOOC, I decided to try to pull together the various statistics floating around online about MOOC completion rates. I’m trying to see if any differences emerge on the basis of platform or the assessment methods used.

My draft graph synthesising everything I’ve found so far can be found here: http://www.katyjordan.com/MOOCproject.html Clicking on any of the data points will pull up a bubble with more information about that course, and a link back to the data source.

(note: the interactive version of the chart uses javascript. It has problems with some versions of Internet Explorer; I’ve found it works more consistently with Firefox. If you are having problems, click the pictures below to view screen grabs, although these may not be as up-to-date as the interactive chart).

This is off to quite an interesting start, but I need help sourcing more data and categorising courses according to their assessments.

Courses which I have completion rates for, but need more information about how they were assessed, include:

• 3.091x Introduction to solid state chemistry at EdX, which ended approximately 2013-01

– Used a combination of MCQs and autograding problems sets

• 6.00x Introduction to Computer Science and Programming at EdX, which ended approximately 2013-01

– Used a combination of MCQs and autograding problems sets

• Artificial Intelligence (Sebastian Thrun), ran in Autumn 2011
• Circuits and Electronics, at MITx in 2012 (first time the course ran)

– Used a combination of MCQs and autograding problems sets

• Circuits and Electronics, at MITx/EdX

– Used a combination of MCQs and autograding problems sets

• Computational Investing, Part 1 at Coursera, ended approx 2013-01

– Used MCQs

• Software as a Service, Coursera, ended approx 2012-06

– Used a combination of MCQs and autograding problems sets

• Gamification, Coursera, ended approx 2012-10

– Used a combination of MCQs and peer graded essays

• Introduction to genetics and evolution, Coursera, ended approx 2012-12

– MCQs

• Introduction to Machine Learning (Andrew Ng), 2011

– MCQs and programming assignments

If you studied on any of these courses, please do post a comment here outlining how it was assessed (just MCQs? Peer graded projects? Or something else?). If you know about any other sources of data about MOOC completion rates (how many registered and how many completed) in addition to the ones already in my chart, please do post a comment too (& a link to the data source), and I’ll add them to the chart. Thanks & looking forward to seeing the picture which emerges!

Fundamentals of Online Education: Dude, where’s my MOOC?

After a party on Saturday night, on Sunday I awoke to find not my car missing, but that a MOOC I’d been taking had apparently disappeared. I’d completed the first week of the Georgia Tech course on ‘Fundamentals of Online Education: Planning and Application‘ (FOE), but at the weekend the course was ‘suspended’ after less than a week of its six-week run.

This was a bit of a surprise. Critical accounts of the course and its demise have been circulating (see for example this post outlining in detail the problems with getting into groups, and a critique of the video material here).

Now, I realise I may be a bit of a dissenting voice, but – I didn’t think it was that bad. Not great, but not bad enough to stop.

I agree with the criticism that the groups were chaos; lesson number 1 was that a Google Spreadsheet can’t cut it when you have 40,000 students. However, none of the other Coursera courses I’ve taken before have attempted to get all students interacting, beyond anonymous peer grading exercises. Although I’d not managed to take part in a discussion before the course was cancelled, I was interested to see where it was going and thought kudos for trying. It’s worth remembering that not only had this course not run before, but MOOCs as a whole are still relatively new and there aren’t established ways of doing many things at this scale. This kind of group discussion does appear to be happening successfully in another MOOC I am currently enrolled on, the Infographics and visualisation MOOC at the Knight Center. However, this is at a cost; the course was limited to 5,000 participants at sign-up, and my impression is that only a fraction of these are still actively participating in the course.

The quality of the video lectures was not great, but apart from not being HD or having a talking head as the lecturer goes through the slides, narrated Powerpoints have been common in the other Coursera MOOCs I’ve used. My impression was that the week 1 lectures were particularly dry in order to just get through a wide range of basic concepts, and that it would probably get better as the course went on; I was prepared to give it the benefit of the doubt, but sadly it doesn’t look like I will find out.

Of course, there is a huge irony in this case as it was a course about developing online courses. I think that there may have been a couple of other factors at play in its demise. It launched on the same day as the University of Edinburgh course E-learning and Digital Cultures (EDCMOOC), which has an overlap in focus, and this invited comparisons from students enrolled on both courses. The EDCMOOC also seeks to get learners actively participating in group discussions, but does so via social media channels. This has been more successful, but has also attracted some criticism from students who would rather not use these channels or find the volume of discussion across several platforms hard to manage – see this post, for example. Maybe finding a way for the platform to better support discussions is something that Coursera needs to work on more generally.

Also, it’s fairly unlikely that in a class of 40,000 everybody will be happy, and there will always be dissenting voices. The biggest disappointment about the course for me was that in the end, after having invested my time in the first week, that the course shut down. I would have much rather seen it regroup and maybe just stick to the videos and multiple choice questions – it was my impression that it was only a beginner-level course anyway. In the discussion forums which remain on the course site, there are many students who would like to see it return and are still keen to learn in spite of the problems. At the time of writing, the course still appears in my list at Coursera, so it remains to be seen whether this is the end.

It also remains to be seen what the extent of the repercussions of a ‘failed course’ will be for Coursera and perceptions of MOOCs more generally. As I’ve got one day left of my Topsy trial, I’ll finish off with a chart of sentiment analysis on Twitter for the keyword ‘coursera’ in the past month, which took a downward dive in the last three days (data collected 2012-02-04, click picture to enlarge):

So in conclusion, I’m not saying that folks should put up with shoddy materials ‘cos it’s free, but rather this example highlights that mainstream MOOCs are still in a kind of ‘beta’ phase – and everyone is learning from it, one way or another, even those who are seasoned e-learning professionals. The next iteration of the course would probably have benefited from the course running to completion, so that more systematic feedback could have been elicited about its whole structure. I really hope that we do see a second run and this experience hasn’t put the instructors (or the other students) off completely.