MOOCs, my PhD, and a couple of small requests

So much has happened since my last post, where to start! The past year has flown by. For most of 2014 I was on maternity leave, but I did take a couple of MOOCs, completing my first edX courses. I also started the MOOC research literature browser to help out anyone writing a thesis or paper on MOOCs – many thanks to everyone who has got in touch to recommend additional papers.

Recently, I’ve started data collection for the main part of my PhD (which is not about MOOCs, but academic social networking online – if you’re interested, do check out my recent paper in First Monday). This is going to keep me very busy for the foreseeable future – and I have two small favours to ask:

  • Contribute to the MOOC literature browser: I’m not going to be able to keep up with new MOOC-related publications as quickly, so I’ve set up a Google form to allow you to add information about new papers. Contributions won’t be ‘live’ immediately but by submitting all the information this way, I can just copy everything straight into the live database in batches once a week.
  • Take part in my PhD study: I’m currently running an online survey about academics’ use of social networking sites and social media, which you would be welcome to take part in! It is open to academics from any disciplines and job positions, including PhD students. There is also the option to participate in further activities using network visualisations. The survey will be open until the end of January and can be found here: https://openuniversity.onlinesurveys.ac.uk/sns

Many thanks if you can help out on either count :-) Finally, in case you were curious (I was!), this is what you get if you put all of the abstracts from the 256 items currently in the MOOC research browser into Wordle:

moocliteraturewordle

No prizes for guessing the big ones! Thanks again and have a great festive season :-)

Leave a comment

Filed under Uncategorized

So long and thanks for all the MOOCs!

This is likely to be my last post for a while as I’m embarking upon a new project – parenthood! (There isn’t a MOOC for that!).

My partner Tom and I are expecting the imminent arrival of a baby girl. So I (probably) won’t be blogging actively for a little while, but I wanted to write a post to say a big thank-you to everyone who has read and contributed to my blog. It started out as a place for me to jot down things which I found interesting about MOOCs as I got involved with them as a learner, and as it happened, lots of other people were interested too – particularly when I started (just out of curiosity) looking at completion rates. Many thanks to everyone who has contributed data to my chart, and all the folks who took this as a starting point for interesting debates about openness of data, what completion means in the context of MOOCs, and bringing an evidence-based perspective to the MOOC hype. I’d never intended the chart to be some kind of MOOC league table, but rather I view it as a starting point for engaging with questions about how completion rates could be enhanced in order to make access and achievement more equitable. I might be a bit slower to add things to the chart, but please do continue to comment and contribute data :-)

The blog has also made a contribution to the academic body of work related to MOOCs; I’ve got a journal paper based on trends in MOOC enrolment and completion currently in press and due for publication in the next issue at the International Review of Research in Open and Distance Learning (IRRODL), and have recently finished my part of a MOOC Research Initiative project grant I hold with my supervisor entitled “Characteristics and completion rates of distributed and centralised MOOCs”. Martin recently presented our work-in-progress at the MRI conference in Texas (I would have loved to have gone, but was too far along with the pregnancy to be allowed on a plane!); he’s been blogging about the work he presented from the project here, and has set up a blog for the learning design part of the project here. I’ll post links here to more project outputs as they happen further down the line.

Bye for now – many thanks once again and have a great festive season – and keep on MOOCing :-)

Edit: 25th February 2014

My paper in IRRODL is out now! Jordan, K. (2014) Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distance Learning, 15(1), 133-160.

Edit: 6th April 2014

I’ve written up my work on networks of co-studied subjects as a short paper, which has been accepted for publication in the International Journal of Emerging Technologies in Learning (IJET). It’s due to be published in August, and will post a link here when it comes out.

I’ve also started putting together a MOOC research literature browser, using the same tool (Exhibit) as I used for the completion visulisation. It’s a work in progress, and am still populating the citations and tags etc. The search box will search across all the abstracts (when I finish adding them ;-) which is quite handy too. Again, works best in Firefox and Safari. If you’d like to suggest any papers to add which aren’t currently included, please do post a comment here with the info and I will add them :-)

5 Comments

Filed under Uncategorized

Adventures in vicarious learning with Online Games: Literature, New Media, and Narrative

I’ve recently finished a MOOC on ‘Online Games: Literature, New Media, and Narrative’, which ran from 9th September to 21st October from Vanderbilt University (via Coursera). This one had found its way into my list of courses not for particular career relevance or utility, but it had just piqued my interested because I am a bit of a fan of World of Warcraft (which, ironically, has been rather neglected since I started studying MOOCs in my spare time instead!).

Edward Burne-Jones - The Briar Wood

Edward Burne-Jones – The Briar Wood

Anyway, it had been on my course list for ages and coincided with me being rather busy, but I thought I’d give the first week a go and see how it went. And I was hooked! The MMORPG in focus was not World of Warcraft, but Lord of the Rings Online. The course was quite diverse; using Tolkiens’ first Lord of the Rings book as a starting point and anchor throughout, the course explored game formats, media studies, pre-Raphaelite art, and Romance literature. It was fascinating, and I have no idea how I would be able to study such a unique topic if it wasn’t for this MOOC. I think, out of the broad range of topics included in the course, at its core the course was primarily an English Literature course. This is not something that I would necessarily go out of my way to study, but in combination with the other topics, was compelling. It was a powerful reminder of the value of MOOCs of being able to experience a wide range of subjects with no negative consequences if you discover that actually you aren’t so fond of a subject, and might discover you really enjoy something unexpected.

The fellowship of the MOOC

The fellowship of the MOOC

As well as the appeal of the topics covered by the course, there were two other things which made this course a winner for me. The instructor, Professor Jay Clayton, clearly loves the subject and his enthusiasm was infectious (even mediated by the internet!). The course contents were still delivered by video, but mainly used an approach which I’d not seen before in a MOOC. Rather than being lecture-like, most of the videos were based on more of a tutorial type format (see thumbnail), where Professor Clayton was joined by several graduate students in a discussion about the topic in focus. This format really worked for me; I found it to be much more engaging than just a lecture. The students brought their own examples which enhanced the discussion, and seeing the turns in discussion which led to a shared understanding was much more useful than simply being given one opinion. (At least that’s what it felt like to watch – it may have been completely staged for all I know!) Although it might not be to everyones’ taste, I found this kind of vicarious learning very valuable and it is something which I’d like to see more of. I’d be interested to hear about any other examples of approaches in MOOCs which have used video in innovative ways which do not simply replicate the lecture model – if you’ve experienced any other innovative approaches please do leave a comment :-)

1 Comment

Filed under Uncategorized

Visualising the network of co-enrolled course subjects

MOOC_networkHave you ever wondered which MOOC courses students study together? I’ve been thinking recently about whether MOOC students choose to enrol in combinations of courses from the same subject area, or are more interdisciplinary in their studies.

To look into this, I looked at the combinations of courses listed on public Coursera profile pages. I collected the information in a spreadsheet and (using Gephi) made a network graph – where nodes represent courses, and a link is present if a student has enrolled on both courses (by doing this, the data is anonymised and doesn’t contain any information about students or links to their profiles). The links are weighted so that the more students who took these two courses together, the thicker the link is. I used the modularity algorithm in Gephi to try to detect communities within the network, and colour-coded the nodes to reflect this.

Thanks to a handy plugin for Gephi from Oxford Internet Institute, I’ve been able to create an interactive, online version of the graph to share, which you can view here (opens in a new window).

2 Comments

Filed under Uncategorized

The challenge of meaningfully assessing discussion participation

Since my last MOOC course update (E-learning & Digital Cultures), I’ve completed four more courses, including: Introduction to Communication Science (University of Amsterdam); Social and Economic Networks (Stanford, Coursera); Infographics and Visualization (Knight Center for Journalism in the Americas); and ‘Art and Inquiry: Museum Teaching Strategies For Your Classroom’ (New York Museum of Modern Art, Coursera). I chose to study some because of relevance to my work, and others just out of interest and curiosity. All were enjoyable and interesting courses overall. I’m not going to go into detail about the first two courses as they both followed the now fairly standard model of video lectures and multiple choice questions, but going to focus on the second two as these are the first MOOCs I’ve taken which require forum participation in the way the courses are assessed.

The Infographics and Visualization MOOC started in mid January and lasted for six weeks. It was my first non-Coursera MOOC (the Communication Science one was also independent of a MOOC platform, but this started later), and used Moodle as its virtual learning environment. The following criteria needed to be met to gain a certificate (in addition to paying $30 – it was also the first and only certificate I have shelled out for):

  • Week 1: Participated in the discussion by posting at least twice in the following forum: Critique a graphic forums. Submitted Quiz 1 with at least an 80% correct.
  • Week 2: Participated in the discussion by posting at least twice: Critique a graphic forums. Submitted Quiz 2 with at least an 80% correct.
  • (Week 3: Participation in forums was not mandatory this week.)
  • Week 4: Participated in the discussion by giving feedback to at least three of your classmates’ projects. Submit your project link about tenure (assigned in week 3) in the designated forums.
  • Week 5: Participated in the discussion by giving feedback to at least three of your classmates’ final projects. Submit your project link about personal taxes (assigned in week 4) in the designated forums.
  • Week 6: Participated in the discussion by giving feedback to at least three of your classmates’ final projects.

After the first couple of weeks, students were required to post links to projects they had undertaken according to various briefs, however these weren’t formally peer graded – as long as you posted something, and posted comments about the work of at least three others, this met the assessment criteria. It became quite difficult quite quickly to keep up with the discussion forums, and my inbox was deluged with forum post notifications. I soon gave up any hope of being able to keep up, and adopted a strategy of being a bit of a forum ninja, sneaking in and posting the required number of comments and giving up on the possibility of returning to the discussions. I suspect that other students were also using the same approach, as I didn’t think that the comments posted about my work were as useful or considered as peer reviews I’ve received via Coursera generally have been.

The Art & Inquiry course used a similar mix of assessments; to gain a certificate, students were required to achieved over 70% across automated quizes (two quizzes, combined weighting of 25%), forum posts (four topics, combined weighting 25%), and a peer assessed project (50%). It wasn’t made clear exactly how the forum participation was assessed, although the general impression was that as long as you posted the required number of times, you would satisfy that part of the assessment criteria. From the outset, the course set out with an ethos of rewarding active engagement; “your success in this course is based on your level of engagement and participation, and not on memorizing facts or grades.” (course site overview text).

I think that this is an admirable goal, but not sure that simply counting whether students have made a required number of posts is really assessing this. Assessing collaboration and interaction with students is still a very open challenge for MOOCs. Is it simply not achievable at this scale, and not a realistic part of ‘MOOC pedagogy’? Or does it require thinking and development of solutions beyond the e-learning tools (such as forums) which have served smaller distance learning courses well, but struggle to scale up? The jury is out for me at the moment – I’d love to hear your thoughts and ideas about alternative ways of assessing this in MOOCs :-)

Something else which the Infographics and MoMA courses have in common is the fact that the classes are not archived – both course sites and materials becoming inaccessible shortly after the courses ‘ended’ (the MoMA course site is still accessible at the moment, but says it will close after a month). This seems to be happening more frequently with Coursera – for example, I was enrolled on a course on Systems Biology, which I’d not been ‘actively’ engaged with, but wanted to dip into the materials at my own pace later in the summer in preparation for a course on Network Analysis in Systems Biology. However, the course site promptly closed at the end, and this hasn’t been possible. This is a bit at odds with the ‘Open’ in MOOC, and not very helpful for students auditing the materials.

Edit 2013-09-21: Recently I’ve read two other blog posts asking important questions about MOOC discussion fora. See also Alastair Creelmans’ post ‘The silent majority – why are MOOC forums counterproductive?’, and ‘MOOC Discussion Forums: barrier to engagement?’ from Phil Hill.

1 Comment

Filed under Uncategorized

A quick look at gender differences in MOOCs

I’ve been having a bit of a Twitter chat today with @dkernohan and @meatyloafy about sources of data for MOOC student demographics, which led to wondering whether the percentages of female and male students in MOOC courses reflects gender differences in traditional university course subjects.

I’d pointed out that gender preferences vary according to course (using the examples of the six Edinburgh MOOCs, in their report here), and David suggested that HESA (the Higher Education Statistics Agency, which collects data about UK HE) could be a useful source for comparison.

So, using the six Edinburgh courses as examples of MOOCs, I had a quick go at mapping them on to their related HESA subject areas, and looking at the percentage of female and male students who had studied these fields in the most recent year (2009/2010, using students at all levels of Higher Education) for which data was available. The data for the MOOCs, and their related HESA subject areas, was as follows:

EdinburghMOOCsHESAGender
Source: MOOCs @ Edinburgh 2013 – Report 1 and HESA Table 7, Qualifications obtained by level, gender and subject. Click on the table to enlarge.

Raw data can be found in this Google Doc. I wasn’t 100% sure about categorising ‘Critical Thinking in Global Challenges’, but categorised this as Medicine on basis that the instructors are based in Biomedical Sciences.

Mooc gender scatterplot

While there is a bit of variation between the two, overall the figures are remarkably similar, so it would suggest that gender differences in MOOC courses do reflect gender differences in traditional courses and subjects (at least, when considering the six courses shown, and comparing to the UK HE sector anyway).

See also: Sergiy Nesterko, Gender Balances: A look at the makeup of HarvardX registrants

Leave a comment

Filed under Uncategorized

xMOOC? cMOOC? EDCMOOC – E-Learning & Digital Cultures

I’ve been meaning to jot down my notes on the E-Learning and Digital Cultures (EDCMOOC) course for some time; last week, the University of Edinburgh published a detailed report on their Spring 2013 Coursera offerings, which has given me a bit of a prod to write this post.

The course began on 28th January, and ran for 5 weeks. In a sense, it had begun before the start date, as the course team had encouraged students to connect through social media channels in preparation. This had passed me by though as I’d not been paying enough attention to the pre-course emails. It seemed like a nice idea, but on the morning of the 28th when the course formally began, I was a little taken aback by a tweet:

Screen Shot 2013-01-28 at 09.58.22

I was a bit troubled by the view that there were already two groups of students, and a perception that the mass of students who had not engaged in the pre-course (including myself) might spoil it for those who were already engaged. It nearly put me off the course right from the start, but I couldn’t resist having a look at the course content which looked too interesting to miss, so I got stuck in.

Somewhere between a cMOOC and an xMOOC

From the word go, this course was a bit different to my previous Coursera experiences, in the following ways:

  • There were no video lectures; rather, the course material included videos from other sources via YouTube, and reading materials.
  • There wasn’t just one professor at the helm, but a team of five academics.
  • All the course materials were made available from the start, not by weekly release of content.
  • No weekly assessments – in fact no quizzes at all. The course was entirely assessed by a single peer-graded assignment, the creation of a ‘digital artefact’ in relation to the course themes. The grading rubric was also fairly open to interpretation.
  • Strongly encouraged to interact with other students and have discussions – be it via the forums or any social media channels.

Initially, this was a bit disorienting – not because I prefer video lectures and linear progression, but simply because that is what I’ve become used to from my experiences with Coursera so far. However, it was easy to adjust and having the extra flexibility through the materials being ‘live’ from the start and not having weekly deadlines made it much easier for me to fit the course around my other time commitments.

There were some limitations with this arrangement though. Some students pointed out that YouTube videos aren’t accessible in all countries, and you can’t download the videos to watch offline like you can with most Coursera video lectures. It was also easy to feel overwhelmed by the volume of social media postings related to the course. Although I didn’t contribute much to the social media discussions, I did find it useful to dip in to the Twitter chat every so often. Across the various communication channels, the course team were quite active in answering questions and contributing to discussions, which was helpful. In this sense, there was much more of a tutor presence despite not having video lectures.

Digital artefacts and assessment

I chose the topic of learning analytics for my digital artefact, as I thought it offered a lot of potential to be set up in terms of the utopia/dystopia course theme. To make my artefact I used Thinglink, a tool for creating interactive images, which I had not used before but had been introduced to through the course. If you’d like to have a look at my digital artefact, it can be found here. From the small proportion of artefacts that I saw (approximately 2,000 artefacts were submitted), the diversity was fascinating; a collection of digital artefacts from the course can be found here.

The digital artefact was a very different assessment model to any of the other MOOCs that I’ve taken to date. It was ‘all or nothing’ – gaining a certificate for the course rested entirely on this one piece of work. However, this pressure was tempered by the fact that as long as you submitted an artefact and carried out peer assessments, you would get a certificate regardless of how your artefact fared in the assessment process.

Given the emphasis that had been placed on participation rather than assessment throughout the course, and that the grading rubric only allowed artefacts to be scored out of 2 (yes, really – 0, 1 or 2 was the range), I was surprised when the course team announced after the peer grading exercise that students with a median score of 1.5 or higher would receive a Statement of Accomplishment with Distinction.

The digital artefact submission process had potential for some security issues, as all you had to submit in Coursera was a hyperlink to your artefacts’ public location online. There is nothing but the honour code stopping students from submitting a hyperlink to anything, whether they made it or not. When the next iteration of the course takes place, there will be an existing body of digital artefacts, hashtagged and easy to find, ‘out there’ already. Two things which could reduce the potential for plagiarism might be: (i) for the platform to record the previously submitted URLs and not allow duplicate submissions; (ii) for students to also submit a short reflective text about their artefact and what they were seeking to achieve (this would also assist in the peer grading process, as some students felt their work had been misinterpreted).

When the certificates were issued, I was also struck by the phrasing of the course description:

EDC_certtext

It was the inclusion of the bit about it being an “introductory undergraduate-level course” that I found interesting. None of my other Coursera certificates include any statements about the level of the material, and I’d thought throughout that it was essentially a ‘taster’ of the Edinburgh E-learning MSc course.

Final thoughts

For me, the course had its ups and downs, but it was definitely a worthwhile experience overall due to the compelling topics and enthusiasm of the course team. Having a team rather than a single academic I suspect was key to the success of this course structure (but possibly made it more expensive to run?). The emphasis of the course was placed on the ‘digital cultures’ rather than the ‘e-learning’ bit; the course could be improved by readdressing this balance as it left me wishing there had been ‘a bit more’. But maybe that’s the idea ;-)

2 Comments

Filed under Uncategorized