MOOC completion rates show a consistent pattern, averaging 12% with a number of factors being correlated, and learning design analysis reveals different patterns of MOOC usage of resources.
Role: Co-principal investigator
Other team members: Martin Weller (Lead investigator)
Funding body: Gates Foundation via the MOOC Research Initiative
Duration: October 2013 to April 2014
Website: https://web.archive.org/web/20150321234130/http://www.moocresearch.com/reports (Internet Archive).
1. What are the completion rates for MOOCs?
2. Have these completion rates varied over time?
3. What factors might affect completion rate?
4. What technologies are favoured by the two types of MOOCs?
5. What pedagogies are used on different types of MOOC?
There are two elements to the project: completion rates analysis and learning design analysis.
Completion rates results:
This analysis is based upon enrolment and completion data collected for a total of 221 MOOCs, including centralised and decentralised courses from a range of MOOC providers.
Completion rates (defined as the percentage of enrolled students who completed the course) vary from 0.7% to 52.1%, with a median value of 12.6%. Note that there are a variety of definitions of ‘course completion’ in use. Some sources prefer to define completion rate in terms of the percentage of ‘active users’ who complete rather than out of the total enrolment. However, there is a statistically significant relationship between total enrolment and active users – active users being approximately 60% of the total enrolment. Given this consistent relationship and the greater availability of data in terms of percentage of total enrolment, this is the measure which has been used for comparison here.
Rates of active use and assessment submission across the course of a MOOC, revealed that Weeks 1 to 2 appear to be critical in achieving student engagement, after which point the proportion of active students and those submitting
assessments levels out, with less than 3% difference between them.
The number of MOOC courses available has increased over time. As a result, there is a statistically significant negative correlation between the number of students enrolled on individual courses and time, while there is a statistically
significant positive correlation between completion and time. So over time, enrolments on MOOCs have fallen while completion rates have increased.
Learning design analysis:
Learning design is an approach to bring design thinking to course production. At the Open University there is a Learning Design project which works with course teams to provide tools that help this process. Two of the main tools used at the Open University are the Module Map and the Activity Planner. The Module Map details the resources required to deliver a course. The Activity Planner requires educators to think about the activities that learners will undertake and categorise them under 6 different categories.
This project took these two tools and applied them to a range of MOOCs. The intention was to see if patterns of learning design emerged from different MOOCs, using these representations. This project also had a separate strand
examining completion rate data of MOOCs, which has been conducted by Katy Jordan. The Learning Designs were created by Martin Weller, Sheila MacNeill & Hannah Gore.
14 Module Map representations were created, using the following process:
A range of MOOCs were selected to cover the major MOOC platforms, and different MOOC types (xMOOCs and cMOOCs).
The researcher registered for MOOCs based on subject and availability over the time period.
The researcher logged all resources used in the MOOC, classifying them under their main use according to the categories: Guidance and support; Content; Demonstration & reflection; Communication.
This data was represented using infogr.am to give two main views: The resource type (number of resources of each type eg video, text, etc) and resource function (number of resources under each of the categories).
The following MOOCs were mapped using this process:
• Udacity Introduction to Psychology
• Udacity Introduction to Statistics
• University of Amsterdam Communication Science
• Coursera Elearning and Digital Cultures
• Coursera Social Network Analysis
• Coursera Neuroethics
• EdX Globalisation Winners and Losers
• FutureLearn Ecosystems
• FutureLearn Fairness and Nature
• JISC/OU Learning Design
• LAK Learning Analytics
• Open2Study User Experience in Web Design
• OU Openness in Education
• University Mary Washington DS106
A number of factors affect completion rates in MOOCs. From the different MOOC providers Open2Study had significantly improved completion rates. This may indicate that short courses are more effective. Completion rates vary significantly according to platform (although sample sizes vary greatly), number of students enrolled (larger courses having lower percentage completion) and assessment type (courses using auto grading only having higher completion rates).
Factors which did not exhibit significant differences include whether courses are centralised or decentralised in nature (note variety in sample sizes) and university scores (which are used as the basis for the Times Higher Education World University Rankings).
The Learning Design mappings provide a useful ‘at a glance’ view of a MOOC, and allow for categorization of MOOCs by approach, similar to that of food labeling. This work emphasizes that there are many different types of MOOCs, fulfilling different functions for different audiences. By having shared representations of these it becomes possible for learners and MOOC designers to ask “what sort of MOOC do I want this to be?”
Weller, M. & Jordan, K. (2013) Characteristics and completion rates of distributed and centralised MOOCs. Presentation at the MOOC Research Initiative conference, University of Texas at Arlington, USA, 5-6 December.