Moodle & MOOCs: Bringing professional legal education mainstream - the MOOC experience at the Law Society of Ireland

Moodle & MOOCs: Bringing professional legal education mainstream - the MOOC experience at the Law Society of Ireland.

Freda Grealy, Steve Collender, John Lunney, Rory O'Boyle [1]

Abstract

Since 2014, the Diploma Centre at the Law Society of Ireland (LSI) has implemented a programme of MOOCs; 'Massive Open Online Courses' in key areas of practice. This paper charts this online learning experience from a number of perspectives. Firstly we explore the objectives behind the Diploma Centre's MOOC programme, and how this public legal education initiative aligns with a broader mission to widen access to justice and to the legal profession. Secondly, framed by a set of practical tips for others who may be developing MOOCs, the authors discuss their instructional design process and the opportunities presented in utilising a bespoke design of the Moodle learning management system. Quantitative and qualitative feedback from MOOC participants is provided which in turn is used to implement best practice for our MOOC course design.

Keywords : massive open online course; public legal education; online learning; learning management system; moodle; instructional design.

1 Introduction

The Diploma Centre at the Law Society of Ireland (LSI) has provided continuing professional legal education for over twenty years, offering a diverse range of postgraduate courses which are available both onsite and online. These continued professional development (CPD) courses are open to solicitors, trainees and barristers, with a number of courses welcoming suitably qualified non-legal professionals. [2] Providing student-centred blended learning options has been key to maximising flexibility for our participants, many of whom are based outside of Dublin. In turn this has boosted the profile of the Diploma Centre as a hub for online learning. [3]

Since 2014, the Diploma Centre has delivered an annual, free online course in a discrete legal topic that is open to all and specifically designed for large numbers to participate in. The annual Massive Open Online Course (MOOC) embraces the LSI's public interest mission to provide access to legal education and the legal profession.As Conole states "the value of MOOCs is to promote social inclusion, coupled with them making traditional institutions reconsider the educational offering they are providing and what is distinctive about their institutions", which captures the dual motivations for the Society's MOOC (Conole, 2016, p.251). To date, the MOOCs have attracted over 10,000 registrations from people in over seventy countries.

The timeline below illustrates our MOOC history so far, beginning with an inaugural course in Aviation Leasing and Finance in 2014 and culminating in our Sports Law offering in May 2018.

Fig. 1: Timeline of LSI MOOCs 2014 - 2018

The primary objectives of the MOOCs at the Law Society of Ireland have been to design and deliver courses that:

· Fit with the LSI's mission to provide public legal education and widen access, i.e. courses that are easily accessible and free for everyone;

· Provide individuals and businesses with a greater understanding of the law;

· Provide educational value and highlight the LSI's commitment to embrace technology and online learning;

· Build a better learning experience by encouraging engagement and activity; and

· Create a positive online experience for students to incentivise uptake of further paid courses run by the Diploma Centre.

The Diploma Centre MOOC courses are delivered online over five weeks and feature contributions from leading lawyers and industry experts.Emphasised within is the value of MOOCs as an outreach tool to widen access to education for the public, enhance the profile of the offering institution, and extend the reach of our brand.

This paper offers a reflection on the experiences and best practices adopted by the LSI over five years in the delivery of MOOCs. This was achieved by responding to recent research which identifies shortfalls in how MOOCs are run (see below), and developing methods to improve MOOC offerings.

The instructional design process is outlined, as is the extensive list of features employed in the MOOC, including how and why these features were utilised. Thereafter, the paper reviews the results of these various design implementations and the impacts on the learning experience. Finally, the paper concludes with our plans for future development.

2 Literature review

In 2008, George Siemens and Stephen Downes co-taught a class thought to be the first to use the term MOOC. The course, called "Connectivism and Connective Knowledge," was presented to 25 tuition-paying students. In the decade since, MOOCs have been adopted by many of the top learning institutes such as Harvard University and Massachusetts Institute of Technology as a means to reach an unprecedented number of students (Zawacki-Richter et al., 2018). MOOCs have increased opportunities by building learning communities on a global scale (Mahraj, 2012) without the prohibitive costs associated with higher education (Ruth, 2012).

A number of different categories of MOOCs have emerged in that period which Siemens (2012) distinguishes as cMOOCs, xMOOCs and quasi- MOOCs. Conole (2016) noted that early examples of MOOCs were defined as cMOOCs and described them as involving "no formal learning pathway or correct set of prescribed tools; each learner adopted their own approach (Conole, 2016, p.239). The xMOOC is the category of MOOC that the LSI series most closely resemble. These are defined, Khalil et al (2015 ) also helpfully list the fundamental elements of xMOOCs and use the observed crucial elements as a means of designing an evaluation grid for xMOOCs. The listed essential factors of curriculum, videos, self-testing units, accompanying material, asynchronous communications, assignments, technical implementation and certificates aligns with Conole's ( 2016) broader definition of the xMOOC as being "more individually focused, didactic MOOCs, where the emphasis is on learning primarily through content and videos, supported by e-assessment elements". [4]

MOOCs are not an independent phenomenon and are heavily linked to developments in the fields of distance learning, with the potential to remove physical barriers to learning and provide equality of opportunity within education (Zawacki-Richter et al., 2018). The adoption of MOOCs in Europe has been gradual (Gaebel, 2012), and although they have been in existence for ten years, they remain a relatively recent phenomenon in Ireland. Hew (2014) states that "MOOCs areincreasingly viewed as a marketing tool—a cost-effective way to recruit students into a fee-paying course from the host institution." Irish universities may not see the potential or have the need to compete for students at the same scale as universities in countries such as the US and the potential reputational advantage advanced by MOOCs is also seen as unnecessary bearing in mind the potential costs and resources associated with developing and implementing a MOOC.

Since their inception, MOOCs have been subject to criticism. In a study of MOOCs organised between January 2010 to June 2013, Chen (2014) identified the key weaknesses of this form of learning, which include high dropout rates, questionable course quality, lack of certification, ineffective assessment strategies, and technical issues preventing access. More generally, addressing the challenges for learning posed by physical distance has been a major barrier for online learning as a whole.

Research has indicated a significant increase in student performance within blended learning courses in the 21stcentury, with a related increase in such students attaining the relevant learning outcomes (Means, Murphy & Bakia, 2013). However, online learning can still be regarded as problematic and the quality of learning offered may be viewed with suspicion (Chen, 2014).

Academic literature on MOOCs has tended to focus on the registration phase for MOOCs (i.e. when students sign up to undertake the course), the active phase (while the course is in session), or the completion phase (when the course is finishing) (Hew, 2014). Successful strategies utilised in traditional online learning such as instructor presence (Das, 2012) and choice of activities (Kelly, 2012), have much to offer MOOCs in terms of best practice guidelines to increase engagement online.

In terms of challenges facing MOOCs, Hew and Cheung (2014) have identified four key issues hindering their success from a student viewpoint: (a) difficulty in evaluating students' work; (b) having a sense of speaking into a vacuum due to the absence of immediate feedback; (c) external demands of time and money upon students; and (d) a lack of student engagement on discussion forums.

More recent research advises educators to 'harness the enormous opportunities that MOOCs might afford for providing access to knowledge and education, or as Conole (2016) describes it to 'democratise learning', whilst equally addressing problematic issues like high dropout rates and sustainable cost models' (Zawacki-Richter et al., 2018, p.7). Research also indicates that the participant's individual learner profile is highly predictive of their MOOC experience, with more successful MOOC learners having highly developed study pattern (Conole, 2016, p.240). This paper sets out to accomplish this and identifies the opportunities and challenges in turn.

3 Instructional design process

3.1 Overview

Below, we summarise key elements of the MOOCs as implemented by the LSI. There is a focus on the strategies adopted in our most recent iteration, namely the 2018 Sports Law MOOC, simply because it incorporates our most recent representation of best practice and technological enhancements; all of which were informed by our previous MOOCs.

The format and structure of LSI MOOC courses is provided in Figure 2 below. This represents the process by which each MOOC is planned and delivered. Following that, we describe in more detail each element of the MOOC, reflecting on what we have learnt with regard to each element over five years of providing MOOCs.

Course

Length

Estimated workload

Certificate

Video recordings

Other resources

Assessment

Notes

Introduction to a discrete legal topic (e.g. employment law, technology law, Sports law )

5 weeks

3 to4 hours minimum per week

Paid digital certificate; optional

Recorded presentations with captions option available

Running time typically 10 - 20 minutes

Videos chunked into shorter sections

Playback options to enable personalised view and alternative speed options

Option to download audio podcast of recording

Embedded within Panopto webcast window

Interactive Features - search, note taking, bookmarking

Transcripts of videos

Lecture slides

Further reading

Course forums

Live question and answer sessions

Twitter tasks

Online surveys

Weekly lesson quizzes with badges awarded

Required to watch all video content

Required to make a minimum of three discussion forum posts

An SQL report determines whether students have successfully completed the course.

Unlimited attempts on quizzes

Course can be completed in a self-paced manner

Videos may be watched on playback at any time

Options for further study upon completion - paid Diploma course


Fig. 2: Structure of LSI MOOC courses as of 2018

3.2 Features of the online learning environment

There are now a multitude of platforms hosting MOOCs and in 2017, 78 million students took 9,400 MOOCs from over 800 Universities worldwide (Peters 2018). [5] With such variety comes inconsistency in quality. Margaryan et al (2015) note a common criticism of MOOCs is that they are 'well packaged; [but] their instructional design quality is low'. They propose that a reason behind this is that MOOCs are not evaluated from an instructional design perspective which would necessitate expert evaluation rather than solely a focus on learner feedback. Feedback from previous MOOCs was incorporated into the design stage when working with e-learning partners on the project. Evaluations were used to assist in evolving and adapting the learning environment to best suit the student and in future iterations we propose to develop adjustable course content dependent on student performance. Comments regarding ease of access to lectures and important course updates were factored into the re-conception of the learning space as a malleable environment, with the most relevant information appearing prominently upon accessing the course. How students gain recognition for their learning is important, with the Society adopting both badges and the issuing of digital certificates of completion.

The proliferation of social network sites has also created an observable effect on the design of e-learning platforms in recent years. VLEs such as Moodle and Blackboard have introduced features similar to Facebook and other social networking sites, such as liking/sharing features, alerts, and instant notifications. Such developments have transformed learning management systems in ways that make them modern, student-centric and more conducive to students' daily web routines (Ronan, 2015).

3.2.1 Configuring the MOOC Hub (Moodle)

From the outset of our MOOC series, we decided to host the MOOCs on our existing learning management system (LMS), Moodle. We recognised that this would provide us with an opportunity to improve our LMS and potentially impact positively on our standard diploma offerings. Additionally, the choice of this delivery option was a means of achieving cost effectiveness in offering a course of this type free of charge to the public.

Student and staff feedback to the initial adaptations of our standard LMS template for our early MOOCs was a motivating factor to anAutumn 2016 project to overhaul the Diploma Centre's LMS. The then iteration of Moodle was updated and rebranded as the 'Diploma Hub', with an array of new features. The goal of the projectwas tore-design the online space for students and transform the LMS from a simple content repository to a more visually appealing interface and facilitate a versatile online learning community.

We aimed to create an intuitive and visually appealing interface requiring as little thought from students as possible to navigate, and accounting for users of all technical abilities.Using a website that doesn't require thought to engage with feels effortless, whereas websites with confusing interfaces tend to drain users of energy and enthusiasm(Krug, 2014), which in turn could lead to high drop-out rates.The appearance of web objects (such as their size, colour,and layout), their given names,and the small amounts of carefully crafted text should 'all work together to create a sense of nearly effortless understanding'(p.19).

3.2.2 Course interface

The course interface for 2018 was represented as a vertical series of coloured tabs: Modules, Discussion Forum, Learning Resources, and Certificate of Completion.

Fig. 3: Course interface with clickable tabs

Modules : This was the default landing page and contained all lecture videos, with the most recent weekly session moved to the top. Play buttons were digitally added to the webcast thumbnails to encourage clicks. Short descriptions were posted beneath each video and all supplementary activities, such as the weekly quiz and further reading list, were posted at the bottom of each module.

Fig. 4: Lecture video thumbnail with digital play button and short description text

Discussion Forum : The Discussion Forum tab featured a red notification bubble indicating the number of unread messages. Clicking on this would lead to the forum 'menu', as each module featured its own forum in addition to a general discussion forum and announcements page. Each weekly forum was also available under the Modules tab, and hyperlinking from multiple places created a webbed approach through which students could access the threads from all relevant areas.

Learning Resources : The Learning Resources tab was assigned for general course material such as the timetable, webcast guide, and weekly transcripts.

Certificate of Completion : The Certificate of Completion tab contained all information regarding the purpose of the certificate and how to purchase it, including a visual overlay of the weekly badges and a link to the purchase portal once the certificate became available.

3.2.3 Flexible Moodle blocks

The right-hand side of the screen featured an array of HTML blocks, which were configured and manoeuvred in correlation to their relevance as the course progressed. A 'latest announcements' block featured all course updates posted by the course team. A Twitter block displayed all tweets using the hashtags #SportsLawMOOC or #SportsLaw, enabling students to follow, in real-time, Twitter commentary concerning the course and encourage interaction. A dedicated 'Friday release block' featured thumbnails linking to each Question & Answer session, which were also posted to the Modules page. Following the release of the first quiz, a 'Your badges' block was displayed prominently at the top right-hand corner to enable students to track their progress through course completion. Towards the latter section of the course, a graphic block was created linking to the upcoming Diploma in Sports Law page, inviting students to continue their studies following the MOOC. There is some evidence for the marketing ability of the MOOC. For example, student numbers on the Diploma in Employment Law fell almost consistently each year between 2010 (91 students) and 2016 (37 students). This pattern was reversed following the 2017 MOOC, with student numbers increasing 32 percent, with a further 6 percent increase for the following offering.

The flexibility of Moodle was maximised; manoeuvring the learning space to best suit the learner, and thinking about the arrangement of onscreen content based on student needs. For example, during the early weeks an introductory thread from the discussion forum was pinned to the top of the main session page, including a graphic handwave icon and an invitation for students to upload a user profile icon. 400 students took the opportunity to say hello and add a user profile icon, an increase of over 66 percent from the previous year.

3.2.4 Aesthetic improvements in line with best practice in instructional design theory

Multimedia learning principles were studied and implemented when re-designing the learning space. For instance, research shows that students learn better from a combination of images and text, than from text alone (Clark & Richard, 2011). Learning elements such as the weekly quiz and learning resources were visually enhanced on the basis that a graphically appealing interface would encourage clicks. All standard Moodle formatting was stripped out and replaced with customised hyperlinks with graphics to open each activity.

Fig 5: Before - during MOOC 2016, materials were presented on Moodle in standard configuration

Fig 6: After - enhanced design with customised graphics and helpful descriptions during MOOC 2017

3.2.5 Video production strategies

The main learning resources for each week's release were the online video recordings which featured interviews, panel-style discussions, and short lectures.

Our approach to the development of these resources is influenced by the work of Guo et al (2014). Their extensive research into how video production affects student engagement was presented in an empirical work that was ground-breaking in the context of its scale. Data was measured from 6.9 million video-watching sessions across four MOOC courses. From a practical perspective, they helpfully interpret their results to present a set of video production recommendations (p.42) which provide a template of best practice for educators that we have been able to utilise as a framework for our video production. As a result, we spent considerable time in pre-planning with our contributors to discuss how to make their presentations most effective in terms of engaging students:

  • Short presentations,with topics appearing on title boards, grouping the content into separate sections;
  • Simultaneous stream of PowerPoint and talking head;
  • Use of interview style segments with topics appearing on title boards to chunk the content into separate sections, and with interview-style responses from the speaker. Such techniques gave the impression that the presenter was speaking directly to a student.
  • Working with a production team to record and edit the presentations; and
  • Record informally with a personal feel, often in the subject matter expert's office.

The 2018 Sports Law MOOC was the first offering to embed the videos within a video capture viewer. In previous MOOCs, the videos were hosted on YouTube. The adaptation of Panopto [6] for video management enabled the incorporation of a series of embedded interactive features, including;

(1) providing captions which could be enabled within the webcast viewer for students to follow along;

(2) a function to allow searches for any word spoken or shown on screen;

(3) navigational controls to move directly to specific sections of the video;

(4) podcast download feature to allow offline review of audio content;

(5) note-taking facility to add synchronised comments, which save to the user's account and are retained when viewing later;

(6) visual bookmarks within the lecture window, wherein a timeline displays each title board and allows students to move to the desired point in the presentation;

(7) play speed controls to increase or decrease playback speed as desired; and

(8) personalised viewing options.

These features further promoted student engagement by enabling students to have an active rather than a passive viewing experience.

3.2.6 Discussion forums

In order to successfully complete the course, students were required to have actively engaged with the discussion forum by posting at least three times to any of the forums over the duration of the course. The posts needed to be meaningful, more than one-sentence contributions, and contain at least fifty words each. An SQL report was configured to determine whether students had successfully met the criteria for completion, including the posting requirement. Integrating the discussion forum requirement with the assessment was an effective means to increase both the volume of conversation online and the quality of comments offered by participating students. As Nagel and Kotzé (2010, p.6) have found, 'when students engage in online activities and take responsibility for the quality of interaction, they can have a superior learning experience'.

The use of discussion forums on online courses has also been proven to correlate with higher student retention rates and higher grades upon completion (Coatzee et al., 2014). Online socialisation has become an important component in the success of the MOOCs, with participants encouraged to interact on the discussion forums with fellow students. A dedicated thread was also available for students to post any technical queries, and a member of the course team was assigned to monitor and respond.

3.2.7 Live 'question &answer' sessions

Lack of instructor accessibility is one of the biggest criticisms of large-scale free online education initiatives (Warren et al., 2014). In respect of that potential criticsm, for the 2017 MOOC, each Friday a lunchtime 'question & answer' (Q&A) session was filmed at the offices of legal experts contributing to the course, whereby students could tune in and engage in discussion with the experts via a live video chat.

These Friday sessions adapted a more 'ad-hoc' format than the televisual pre-recorded presentations released on Tuesdays. Lectures were filmed using the in-house camera equipment and IT resources, captured using free video recording software ('Open Broadcaster System' (OBS)), and streamed via the Diploma Centre's YouTube channel. A synchronistic message board known as 'CoverItLive' functioned as a moderated questions facility for students to connect with the lecturer, and appeared as a text stream connected to the YouTube link (this feature was replaced by the Panopto system in 2018, which features a discussion facility for students within the webcast viewer).

Prior to 2017, a simple discussion forum thread was employed to facilitate the chat, and it was felt that a more dynamic format was required to encourage interaction. Students were encouraged to log in to the lunchtime Q&A sessions and participate in real time, introducing a live element to otherwise pre-recorded content, all the while using free video technology to drive the initiative.

Providing an additional lecture on the Friday also staggered the weekly content release to avoid a seven day gap between session materials. With 'loss of course rhythm' established as one of the two top reasons why students drop out of MOOC courses (Nawrot & Doucet, 2014), it was important to not only keep to a regular release schedule, but also ensure the MOOC was sufficiently spread out to maintain interest over the five weeks.

3.3 Providing opportunities to test knowledge

Online quizzes play an important role in active learning, which is one of the features most strongly associated with successful MOOC initiatives. The value of quizzes within online education lies within their function as a motivator to maintain students' interest.

A study in 2008 by faculty at Georgia Southern University noted a correlation between online quizzing and increased student engagement in class (Marcell, 2008). Students were also found to be more likely to have read the assigned materials where a quiz option was provided. In 2012, a study at Harvard University concluded that online quizzes improve retention of course material, reduce student distraction during online learning sessions, and reduce student assignment anxiety (Szpunar, Khan & Schacter, 2013).

In MOOC courses with thousands of students, a greater number of students view the videos than undertake the assignments required for the assessment (Coffrin et al., 2014). This was an important consideration for the Diploma Centre in delineating an assessment strategy. Providing non-automated assessments, such as assignment questions involving individual assessment and feedback, would require ample resources from staff. The logistics and time commitment involved in building online quizzes, however, are no different for a wide scale course with thousands of participants than a few dozen students. Feedback is provided immediately and without the need for team resource involvement beyond the initial configuration, so the implications for resources are not onerous. Providing instant results for students also creates an immediate feedback loop, allowing students to make adjustments and solidify knowledge learned (Csikszentmihalyi , 1990). As one student noted in the 2017 course evaluation survey, ' the concept of being both taught and assessed straight afterwards... as opposed to being taught for 6 months or a year and then be assessed, is the perfect way to learn.'

As the attrition rate for students on free online courses is high, the quizzes were adopted as a means to keep students active with the course. Each week, videos were released on Tuesdays, with the quiz and a supplemental live lecture or Q&A session following on Fridays. This staggered release strategy encouraged students to return later in the week, rather than consuming all content at once and potentially losing interest or forgetting about the course. Upon successful completion of each quiz, students would receive a badge which displayed prominently in a 'My Badges' block at the top right of their screen. This helped to 'gamify' the course and provide a visual incentive for students to both test their knowledge and stick with the course until completion.

3.4 Providing a certificate of completion

The Diploma Centre adopts a similar cost model to online learning providers such as Alison and Coursera. Essential materials such as the learning content itself are provided for free, while for anything non-essential, students have the option to purchase add-ons if they choose. Porter's study of MOOC economics finds that the majority of MOOCs are based on this 'freemium' model, whereby 'a certain amount of a product is available to all, freely, whilst other parts of the products are charged for' (Porter, 2015, p.57). Many students undertake MOOCs as a means of adding to their professional knowledge and skills and future employability. They are keen to receive an official document confirming their successful completion of an LSI course. There is an associated cost to this administrative burden and as a result a certificate of completion is provided for a nominal charge. Since 2017, in an effort to automate and reduce the administrative time cost, this has been available in digital download format as opposed to a printed document, which previously was posted to a student's provided address.

Given the difficulties in administering a course for 3,200 students, processes that could be automated were considered for development in order to free up staff time. A report was configured to determine student eligibility for the certificate of completion automatically, as was a self-service feature for students to download their own certificate via the Moodle website, once criteria were met.

4 Results

The following section looks at some of the positive impacts of the MOOC on student numbers and engagement on the actual MOOC, in addition to the wider impact on other paid courses provided at the Diploma Centre. A statistical analysis was conducted which assessed the profile of registrants over the years 2016 to 2018 under the headings of age, location, gender and education level, and compared with the participant profile of students who follow the course through to completion. (see Appendix A)

4.1 Student numbers

Participant numbers on the MOOC have undergone an upward transformation - from a student count of 900 on the inaugural offering in 2014 to over 3,200 in 2017, and a cumulative total of 10,000 students over the five year period since 2014.

Fig. 7: MOOC student numbers 2014 - 2018

The figure above demonstrates an increase year on year from 2014 to 2017. The registration figure dips for 2018 and one possible explanation for this is the topic considered, namely Sports Law. This topic was more bespoke and did not have as much widespread appeal as that of the previous year, which dealt with Employment Law.

4.2 Higher completion rate

As the literature on MOOCs attests not all those who register may take an active part and whilst Jordan (2013) notes completion rates averaging at 13%, Wasserman (2014) has a more positive view and concludes that all those who register to a MOOC demonstrate an interest in learning and usually engage with the material beyond browsing and therefore form part of the valuable marketing base. High dropout rates are not necessarily negative, with some evidence suggesting that learners simply take what they need from the offering (Conole, 2016), Ronkowitz, 2013). Also Qayyum (2017) argues that high dropout rates can be misunderstood as it is not entirely appropriate to view them as courses in the conventional sense, with many enrolling on MOOCs simply to use them as a resource.

In order to successfully complete the course, participants were required to have met the following criteria:

  1. Watched all of the course content.
  2. Successfully completed all multiple choice quizzes - with a mark of 80 per cent on each or higher - and received four badges as a result.
  3. Actively engaged on the Discussion Forum by posting at least three times to any of the forums over the duration of the course.

Research from 2013 indicates that typically less than 7 percent of students complete MOOC programs (Times Higher Education, 2013). HarvardX and MITx have gone further, reporting in January 2017 that only 5.5 percent of people who enrol in one of their open online courses earn a certificate (Massachusetts Institute of Technology, 2017).

In total, 407 students completed the 2017 Employment Law MOOC, which represents 12.6 percent of the total student count and over double the international standard completion rate. The median completion rate for Diploma Centre MOOCs over the period 2014 - 2018 is 8.5 percent, with the years since the overhaul of the Moodle facility holding a median completion rate of 11.7 percent.

Fig 8: Course completion rates 2014 - 2018

There was no significant difference in demographic profile between the initial registrants and those who followed the course through to completion.Over the three years 2016-2018, the breakdown of students who registered in terms of age, gender, location and education level was statistically similar to those who completed the course (Appendix A).

4.3 DIGITAL CERTIFICATE OF COMPLETION SALES

Of the students who completed the 2017 course, 55 percent also opted to purchase the certificate of completion. This indicates that although the course was free and the certificate optional, more than half of those students who finished the course felt the need to invest in a certificate to demonstrate their undertaking.

'Going digital' led to savings in terms of printing and postage costs, reduced wait times through removing the requirement to obtain signatures from executive staff prior to issue, and improved feedback in the survey responses. There was also an increase in certificate sales, with 224 digital certificates sold compared to 193 physical certificate sales in the previous year - a profit increase of 14 per cent.

4.4 Crossover to diploma courses

From a business perspective, the MOOC initiative has created positive buy-in for diploma courses.. The opportunity for student conversions onto more in-depth diploma courses on a similar topic is one of the positive impacts and an endorsement for our learning approach and overall student buy-in. The chart below illustrates the crossover rate from each year's MOOC to the subsequent diploma or certificate course which followed in the relevant subject area. For instance, of the seventy-eight students who undertook the postgraduate Certificate in Data Protection Practice in Autumn 2016, twenty-three of these had undertaken the previous summer's MOOC.

Fig 9: Crossover rate 2016-2018

4.5 Expanding reach to non-legal professionals

The demographic analysis showed consistency in the education level of students signing up every year. In all cases, over 50% of participants were qualified to postgraduate level, indicating the high education level of students undertaking the MOOC.

Education

Secondary school

Undergrad cert/diploma

Undergrad degree

Postgrad

PhD

2016 Total Registered %

6%

10%

31%

50%

3%

2016 Total Completed %

4%

9%

34%

52%

1%

2017 Total Registered %

5%

10%

33%

50%

2%

2017 Total Completed %

4%

8%

31%

56%

1%

2018 Total Registered %

6%

10%

34%

48%

2%

2018 Total Completed %

5%

8%

34%

52%

1%

Fig 10: Education level of participants 2016-2018

The proportion of students from non-legal backgrounds undertaking the MOOC has averaged 37 percent of the total participants from 2016 to 2018. In 2017 for instance, over 600 students identified themselves as non-legal professionals. Grouping by education status, 1,328 students (41 percent) identified their highest education level attained as within the non-legal sphere (e.g. other undergraduate, other postgraduate). 29 percent of the students who took the sign-up survey identified themselves as Human Resources professionals. Below is a summary of the professional backgrounds of those attending the 2017 Employment Law MOOC:

Top five professions by student count (2017):

Solicitor 687

Human resources 207

Trainee 179

Other legal/paralegal 133

Student 83

Fig 11: Student background by year 2015 - 2018

4.6 Demand from international audience

The practices employed on MOOC offerings by the Diploma Centre were a huge driver in unlocking the potential for development of online learning initiatives generally. The survey responses and feedback from students affirmed a strong interest and uptake internationally for online learning, with over 334 students on the 2017 MOOC indicating that they were based outside of Ireland. Appealing to international students helped expand the Diploma Centre's target market for paid up certificate or diploma courses running in autumn.

Our three year analysis showed a significant increase in internationally based students signing up for the course, rising from 4 percent in 2016 to 13 percent in 2018.

Location

International

National

2016 Total Registered %

4%

96%

2016 Total Completed %

6%

94%

2017 Total Registered %

10%

90%

2017 Total Completed %

3%

97%

2018 Total Registered %

13%

87%

2018 Total Completed %

5%

95%

Fig 12: Student registrants by location 2016 - 2018

4.7 Final surveys

Responses from the final MOOC evaluation surveys have been considerably positive. Questions in 2017 and 2018 were tailored around improvements within the Diploma Hub and the usefulness of the learning materials. Many students spoke very highly of the experience in comparison to MOOCs run by other institutions. One student commented 'I felt that the videos were really well produced and watchable. This is not always the case with MOOCs I have taken. The production quality was high and the videos were engaging.'

· 98 percent of students surveyed indicated the course met or exceed expectations. This increased to 100 percent in 2018.

· 93 percent of students who viewed the live Q&A sessions rated them as 'good' or 'excellent'. This increased to 98 percent in 2018.

· 73 percent of students indicated they intend to undertake further study in the subject area.

A more detailed analysis of surveys over the three year period 2016 - 2018 is provided below:

Total number of survey responses

2016

2017

2018

186

309

49

Course met or exceeded expectations (percent of students who agreed)

2016

2017

2018

95

98

100

The figures below indicate the percentage of students who described the following resources as 'good' or 'excellent':

Quality of online learning environment

2016

2017

2018

97

98

100

Quality and usefulness of live Friday sessions (of those who took part):

2016

2017

2018

93

93

98

Quality of lecture videos

2016

2017

2018

n/a (q not asked)

98.7

100

4.8 Consistency of student engagement

*At the time of writing, data from the 2018 course only was available.

MOOCs are commonly associated with a bottleneck of student activity in the initial phase followed by a steep drop-off in engagement thereafter. Evidence from our most recent program shows a high level of student engagement across the subsequent weeks of the course. Howarth et al introduce the concept of the 'funnel of participation' with drop off occurring at various stages along the way namely "awareness-registration-activity- progress- certification" (2016, p.76). Drop off is dependent on the goals and motivating factors of those taking part and whether these goals have been stated and involves the participants continued exercise of value judgements about whether to continue to the next stage or not.

4.8.1 Video watch time

The chart below illustrates consumption of weekly video content by students. Each segment represents the total number of minutes consumed from each week's video resources. After an initial (though not severe) drop in the first week, there is relative consistency in the weeks that followed. Week one accounts for 30 percent of the total video watch time, with weeks two to five averaging 18 percent of the total.

Fig 13: Minutes consumed per weekly video resources 2018

4.8.2 Quiz attempts

The number of quiz attempts declined after week one, though not sharply. Week one secured 27 percent of all quiz attempts with weeks two to five averaging 18 percent of the total.

Fig 14: Number of attempts per weekly quiz 2018

4.8.3 Discussion threads and replies

Illustrated below are (a)the number of discussion forum threads created each week and(b)the number of replies per thread, demonstrating relative consistency each week.

Fig 15: Discussion forum threads per weekly module 2018

Fig 16: Discussion forum replies per thread 2018

5 Discussion

With five years of experience in the delivery of MOOC programmes and guided by best practice, the Diploma Centre has adapted its approach and implemented a number of key strategies to combat the perceived shortfalls of this form of learning. Most importantly, focusing on the user experience for online students can lead to increased course consumption and follow-through on weekly content. This takes the form of understanding both good web design and aesthetic concepts to enhance the user experience, and offering high quality learning content with clear learning objectives to interested students. The user experience has greatly improved through the enhancement of our Moodle system to support student interaction.

The process of running the MOOC programme has highlighted the importance of adapting digital lecture content to suit the medium. As online and onsite learning are separate domains, they require different practices to perform optimally for the learner. For instance, producing shorter 'bite size' lectures to keep students focused is a strategy that can be applied to other courses that use the online lecture capture facility. Making online and blended learning courses more 'MOOC-like' has been advantageous in securing students' attention, promoting engagement, and harnessing the web environment for students' benefit.

The MOOC has also functioned as an excellent 'sandpit' or testing ground for new learning approaches, with feedback from students influencing the team's decision to incorporate new technologies and increasing the team's ability to produce high quality learning content. Free online technology platforms such as YouTube and OBS [7] have been utilised in a learning context and subsequently integrated into many Diploma Centre courses.

Digitising processes which would be considered onerous in terms of course administration for thousands of students (such as an automated system to administer the certificate of completion) has provided the double benefits of freeing up staff resources and saving on cost.

By connecting more effectively to students on a virtual basis, the Diploma Centre has boosted its reputation as a flagship provider for online learning and secured international buy-in for the MOOC programme each year. From a business perspective, the MOOC programme has become an invaluable tool to introduce students from alternative backgrounds to the LSI. MOOCs can become sustainable (or even profitable) through the conversion of even a small proportion of students from free MOOCs to paid courses run by the offering institution. On the evaluation feedback form, over a quarter of students indicated their motivation for undertaking the course was 'to avail of the opportunity to study with the LSI', with the positive experience enhancing the institution's reputation for excellence in learning.

Positive feedback, a significant public relations value, and high return rates from students have confirmed the sustainability of the LSI's MOOC offerings.

5.1 Further developments

In 2017, the Diploma Centre was awarded a grant by the European Commission for the provision of a training course on the EU Victims' Impact Directive using the existing MOOC infrastructure. The course utilised the same format as the Employment Law MOOC, following the overhaul of Moodle, and attracted interest from the Gardaí [8] , Victims Support Groups, and related professions in Ireland and abroad. The MOOC programme has been instrumental in expanding outwards to European partners and non-government organisations within Ireland. This was the first opportunity to run a second MOOC in a given year, and the first to incorporate project partners at an international level.

One challenge which the Diploma Centre seeks to address on upcoming online initiatives is to provide further opportunities for problem-centric learning . This has been ranked as the most important out of five leading factors for promoting engagement in a landmark study of three highly rated MOOCs in 2014 (Hew, 2014). A problem centric instruction is 'concerned about teaching learners the necessary concepts or skills in order to understand or solve real-world tasks (p.10)'. The Diploma MOOC lecture presentations take a practical focus and seek to provide information to assist participants to gain greater understanding of the problems encountered by them in their particular domain of work or interest. For example, the Sports Law MOOC featured a module on sports clubs, with practical insight on areas of liability for persons working and volunteering within clubs, including guidance on the protection of players and management. It should be noted that this information should not be confused with legal advice and each page on the MOOC site contains warnings to this effect. The provision of video resources with comprehensive legal knowledge would be appropriately categorised as expositional rather than problem-centric in nature, however. In recent offerings, the Diploma Centre team has constructed simple problem scenarios on the discussion forum, and 'Twitter tasks' for students to tweet a response to, in order to introduce a problem-solving element to the student interaction. Further MOOC programmes will seek to expand on this learning strategy by using a problem-solving approach to create meaningful engagement amongst students online.

6 Conclusion

This paper considers the educational practices in successfully delivering a MOOC by focusing on the instructional design process, finding technology-based solutions to learning problems, and adapting an 'open access' approach to public legal education. The goal was to widen the provision of public legal education on a large scale and to introduce thousands of potential Diploma Centre students to our learning resources while increasing their understanding of the law. This digital initiative opened up the LSI to a large audience of legal and non-legal professionals alike and demonstrated the opportunities of utilising the Moodle learning management system in the provision of a successful MOOC programme.

The implications and key suggestions for practice and policy in relation to MOOCs are to focus on the user experience for online students (i.e. strategies for design of virtual learning environments such as Moodle), as these can lead to increased course consumption and follow-through on weekly content. Peer-to-peer learning is an essential component of MOOCs, and can be leveraged effectively when integrated into assessment methods. Finally, we posit that MOOCs can become sustainable through conversion of even a small proportion of students from free MOOCs to paid courses run by the offering institution and that they need not incur a large cost base to run if a similar model to the LSI model is followed.

Bibliography

Chen, Y. (2014) 'Investigating MOOCs through blog mining', The International Review of Research in Open and Distributed Learning, 15(2), pp. 85-106. doi: 10.19173/irrodl.v15i2.1695

Clark, R. C. and Mayer, R. E. (2011) E-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning (New Jersey: John Wiley & Sons, Inc.)

Coatzee, D., Fox, A., Hearst, M. A. and Hartmann, B. (2014) 'Should your MOOC forum use a reputation system' in Proc. CSCW 2014, pp. 1176-1187 (New York: ACM Press)

Coffrin, C., de Barba, P., Corrin L., and Kennedy G. (2014) 'Visualizing patterns of student engagement and performance in MOOCs' in Proc. LAK 2014 , pp. 83 -92 (New York: ACM Press)

Conole, G. (2016) 'Designing effective MOOCs Educational Media International, 52(4) p. 239-252.

Csikszentmihalyi, M, (1990) Flow: The Psychology of Optimal Experience (New York: Harper & Row) cited by Sun, R. (2012) 'The role of instant feedback in education', Huffington Post . Available at: https://www.huffingtonpost.com/robert-sun/the-role-of-instant-feedb_b_1660459 (accessed 12 September 2018)

Das, S. (2012) 'Increasinginstructor visibility in online courses through mini-videos and screencasting', Online Student Engagement Tools And Strategies, 9

Gaebel, M. (2015) 'E-learning in European higher education institutions - results of two EUA studies in 2014 and 2015', European Higher Education Area . Available at: http://bolognayerevan2015.ehea.info/files/Michael%20Gaebel.pdf (Accessed 20 December 2016)

Guo, P.J., Kim, J. and Rubin, R. (2014) 'How video production affects student engagement: An empirical study of MOOC videos' in Proceedings Of The First ACM Conference On Learning@ Scale Conference 2014, pp. 41-50. (New York: ACM Press)

Hew, K. F. (2014) 'Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCs', British Journal of Education Technology, 47(2), pp. 320-341. doi: 10.1111/bjet.12235

Hew, K. F. and Cheung, W. S. (2014) 'Students' and instructors' use of massive open online courses (MOOCs): Motivations and challenges', Educational Research Review, 12, pp. 45-58. doi: 10.1016/j.edurev.2014.05.001

Kelly, R. (2012) 'Tips from the pros: 4 ways to engage students', Online Student Engagement Tools And Strategies, p.10

Khalil, M., Brunner, H. and Ebner, M., 2015. Evaluation grid for xMOOCs. International Journal of Emerging Technologies in Learning (iJET) , 10 (4), p.40

Krug, S. (2014) Don't Make Me Think: A Common Sense Approach To Web And Mobile Usability. 3rdEd. USA: New Riders, p 18

Mahraj, K. (2012) 'Using information expertise to enhance massive open online courses', Public Services Quarterly, 8(4), pp. 359-368. doi: 10.1080/15228959.2012.730415

Marcell, M. (2008) 'Effectiveness of regular online quizzing in increasing class participation and preparation, International Journal for the Scholarship of Teaching and Learning, 2(1), art. 7. Available at: https://doi.org/10.20429/ijsotl.2008.020107 (Accessed 29 August 2018)

Margaryan, A., Bianco, M., & Littlejohn, A. (2015) 'Instructional quality of massive open online courses (MOOCs)', Computers & Education, pp. 80-77. doi: 10.1016/j.compedu.2014.08.005

Massachusetts Institute of Technology (2017) 'Study of MOOCs offers insights into online learner engagement and behavior', MIT Office of Digital Learning. Available at: https://blog.edx.org/study-moocs-offers-insights-online-learner-engagement-behavior (Accessed 29 August 2018)

Means, B., Murphy, R. and Bakia, M. (2013), 'The effectiveness of online and blended learning: a meta-analysis of the empirical literature', Teachers College Record, 115(3), pp. 1-47

Nagel, L., and Kotzé, T.G. (2009) 'Supersizing e-learning: What a CoI survey reveals about teaching presence in a large online class', Internet and Higher Education. doi:10.1016/j.iheduc.2009.12.001

Nawrot, I. and Doucet, A. (2014) 'Building engagement for MOOC students' in Proc. International World Wide Web Conference, 2014, pp. 1077-1082. (New York: ACM Press)

Porter, S. (2015) 'The economics of MOOCs: A sustainable future?', The Bottom Line, 28(1/2), pp. 52-62. doi: 10.1108/BL-12-2014-0035

Ronan, A. (2015) 'Whatever happened to course management systems?', Edudemic. Available at: www.edudemic.com/whatever-happened-course-management-systems (Accessed 29 August 2018)

Ronkowitz, K. (2013). Rethinking lurkers in the MOOC experience. http://www.serendipity35.net/ index.php?/archives/2826-Rethinking-Lurkers-in-the-MOOC-Experience.html

Qayyum, A. (2017) 'A Maturing View of MOOCs'. American Journal of Distance Education, 31(2), pp 78-79

Ruth, S. (2012) 'Can MOOCs and existing e-learning efficiency paradigms help reduce college costs?' SSRN Electronic Journal. doi: 10.2139/ssrn.2086689

Siemens, G., 2013. Massive Open Online Courses: Innovation in education, Open Educational Resources: Innovation, Research and Practice, RoryMcGreal, Athabasca. UNESCO, ed , 1 , p.268.

Szpunar K., Khan N., and Schacter, D. (2013) 'Interpolated memory tests reduce mind wandering and improve learning of online lectures', PNAS, 110(16). Available at: http://www.pnas.org/content/110/16/6313 (Accessed 29 August 2018)

Times Higher Education (2013) 'MOOC completion rates "below 7%"', Times Higher Education. Available at: https://www.timeshighereducation.com/news/mooc-completion-rates-below-7/2003710.article (Accessed 29 August 2018)

Warren J., Rixner, S., Greiner, J. and Wong, S. (2014) 'Facilitating human interaction in an online programming course' in Proc. SIGCSE 2014, pp. 665-670. (New York: ACM Press)

Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018) 'What research says about MOOCs - An explorative content analysis', The International Review of Research in Open and Distributed Learning, 19(1)


Appendix A : MOOC demographic profile 2016 - 2018:

Gender

Male

Female

2016 Total Registered %

64%

36%

2016 Total Completed %

51%

47%

2017 Total Registered %

29%

71%

2017 Total Completed %

37%

63%

2018 Total Registered %

55%

45%

2018 Total Completed %

50%

50%

Age

20-29

30-39

40-49

50-59

60-69

Over 70

2016 Total Registered %

23%

38%

25%

11%

3%

0%

2016 Total Completed %

22%

39%

22%

11%

6%

2017 Total Registered %

24%

37%

25%

12%

2%

0%

2017 Total Completed %

24%

39%

23%

12%

2%

2018 Total Registered %

25%

33%

25%

12%

3%

0%

2018 Total Completed %

24%

29%

23%

14%

3%

0%

Location

International

National

2016 Total Registered %

4%

96%

2016 Total Completed %

6%

94%

2017 Total Registered %

10%

90%

2017 Total Completed %

3%

97%

2018 Total Registered %

13%

87%

2018 Total Completed %

5%

95%

Education

Secondary school

Undergrad cert/diploma

Undergrad degree

Postgrad

PhD

2016 Total Registered %

6%

10%

31%

50%

3%

2016 Total Completed %

4%

9%

34%

52%

1%

2017 Total Registered %

5%

10%

33%

50%

2%

2017 Total Completed %

4%

8%

31%

56%

1%

2018 Total Registered %

6%

10%

34%

48%

2%

2018 Total Completed %

5%

8%

34%

52%

1%

[1] Dr. Freda Grealy (Head of Diploma Centre, Solicitor, Law Society of Ireland), Steve Collender (Multimedia Coordinator, Diploma Centre), John Lunney (Solicitor, Course Manager, Law Society of Ireland), Rory O'Boyle (Solicitor, Course Manager, Law Society of Ireland).

[2] Certain areas of CPD such as conveyancing, advocacy skills and litigation courses are reserved solely for solicitors. Other topics such as employment law, contracts, aviation finance have broader appeal and are offered to those with experience and working in these areas. Many attend for the purpose of enhancing their knowledge or to better perform their current role whatever that might be and clearly not with the intention of providing the public with legal advice following their studies. .

[3] For further discussion on the development of blended learning at the Diploma Centre, Law Society of Ireland, see Grealy, F. (2015) ' Mobile professional learning for the legal profession in Ireland - a student-centred approach', The Law Teacher , 49(3), pp. 303-322, DOI: 10.1080/03069400.2015.1035529 .

See also Grealy, F., Kennedy, C., & O'Boyle, R. (2013) 'Webcasting of Continuing Professional Development Courses: an effective learning and revisionary tool?' European Journal of Current Legal Issues , 19(4) andBainbridge, J., Counsell, K., Grealy, F., Maharg, P., Mills, J., & O'Boyle, R. (2013) 'iPads in Legal Learning (iLEGALL): mobile devices in professional legal learning', European Journalof Law and Technology , Vol 4(1)

[4] Ibid., 240

[5] Peters, D. 2018. MOOCs are not dead, but evolving. 22 February. University Affairs. Accessed 28 March 2019. Available at https://www.universityaffairs.ca/news/news-article/moocs-not-dead-evolving/

[6] See https://www.panopto.com/

[7] Open Broadcast Software

[8] Gardaí is a Gaelic word and the name for the Irish Policing Authority.