Diana Laurillard 8:18 AM These are not good examples of 'personalised learning and mastery' surely? - not given the sophistication that e-learning is capable of these days. Geoff Crisp 8:20 AM presumably there will be a wide variety of tasks in MOOCs, just as in other formats? Diana Laurillard 8:20 AM With questions of that kind you could learn those facts by just taking and retaking the test, and never taking any course. Better to use real examples? Geoff Crisp 8:22 AM xMOOC and cMMOC differences? Wendy Taleo 8:22 AM taking tests multiple times is familiar for the VET sector Sally Jordan 8:22 AM @Diana - there was an example of a free-text question in an earlier slide (though not a great one)- those who know me will know I want to know fact you can push that one in Coursera. Bernard Nkuyubwatsi 8:22 AM I like peer grading. A way to practice providing constructive feedback, a critical skill even in real life Wendy Taleo 8:23 AM @Geoff..the best MOOC I've done has been cMOOC Geoff Crisp 8:23 AM MOOCs are good format for self and peer assessment formats Diana Laurillard 8:24 AM I don't think so Geoff. xMOOCs as more didactic are fine. It's just that MCQs of facts do not make good exemplars. Wendy Taleo 8:24 AM I've particpated and seen peer assessment work in a MOOC, have yet to implement one in Blackboard Learn Bernard Nkuyubwatsi 8:25 AM cMOOCs are probably good for lifelong learning and network creation, not good for didactic learning Wendy Taleo 8:25 AM @Bernard agree, good for lifelong learning. Diana Laurillard 8:26 AM Agree - peer ass't is a very good pedagogic practice if run well. peer grading mostly viable, and probably as well correlated as lecturers would be! Mathew Hillier 8:26 AM given this distrubution, how does this look for students who have yet to get a degree? Johnny #3 8:26 AM MOOCs, for all their limitations, are making education available for huge no's that wouldn't otherwise have the chance Geoff Crisp 8:27 AM interesting issue with MCQs. When we wanted to move people on with assessment questions in transforming assessment we started with MCQs because we knew that was were a lot of people were. We then showed how to URL link resources inside the question so students needed to use a digital resource to respond to the question. It allowed the teacher to change the MCQ to a more conceptual mode rather than just declarative knowledge Diana Laurillard 8:27 AM That's nice! Gordon Joughin 8:27 AM Sure is! Sally Jordan 8:28 AM Good point Geoff David Callaghan - Edge Hill University 8:28 AM Peer grading allows the student a glimpse of academia’s 'secret garden' - I've used it very (extremely) successfully with a large cohort. Carol Gray 8:28 AM So we won't ever completely replace F2F!! Wendy Taleo 8:29 AM Any way to deal with submission times across different time zones? Can cause problems if students miss the submission time because of mis understanding the timezone of the originating University Connie (Curtin) 8:29 AM As would embedding video or other media, such as an interactive simulation. The issue isn't with how you ask the question or how students respond but rather the quality of the question. David Callaghan - Edge Hill University 8:29 AM @wendy - have a countdown rather than specific times. Geoff Crisp 8:29 AM Why do we have such a prescriptive approach to submission? Why not a window of time for submission? Wendy Taleo 8:29 AM @david interesting. Diana Laurillard 8:30 AM When we first ran student comms for a philosophy course at the OU in the 90s the lecturers were horrified at the way students' led each other astray in answering each others' questions, and wanted to stop the whole thing! - rather than learning from this experience how easy it is for them to misunderstand. It was an experience that taught them a lot. it taught me you must always have a way of monitoring student discussion outputs at some point. David Callaghan - Edge Hill University 8:30 AM @diane - but how can we monitor 100K? Wendy Taleo 8:30 AM @Geoff, yes that's another good idea, I think cutoff dates are outdated! Tim Hunt 8:30 AM Yes, but now student discusisons in forums are a significant fraction of the traffic in the OU VLE. And most of them are monitored by ALs. But, good lessons for students in informtion literacy, and backing up your claims with citations. Sally Jordan 8:31 AM For the non-OU folk: AL = asssociate lecturer (tutor) Connie (Curtin) 8:31 AM Due dates also foster the last minute attitude rather than encouraging iterative development of their ideas/learning Janet Gordon 8:31 AM Thanks, @Sally Wendy Taleo 8:31 AM Point from Andrew...the stats show that peer assessment "generally" works....not always Jus 8:31 AM As a student and teacher I think submission windows is a great idea. Diana Laurillard 8:32 AM @David - do you remember a technique called 'pyramid discussion groups'? It's a bit pyramid but technology is good at orchestrating large numbers. David Callaghan - Edge Hill University 8:32 AM @diana - no - but thanks for the pointer - I'll look tomorrow - thanks! Jus 8:32 AM With primary students we have found peer and teacher assessment (based in criteria and a marking matrix) correlate highly Diana Laurillard 8:33 AM Sorry, that was meant to be 'big' pyramid! Connie (Curtin) 8:33 AM Assessment analytics is now seen as a valid subset of learning analytics. Recent paper by Cath Ellis on this. Tim Hunt 8:34 AM What do you mean 'now'? People have been doing that for years. They have just not used a buzz-word to describe it. Of corse, more poeple doing it is good. Bobby Elliott 8:34 AM any link to the cath ellis paper? Wendy Taleo 8:34 AM Talking heads have limited value... Diana Laurillard 8:35 AM @Tim - the AL model is great, but it's what, about 1:25? with 100,000 students that 4000 ALs. We do need some ways of gearing tutor work a little better! Connie (Curtin) 8:35 AM True Tim, I think the advantage is the type of data and amount we now have access to and ability to correlate to other data point Geoff Crisp 8:35 AM Ellis, C (2013) ‘Broadening the Scope and Increasing the Usefulness of Learning Analytics: The Case for Assessment Analytics’ British Journal of Educational Technology Bobby Elliott 8:35 AM thanks Connie (Curtin) 8:35 AM Thanks Geoff , couldn't look it up easily for the iPad Tim Hunt 8:35 AM @Diana, well you ahve to trust students to help each other without supervision. As online interaction gets more common, that may work. David Callaghan - Edge Hill University 8:36 AM @Wendy - perhaps a 'little' talking head to engage the student up front - and then withdraw such support - reminds me of Salmon's 5 stage model. Bobby Elliott 8:36 AM PDF of afore mentioned paper: http://eprints.hud.ac.uk/16829/3/EllisBroadeningBJET_submission.pdf Tim Hunt 8:36 AM Also, Ideas to identify good and bad answers to questions. Are you familiar with the mechanics of the StackExchange sites. Wendy Taleo 8:36 AM @David...yes I think that's a good idea...pity Camtasia doesn't allow you to remove the video after a short time. David Callaghan - Edge Hill University 8:36 AM @Diana @Tim - working together = more like the 'real world'? Geoff Crisp 8:36 AM good use of assessment analytics and gives good feedback to the teacher Andrew Eileen Dessaso 8:38 AM Will the recording include the chat history? There are a lot of excellent links and information that would be of benefit. Diana Laurillard 8:38 AM @TIm - students will help each other of course, whatever we do. But we should not rely on them being able to always do it well. I think the technology is capable of coming up with something better than that. Worth trying for... Andreas 1 8:38 AM @Wendy @David would be great if students can turn on and off the talking head to assist to their personal needs. Wendy Taleo 8:38 AM @Andreas - yes that would be great! give the student control Geoff Crisp 8:39 AM the archive has the chat as well David Callaghan - Edge Hill University 8:39 AM @wendy @Andreas1 - and change the gender of the presenter too? Connie (Curtin) 8:39 AM @ Tim, are you referring to a voting up/down system in the stack exchange sites? Tim Hunt 8:39 AM @Diana, are you thinking of somethign like a forum that automaticaly notifies Moderators about a post that has not received a satisfactory answer after 24 hours, or something like that? Sally Jordan 8:39 AM I agree Geoff - I have found out quite a lot from analysing answers from thousands of students. Hate to return to previous point, but responses to non MCQ questions are more reliable because you know the student really thought that i.e. wasnt't guessing. Typing too fast - hope that makes sense. Diana Laurillard 8:39 AM Good to see Bloom getting an airing - we should not forget that some of the old ideas are good too! Tim Hunt 8:40 AM @Connie, yes. Geoff Crisp 8:40 AM yes, the usefulness of the student responses for the teacher depends on the teacher setting a useful task for the students to do! Connie (Curtin) 8:40 AM @ Tim, would be good then if the system weighted votes so that an expert vote is say worth 10 student votes David Callaghan - Edge Hill University 8:42 AM @Connie - perhaps some automation to weight those students that are closest to the 'expert' opinion? Connie (Curtin) 8:42 AM @david, yes nice idea Tim Hunt 8:43 AM @David, @Connie, StackExcahnge has a 'reputation' score system, that does that sort of thing. I think they have written some articles about how it works. They have taken quite a deliberate, reflective approach to designing how it works in an evolutionary way. Andreas 1 8:43 AM @David - but who would make that judgements about the expertise level of the answers? Since we already establish that the instructor can't read all of them? Peer-"like"ing? Geoff Crisp 8:44 AM good use of MOOCs Andrew Tim Hunt 8:44 AM http://meta.stackoverflow.com/questions/139280/how-did-stack-exchange-bootstrap-its-reputation-system Mathew Hillier 8:44 AM flipped classroom model is going gangbusters here at UQ. Diana Laurillard 8:44 AM @Tim - that's a good idea, but I was thinking more about getting students to discuss and help each other in groups, to produce an agreed answer, submit it to a voting forum along with, say 50 other groups to vote on the best answers, combine these with 100 other groups of 50 groups etc, and eventually the tutor gets maybe 20 answers to read and react to - generating from 100k students's discussions and votes... something like that... Bobby Elliott 8:45 AM interesting idea diana Connie (Curtin) 8:45 AM @tim, that is sort of what I meant be expert, so that it doesn't have to always be the lecturer who validates correctness David Callaghan - Edge Hill University 8:45 AM @Andreas 1 - the system could compare experts review with students reviews - only a few expert reviews would be needed. Tim Hunt 8:45 AM @Diana. Interesting. My initial reation to that is that it seems ot require more synchronisation than you normally require online. May depend on context whether it would work. Diana Laurillard 8:46 AM True - but would work within a window of time I think. David Callaghan - Edge Hill University 8:47 AM @Diana @Tim - and remember the international aspect - 24 hour stuff - as well as the numbers. Jus 8:47 AM Really Geoff Crisp 8:47 AM education is a right not a privilege Bobby Elliott 8:47 AM thanks Diana Laurillard 8:47 AM No reason why you couldn't run that process over a day or so, managed by the technology. Hamish Macleod 8:47 AM Thanks. Andrew Claus 8:47 AM Thank you Andrew. Sue Barnes 1 8:47 AM Thank you Kevin 1 8:47 AM how to do incentivise or motivate faculty to get onboard moocs since it is time consuming? Dominique Parrish 8:48 AM Thank you Sue Milne 8:48 AM thank you Linda MacDonald 8:48 AM Thanks that wasd great James Denholm-Price 8:48 AM Thanks Andrew -- exciting: lots of data you have! Liz Elliott 8:48 AM Q what is the financial business case, etc for MOOCs Nick Falkner 8:49 AM Thank you! audley harris 8:49 AM Great concept and excellent presentation. I did a course on Coursera and am impressed with the system bron 8:49 AM great question Connie (Curtin) 8:49 AM Great question, is open the same as free? Jason Tangen 1 8:49 AM Q: Why did you/Coursera decide to go with a time-based model, where students have to take modules at a specific time/rate, rather than use Udacity's "post everything at once" model? It seems that the time-based model doesn't provide the flexibility that MOOCs are designed for. Matthew 8:50 AM especially as the content gets "richer" in terms of graphics n videos bron 8:50 AM @connie no open does not mean free but it does imply easier access Connie (Curtin) 8:51 AM @Jason, it seems that many MOOC platforms are taking this approach and as A participant I find it quite restrictive. Tim Hunt 8:51 AM open source and open educational resources both meen free. Free as is freedom and free as in beer. Phil Marston 8:51 AM Q - Given that the majority of MOOC participants are experienced learners, is it really likely that potential participants that are less experienced (especially at self directed learning) are really able to access this 'open learning'? David Callaghan - Edge Hill University 8:51 AM @Bron @Connie - perhaps free means 'no required fees' from the providing organisation? Connie (Curtin) 8:52 AM I think these two things are related because if the materials were not time restricted then it would open up opportunities to share or use onward. Tim Hunt 8:52 AM @Phil, that is a key point. The experience from teh OU, UK's widening participation department is that inexperiences learners need a LOT of support to succeed in higher education. (But, given support, and their own commitment, they can succeed.) Bobby Elliott 8:53 AM 50% of households in glasgow have no internet access (2013) Tim Hunt 8:53 AM Wow! Jac 8:53 AM Q - as per Phil's question can Andrew offer some comments Gigi Johnson #2 8:53 AM What tends to be the cost -- cash and time -- of creating a MOOC for a partner? What percentage of faculty have done a second? A third? bron 8:53 AM @Tim Hunt - deeply philosophical response - what is freedom really? What is open access really? We all have to survive even if $ don't change hands - would you like to swap my sheep for a MOOC? Kevin 1 8:53 AM thank you for the response Bernard Nkuyubwatsi 8:53 AM Thank you Andrew David Callaghan - Edge Hill University 8:53 AM @Gigi - perhaps faculty don't create a second as many people have just one passion / specialism? Tim Hunt 8:54 AM @bron. Well, I am a software developer working free / open source software. So, for me, it means exactly what it says in the GNU General Public Licence. Gigi Johnson #2 8:54 AM I meant taught it a second time. I have talked to a sample who have said it was great, but have hesitated to step into the breach for a rerun of the process again for the same course. Tim Hunt 8:54 AM Or, similarly, the terms of creative commons licences. bron 8:55 AM @ Tim Hunt but don't you also rely on donations whether time or $? Connie (Curtin) 8:56 AM Self managed learners Tim Hunt 8:56 AM No, the OU pays me a full time salary to develop the software we need. David Callaghan - Edge Hill University 8:56 AM @Gigi - Interesting! I has thought no one 'teaches' a second time?!'Taught' it a second time? My understanding is you create a 'MOOC' and then there is little bron 8:56 AM Tim Hunt 8:56 AM Business models for open source projects, and open source developers, is an interesting topic, but it has been written about before. David Callaghan - Edge Hill University 8:56 AM @Gigi - my keyboard is faulty - I'm going to stop now. Anne Gilmore 8:57 AM Clearly one of the possible outcomes of MOOCs is the ability to create learning materials that are then used by other institutions. What is courseras and its institutional partners view on others using their materials to deliver their own degree programs whether in a more f2f or online environment? Jac 8:57 AM Q - Can Andrew comment - what strategies used to address cross cultural sesnitivies Adrian Norman 8:57 AM Wonder what the percentage is on novice learners doing MOOCs? ie don't have a degree??? Teresa 8:57 AM thank you for an interesting discussion, must go Mathew Hillier 8:57 AM The 'moodle' type of busienss model could work here too. joyce latulippe 8:57 AM how might MOOCs synch up or otherwise support with trends in virtual high schools? Gigi Johnson #2 8:58 AM Thanks for organizing this. Jac 8:58 AM Thanks Andrew, Mathew and Geoff Phil Marston 8:59 AM In case I don't get to ask - my question was related to Education for Everyone - but you need to be 'learning literate' to take advantage of MOOCs - so it's not really for everyone ... Diana Laurillard 8:59 AM not just about affordability, but scale at which each tutor can offer support Anthea 9:00 AM You would need an adaptive learning model to cater for different levels bron 9:01 AM this type of MOOC does not follow principles of first MOOC - connectivism...is more of a behaviourist model of learning...which is better really? Diana Laurillard 9:01 AM Yes, student proctors, as in Keller plan (1960?) is a good idea, and good pedagogy that could be orchestrated by technology - we need those tools Liz Elliott 9:01 AM Q what is the financial case for an organsisation Gigi Johnson #2 9:01 AM @David, there have been some "reruns," with tweaking of the video for the feedback of what worked and coaching the discussion boards and process again. At $3K/hour of video min. est for production (rumored min. average, though the spectrum is wide), one might hope that it can be run another time! I'm most interested in the "syndication" of this great content for later use, so following the question. Diana Laurillard 9:02 AM Keller, F. S. (1974). Ten years of personalized instruction. Teaching of Psychology, 1 , 4-9. bron 9:02 AM great to hear your presentation Andrew Mathew Hillier 9:02 AM Andrew - most welcome! David Callaghan - Edge Hill University 9:02 AM @Andrew - Thank you very much too. Mathew Hillier 9:02 AM audience - please fill in our survey joyce latulippe 9:02 AM Thank You! audley harris 9:02 AM Thank you greatly Gigi Johnson #2 9:02 AM @Andrew -- thanks! Kerri 9:02 AM Thank you Bobby Elliott 9:02 AM thanks everyone especially andrew Hannah Marrinan 9:02 AM Thank you Christine Bassett 9:02 AM Thanks Andrew Hamish Macleod 9:02 AM Bye all - thanks Andrew. Bobby Elliott 9:02 AM cheer geoff Liz Elliott 9:02 AM Andrew thank you very much Mathew Hillier 9:02 AM session recording will be relased shortly - including the chat log. Phil Marston 9:02 AM TQ Abhijeet Lekhi 9:02 AM Thank You! Adrian Norman 9:02 AM Thanks! James Denholm-Price 9:02 AM Thanks Andrew, Geoff & Matthew Carol Gray 9:02 AM thanks Tim Hunt 9:02 AM Thanks. Alison Lockley 9:02 AM Thankyou! Interesting discussions. Anthea 9:02 AM Thnak you Kristin 9:02 AM thank you Arun 9:03 AM Thanks Asha Ginda 9:03 AM Goodnight and thanks Deb Murdoch 9:03 AM thank you