WEBVTT 00:00:20.000 --> 00:00:20.900 Hello from Brissy! 00:00:25.000 --> 00:00:25.900 Good Morning from Pretoria, South Africa 00:00:37.000 --> 00:00:37.900 Hello, from Coimbra, Portugal! 00:00:39.000 --> 00:00:39.900 Hello from the Gold Coast! 00:00:41.000 --> 00:00:41.900 Good daay! πŸ™‚ 00:01:02.000 --> 00:01:02.900 Good Morning! 00:01:06.000 --> 00:01:06.900 Good morning from Rugby, England 😎 00:01:09.000 --> 00:01:09.900 Hi from Newcastle (UK) - not so far from you Stuart, morning! 00:01:11.000 --> 00:01:11.900 Hello from Penrith, Sydney 00:02:57.000 --> 00:02:57.900 no sound 00:02:59.000 --> 00:02:59.900 Can't hear anything. I guess I'll log off and rejoin. 00:03:08.000 --> 00:03:08.900 yes 00:03:09.000 --> 00:03:09.900 yes 00:03:11.000 --> 00:03:11.900 yes Yes yes yes I can hear you yes Yes 00:03:12.000 --> 00:03:12.900 yes Yes yes yes 00:03:13.000 --> 00:03:13.900 yes I can hear Yep, loud and clear. yes yes I can hear πŸ‘ yes 00:03:14.000 --> 00:03:14.900 πŸ‘ all good 00:03:15.000 --> 00:03:15.900 yes loud n clear 00:03:16.000 --> 00:03:16.900 yes πŸ‘ 00:03:17.000 --> 00:03:17.900 Hear you well! πŸ‘ Yes indeed 00:03:18.000 --> 00:03:18.900 Yes good sound! yes yes yep 00:03:20.000 --> 00:03:20.900 😎 yes 00:03:22.000 --> 00:03:22.900 yes πŸ‘ 00:03:30.000 --> 00:03:30.900 Try chrome - i put some tips in a email i just sent you about an hour ago. 00:03:33.000 --> 00:03:33.900 If you can't hear, adjust your device settings via the cog wheel in the bottom right corner of the screen 00:03:36.000 --> 00:03:36.900 Yes, but I had to turn audio on and off yes 00:03:37.000 --> 00:03:37.900 YES 00:03:41.000 --> 00:03:41.900 Yes click on menu upper left and choose use phone 00:03:42.000 --> 00:03:42.900 yes πŸ‘ 00:03:45.000 --> 00:03:45.900 Yes yes 00:03:46.000 --> 00:03:46.900 yes 00:03:47.000 --> 00:03:47.900 Yes we are Australian Catholic University 00:03:51.000 --> 00:03:51.900 yes 00:03:56.000 --> 00:03:56.900 yes Yes yes 00:03:57.000 --> 00:03:57.900 First time! 00:03:58.000 --> 00:03:58.900 yes 00:03:59.000 --> 00:03:59.900 Newcastle has run online exams since 2007 - but differently now in lockdown 00:04:00.000 --> 00:04:00.900 yes Currently conducting. Ys yes Yes 00:04:01.000 --> 00:04:01.900 yes yes Yes yes 00:04:02.000 --> 00:04:02.900 Yes No Yes Yes 00:04:03.000 --> 00:04:03.900 Yes Yes Considering for the first time - yes Yes 00:04:04.000 --> 00:04:04.900 Yes yes considering yes yes 00:04:05.000 --> 00:04:05.900 YEs 00:04:06.000 --> 00:04:06.900 yes Yes! 00:04:08.000 --> 00:04:08.900 Yes Yes Yes 00:04:09.000 --> 00:04:09.900 sort of! yes yes 00:04:10.000 --> 00:04:10.900 yes - sort of. 00:04:11.000 --> 00:04:11.900 currently using online exams, have done so since last year we are conducting online exams - proctoring via ProctorU 00:04:12.000 --> 00:04:12.900 yes 00:04:13.000 --> 00:04:13.900 yes past experience and future interest Moved online in the past two months. Have done some before 00:04:15.000 --> 00:04:15.900 yes considering it Yes in China 00:04:16.000 --> 00:04:16.900 Yes Yes, have been doing it for al long time already 00:04:17.000 --> 00:04:17.900 More than before 00:04:18.000 --> 00:04:18.900 We've used online exam previously (4 yrs), and yes using this year. yes 00:04:20.000 --> 00:04:20.900 UNE been running them for a few years 00:04:21.000 --> 00:04:21.900 yes online alternate assessments instead of face to face yes 00:04:24.000 --> 00:04:24.900 We have ran them for awhile in different areas. I marked them last year. Moving more to online this year because of COVID-19 yes we hold exams 00:04:25.000 --> 00:04:25.900 yes, first time everything online 00:04:27.000 --> 00:04:27.900 Have done for many years but invigilated 00:04:30.000 --> 00:04:30.900 No, but a lot more people are using them 00:04:31.000 --> 00:04:31.900 If anyone needs help I'm available Yes, conducting short online exams, but new to this. Not doing online yet, but we were planning to do so even before the pandemic. 00:04:40.000 --> 00:04:40.900 yes but have also been using for a few yrs by sort of I mean the exam paper will be delivered online and answers submitted online but not truly an online exam 00:04:42.000 --> 00:04:42.900 UNE had a webinar on doing online proctoring exams. 00:04:52.000 --> 00:04:52.900 check our past sessions for examples. 00:05:03.000 --> 00:05:03.900 Critiqued by Philip Dawson here a week or two ago. 00:05:06.000 --> 00:05:06.900 @Mathew - ah interesting will take a look 00:07:40.000 --> 00:07:40.900 all past events recorded here: http://transformingassessment.com/events_past.php 00:08:08.000 --> 00:08:08.900 last week on contract cheating, oral assessment and sessions before that on teaching online and UNE's session on online proctoring. 00:09:08.000 --> 00:09:08.900 Thanks @Matthew. 00:09:49.000 --> 00:09:49.900 links back to SAMR (Puentedura) 00:09:52.000 --> 00:09:52.900 I think that most of us don't do exams for our own or our students' benefit. It is around cheating and society's desire for grading/ranking. We don't do online exam because of the lack of infrastructure support. This is so timely relevant at the moment! 00:10:20.000 --> 00:10:20.900 link to article mentioned - Cramp 2019 - https://ro.uow.edu.au/jutlp/vol16/iss1/10/ 00:10:52.000 --> 00:10:52.900 I think that is the one 00:11:00.000 --> 00:11:00.900 At my uni, online proctoring does not have support and the students don't like the idea of someone watching them. 00:11:17.000 --> 00:11:17.900 the reference list for the citations in the presentation is on the recording page for today's session http://taw.fi/6M20202 - there is a pdf link to the ref list. 00:11:18.000 --> 00:11:18.900 which? 00:11:25.000 --> 00:11:25.900 @Noel.. yes, assessment OF learning, when we and students may prefer assessment FOR and AS learning 00:11:41.000 --> 00:11:41.900 @Anreana - yes there have been reports in the media here in Queensland with students raising concenrs about online proctoring and privacy 00:11:53.000 --> 00:11:53.900 ohhh thanks Mathew 00:12:04.000 --> 00:12:04.900 @Laurine exactly! I just came out of a session pushing those πŸ˜‰ 00:12:10.000 --> 00:12:10.900 @Rae largely misplaced 00:12:34.000 --> 00:12:34.900 @rae yes a big concern for us at SWinburne, hence we are all on Canvas with alternative assessments online proctoring privacy issues in Times Higher yesterday (paywall): https://www.timeshighereducation.com/news/eu-lawmakers-call-online-exam-proctoring-privacy-probe?utm_source=THE%20Website%20Users&utm_campaign=ba9d33250c-EMAIL_CAMPAIGN_2020_05_04_12_56&utm_medium=email&utm_term=0_daa7e51487-ba9d33250c-59860025 00:12:39.000 --> 00:12:39.900 there has been concerns raised about invasive proctoring on most states recently. ANU comes to mind a couple of weeks ago. 00:12:57.000 --> 00:12:57.900 thanks caroline! 00:12:59.000 --> 00:12:59.900 We will be conducting our first online assessments from the 25th of May although we have always been a distance learning institutions. My greatest concern in South Africa is connectivity in rural areas 00:13:41.000 --> 00:13:41.900 @Zelna - yes agree - had issues with that when piloting online proctoring in the past 00:13:46.000 --> 00:13:46.900 @it's a convenient dog whistle for raising outrage 00:13:48.000 --> 00:13:48.900 The student experience will be important to capture as we implement mainly online exams at our uni this semester. Agreed Zelna. 00:13:50.000 --> 00:13:50.900 We’re trying to encourage more take-home exams, having already strongly encouraged open-book. 00:14:11.000 --> 00:14:11.900 realistically, how many students actually hand-write by choice? 00:14:19.000 --> 00:14:19.900 UNE seem to have sorted out the privacy issue 00:14:21.000 --> 00:14:21.900 We are also looking at the student experience, it may be they like the alternative assessments better than the written exams face to face. 00:14:40.000 --> 00:14:40.900 Too early to know - we need to evaluate feedback from this round of online exams necessitated by Covid-19 00:14:57.000 --> 00:14:57.900 I agree Paula 00:15:31.000 --> 00:15:31.900 Our Law school have bought in to a take home exam completed over a 24 hour period. If we can keep them happy then that is a win! 00:15:45.000 --> 00:15:45.900 there was interesting research presented at a seminar in Nov 2018 re assessment decisions comparing handwriting and typed... I think that is on the Transforming Assessment site Mathew? 00:16:01.000 --> 00:16:01.900 my students are upset about having to handwrite! 00:16:03.000 --> 00:16:03.900 In our business courses, we are encouraging open book, authentic, time-released exams where possible - where there is also an element of defence or synthesis of a prior assessment. 00:16:07.000 --> 00:16:07.900 sorry I mean type! 00:16:09.000 --> 00:16:09.900 @rae yes the outcome was no difference 00:16:15.000 --> 00:16:15.900 ie would prefer to handwrite 00:16:34.000 --> 00:16:34.900 I was thinking that they may prefer assessment where they don't have to write 00:16:43.000 --> 00:16:43.900 @Alice we have had some students report that too - but many others happier typing and having the ability to edit their work 00:16:58.000 --> 00:16:58.900 I am also waiting with interest to hear the feedback of our students that have equitable assessment arrangements. Hopefully it may be a better experience for them? 00:17:05.000 --> 00:17:05.900 yes Laurine, I thought they would be pleased by the were not- they think they are quicker at handwriting 00:17:07.000 --> 00:17:07.900 @Andreana yes mine too 00:17:08.000 --> 00:17:08.900 It does take our students longer to type answers online then hand-write. Do others experience this? 00:17:13.000 --> 00:17:13.900 There is a difference between running online assessment and running assessment on locked, on-site down computers. I don't think that we're there yet with online assessment so I've relied on timed, open book, take home exams with online submission (via TurnItIn). Seems to have worked as well and is much easier to read than students' handwriting πŸ˜‰ 00:17:14.000 --> 00:17:14.900 @Stuart - agree technology can enable authentic assessment if we use authentic technologies! 00:17:31.000 --> 00:17:31.900 I am so used to be able to move words around ... handwriting would actually be quite a differnt approach 00:17:32.000 --> 00:17:32.900 Many convenors are asking about whether to add some extra time to the online/open book exams to accommodate typing instead of handwriting. 00:17:44.000 --> 00:17:44.900 Yes that was the general feeling - we have given extra time. I trained as an audio typist at 17 so would have been delighted to type my uni exams!! 00:18:05.000 --> 00:18:05.900 our students reported difficulty in typing equations for math and science papers 00:18:10.000 --> 00:18:10.900 @Danielle - what about the 'poor handwriters'?! πŸ˜‰ 00:18:12.000 --> 00:18:12.900 @Alice you would fly through with that experience! 00:18:13.000 --> 00:18:13.900 Is anyone being pushed to have MCQ online not SA or longer text? 00:18:22.000 --> 00:18:22.900 Realising the Potential.. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/791931/DfE-Education_Technology_Strategy.pdf 00:18:31.000 --> 00:18:31.900 As this is our first time, many of our convenors are allowing extra time to manage the new technology and potential connectivity issues 00:18:49.000 --> 00:18:49.900 Interesting @Matthew... I would think that special considerations might have been sought traditionally in that instance. 00:18:53.000 --> 00:18:53.900 thanks, carolien 00:18:54.000 --> 00:18:54.900 I think with handwriting there is a different thought process that we tend to use compared to when we type. By handwriting it sometimes prompts us to remember different components of what is being asked compared to typing. That is usually the feedback I get. 00:18:56.000 --> 00:18:56.900 Connectivity is definitely an issue (and some powercuts) Re authentic, some of our Accounting academics have been using Excel for online exams for years, with authentic problems and varying data to deter copying 00:19:07.000 --> 00:19:07.900 Between now and our assessment date we need to administer a mock exam in the same format/ medium as the formal assessment will be on. This will help our students understand what to expect and we will also learn from it ito of challenges 00:19:18.000 --> 00:19:18.900 We are allowing extra time - there is evidence in the literature that this is necessary. 00:19:20.000 --> 00:19:20.900 @Laurine, many would be reluctant to release carefully crafted MCQ databank for an online exam 00:19:27.000 --> 00:19:27.900 Yes - recognise the discourses 00:19:51.000 --> 00:19:51.900 Yes - we don't routinely ask students for their feedback about handwritten exam experience 00:19:52.000 --> 00:19:52.900 @Laurine - it has to go back to the learning outcomes ...what is being measured... and the advantage of take home exams is the ability to use text matching ... and a more authentic task 00:19:55.000 --> 00:19:55.900 I suppose one question is : when should exam happen? are they at the end of a course or at another point. I suppose my question is what is an exam online? 00:19:58.000 --> 00:19:58.900 @Zelna yes that is very important to do 00:20:00.000 --> 00:20:00.900 Interesting that we still rely on written responses rather than spoken ones. 00:20:06.000 --> 00:20:06.900 My team also runs Special Consideration, we have new criteria for this upcoming assessment period. Our DVC is very anxious about possible technical issues faced by the students 00:20:09.000 --> 00:20:09.900 my youngest had to have a writer for their HSC as they were no longer able to write for a sustained period 00:20:13.000 --> 00:20:13.900 What do we lose if we 'stop writing' it is a different skill, and advanced skill. so much easier to mark a typed exam as opposed to bad handwriting! 00:20:14.000 --> 00:20:14.900 As a pre-sessional EAP teacher, we need to look at alternate forms of assessment. The ability to measure language proficiency with students having any access to online tools is very hard. 00:20:19.000 --> 00:20:19.900 @Mark, true! 00:20:20.000 --> 00:20:20.900 I've used online exams for 4 years. Issue are diagrams and connectivity in rural areas. 00:20:21.000 --> 00:20:21.900 I'd argue that asking students to type instead of handwrite is a more authentic way to assess *hard 00:20:22.000 --> 00:20:22.900 Have you considered the SAMR model to understand the ways technology can change assessment? 00:20:32.000 --> 00:20:32.900 @Laurine Yes, there are still people who want MCQ - mainly for automated marking 00:20:40.000 --> 00:20:40.900 I use the dictation feature on my phone for shopping lists 00:20:49.000 --> 00:20:49.900 Invigilating online exams in the traditional sense is also extremely hard. 00:20:52.000 --> 00:20:52.900 @Gareth: SAMR model? 00:20:55.000 --> 00:20:55.900 @Peter 😌 00:20:56.000 --> 00:20:56.900 but MCQ are so hard to write - at least to make them meaningful 00:21:13.000 --> 00:21:13.900 Absolutely use of Assistive technologies would still have to consider specific students needs obviously. 00:21:17.000 --> 00:21:17.900 We've had recent success in a higher number of people attending online exams against face to face. Be interesting to know why. 00:21:24.000 --> 00:21:24.900 @laurine 😊 00:21:28.000 --> 00:21:28.900 Is there not research that handwriting leads to better recall? 00:21:31.000 --> 00:21:31.900 @Karen, sure would 00:21:39.000 --> 00:21:39.900 we do have a small number of students with disabilities who are allowed to use pen and paper for an exam where the default is online 00:21:47.000 --> 00:21:47.900 but fewer than have concessions to type handwritten exams 00:21:56.000 --> 00:21:56.900 Computer-maked questions can do much more than MCQs. 00:21:57.000 --> 00:21:57.900 interesting Rebecca 00:22:01.000 --> 00:22:01.900 Ours are short answer 00:22:17.000 --> 00:22:17.900 I like to use authentic technologies, where possible - proctoring in small groups is possible e.g. Teams. 00:22:18.000 --> 00:22:18.900 A large percentage of our disability students want and need computers STEM typing formulae/algorithms etc a challenge for many 00:22:38.000 --> 00:22:38.900 @ghizzi there's tech that can help with STEM 00:22:45.000 --> 00:22:45.900 Authentic task design would be much more preferable to traditional test methods. 00:22:46.000 --> 00:22:46.900 Are academics trained in using MCQs and other objective items in many institutions? 00:22:50.000 --> 00:22:50.900 I've been using a combination of type + upload photos of workings We will still allow students to "write" their answers and then at the end of the session upload it, through photographing their answers etc 00:22:51.000 --> 00:22:51.900 Handwriting is authentic for mathematics. 00:22:53.000 --> 00:22:53.900 I am interested in responses when students lose connection during their exam - what are the responses that you have for students and advice for academics? 00:23:02.000 --> 00:23:02.900 I'm getting quite a few colleagues (I'm HoD) of people having trouble with too much looking at screens in the lockdown. Some colleagues feel that they can't mark/read substantial amounts on screen for health reasons. 00:23:13.000 --> 00:23:13.900 SAMR model example https://www.schoology.com/blog/samr-model-practical-guide-edtech-integration 00:23:18.000 --> 00:23:18.900 pen input1 00:23:24.000 --> 00:23:24.900 have we (in the sector) had, sufficiently, the conversation about if we DO need exams? 00:23:45.000 --> 00:23:45.900 the graph / drawing issue comes up a lot 00:23:48.000 --> 00:23:48.900 We are chunking our 2 or 3 hr exams into chunks to reduce continual screen time 00:23:53.000 --> 00:23:53.900 I worry about investing a lot of time/resources in a particular assessment type and/or technology as it makes people less likely to change or drop that technology. MCQs (as a bank and / or Stacks) is a classic example. We don't want them out because we've invested so much in them. 00:23:54.000 --> 00:23:54.900 there's technology that will recognise handwritten STEM equations 00:23:55.000 --> 00:23:55.900 yes they still have to be learned so included in teaching of course as practice 00:24:00.000 --> 00:24:00.900 @Laurine - yes well said! @laurine I don't think so 00:24:20.000 --> 00:24:20.900 @rae.. agree, and does this concern encourage thinking of cheating? 00:24:29.000 --> 00:24:29.900 We have told our students that is they have a technical issue when doing their on line assessment, they need to take a screen shot or picture of the screen with their phone and then apply for Special Consideration through our normal process. Happy to chat off line about this @Rae anewson@swin.edu.au 00:24:34.000 --> 00:24:34.900 @laurine, I agree. This is leading me to review my assessment strategy ( in a good way) but it is challenging (especially in a short time frame) 00:24:49.000 --> 00:24:49.900 @Noel yes we hear this a lot 00:24:57.000 --> 00:24:57.900 @ Laurine, I would favour move to 100% coursework for next semester to avoid online, open book exams. Students and employers would consider this more robust. Andreana - that is similar to the instructions we give students also. 00:25:23.000 --> 00:25:23.900 especially when students have been preparing for a certain format all year - in the UK we've been teaching since September and suddenly at the end have to change assessment 00:25:32.000 --> 00:25:32.900 It is the first time for us, fingers crossed that we are not inundated. 😎 @Rae Tech troubles are one reason for recommending take-home. Years ago we recommended not to choose the settings that a LMS exam must be completed in a single session. It solved lots of disconnect issues as students could just login again and continue. 00:25:36.000 --> 00:25:36.900 @paula, agree! 00:25:38.000 --> 00:25:38.900 @Laurine - yes I am still mentally working through Phil's session last week - rethinking some things that I thought were sound approaches 00:25:46.000 --> 00:25:46.900 @Paula yes the question of summative assessments for next semester looms large! 00:25:50.000 --> 00:25:50.900 But Stuart may persuade us otherwise! 00:26:09.000 --> 00:26:09.900 Online exams require lots of support, and resourcing the implementation is not something I have seen as a priority , Supporting non invigilated exams require specific functionalities and systems which need investment so the success depends a lot on The level of investment 00:26:10.000 --> 00:26:10.900 Yes @Rae, Phill's session with very thought-provoking. 00:26:21.000 --> 00:26:21.900 @Jo - thanks Jo - yes we allow fo reconnection but the timer is still going... if only the LMS (Bb) paused 00:26:50.000 --> 00:26:50.900 @Jo and this reflects how we work in reality. Stop and start, find info, synthesise... 00:26:56.000 --> 00:26:56.900 NL are streets ahead of most places 00:27:17.000 --> 00:27:17.900 @Jo - yes and I think take home open book exams are a good option 00:27:41.000 --> 00:27:41.900 @Roberta, there are many non-invigilated and invigilated assessment designs you can use that are cost effective (or free!) Interactive orals are one. 00:27:47.000 --> 00:27:47.900 @rae, yes 00:27:50.000 --> 00:27:50.900 @Rae We also required manual timing - the invigilator (or subject coordinator) would close the exam, not the system. This allowed for accessibility extra time needs too. 00:27:55.000 --> 00:27:55.900 @Rae Yes, Moodle is a disaster for us running online quizzes which is one reason I go for "take home" type exams 00:28:38.000 --> 00:28:38.900 @ Noel - you can do override give students exta time on timed Moodle quiz. 00:28:42.000 --> 00:28:42.900 @Noel - ah interesting 00:28:52.000 --> 00:28:52.900 but could be a bit fiddly if there is lots of requests. 00:28:53.000 --> 00:28:53.900 I don't think students can look at it dispassionately 00:29:17.000 --> 00:29:17.900 there's so much anxiety/performativity tied up with exams 00:29:26.000 --> 00:29:26.900 thus if a student has a network outage that lasts 15 mins you could give them 15 mins extra using overide in Moodle 00:29:27.000 --> 00:29:27.900 @Noel, I am really tired of people making vague negative statements about Moodle quiz for exams. If there are specific issues, please communicate them. 00:29:53.000 --> 00:29:53.900 Work in the employability area shows that employers and graduates never mention the word assessment, let alone exams, as being front of mind 00:30:08.000 --> 00:30:08.900 @Jo - yes the extra time ability in the LMS - we have used to accommodate students who require extra time - we have also extended time when using ProctorU when they could confirm the loss of connection and the time, so the student could continue once reconnected, adding the additional time 00:30:19.000 --> 00:30:19.900 a little bit digitalised can include 'blended' eg. drawing a graph using pen/paper then taking a pic with a smartphone and submitting to the LMS as part of a take home exam task. 00:30:21.000 --> 00:30:21.900 good piint Laurine 00:30:35.000 --> 00:30:35.900 The e-exam project i undertook also developed a method to build in network outage protection that worked transparently. a student could complete the moodle quiz without any network. 00:30:48.000 --> 00:30:48.900 in part thanks to Tim Hunts prior work 00:30:55.000 --> 00:30:55.900 sounds valuable 00:31:22.000 --> 00:31:22.900 Did you present on that at that seminar in Nov 18 Mathew? the one in Melbourne? 00:31:38.000 --> 00:31:38.900 it did assume on-campus exams using Live USB stuck to provide the client side part of the system. 00:31:50.000 --> 00:31:50.900 Are we assuming that all students have access to a computer? Online schooling, because of Covid-19, is showing the gap between students is widening, mostly because students from poorer families do not have access to the same resources. 00:32:03.000 --> 00:32:03.900 @ Rae - we did talk about it in late 2018. 00:32:09.000 --> 00:32:09.900 good point Carla - and good internet 00:32:17.000 --> 00:32:17.900 there is a paper that describes it on the transformingexams.com site. 00:32:18.000 --> 00:32:18.900 @Danielle sure but not in courses with 100s of students 00:32:33.000 --> 00:32:33.900 @carla yes and even if they have one access may be contested (As we are seeing with families) 00:32:39.000 --> 00:32:39.900 @Carla agree issue of unequal access 00:32:46.000 --> 00:32:46.900 Thank you, Mathew. 00:32:54.000 --> 00:32:54.900 I think that is a major assumption to make that all have computers. I thought they might but 149 students out of 850 students at my son's school had to borrow laptops from the school. So there is an issue around access to equipment in order to do this. 00:32:56.000 --> 00:32:56.900 Andrew Fluck (who Amathew mentioned earlier) at UTAS has utilised eExams on campus 00:32:59.000 --> 00:32:59.900 Hi @Roberta - we have run these at scale 400-500+ the webinar last week looked at this. They are scalable. 00:33:39.000 --> 00:33:39.900 Sorry - meant Mathew not Amathew! @Danielle- yes impressive use of interactive orals as an exam alternative! 00:33:42.000 --> 00:33:42.900 @Carla. Yes, so laptop loans and other equity initiatives have expanded a lot this session - and some students have asked whether they can do their exam on campus because of poor internet (eg in a share house) 00:33:50.000 --> 00:33:50.900 We are using Canvas, has anyone had any issues with load when doing large scale assessments through Canvas? 00:33:59.000 --> 00:33:59.900 Agree @samscott - quite a difference in kit 00:34:44.000 --> 00:34:44.900 @Danielle ok I ll have a look, my institutions is not into oral exams but in some subjects they may be inetrested 00:34:48.000 --> 00:34:48.900 @Andreana large scale use of a system for a particular thing that you have never done before should always be tested before you do it for real! 00:34:51.000 --> 00:34:51.900 Yes sustainable accessible and pragmatic implementations of tech use essential if decide to go that way 00:34:53.000 --> 00:34:53.900 @Stuart - we face the incompatible and disconnected systems problem already @Roberta this resource will talk you through use and design of interactive orals at scale:https://sway.office.com/yQ2s0Bm3ILkWtGll?ref=Link 00:34:54.000 --> 00:34:54.900 Technology changes the role of lecturers since most of them have to rely much more on IT, learning designers, etc. to do their work. 00:35:08.000 --> 00:35:08.900 esp when using proprietary black box assessment systems. 00:35:18.000 --> 00:35:18.900 It would be great if the future had no exams. 00:35:28.000 --> 00:35:28.900 @Tim yes , but in a pandemic world not so much time to prepare. I have been asking questions of our ITS department 00:35:29.000 --> 00:35:29.900 @Jo, yes! 00:35:32.000 --> 00:35:32.900 These experiences relate to a time before the lockdown. Pre-Covid-19 we used PearsonVue online exam centres, very secure and reliable because technology guaranteed cheating not an option. Open-book online exams become a new type of coursework (just shorter time to prepare)... 00:35:44.000 --> 00:35:44.900 Our VC would love a future with no exams! 00:35:56.000 --> 00:35:56.900 Would student / parents who can get their hands on better computers be better off than?? Something to think about ... all over the world 00:36:34.000 --> 00:36:34.900 @Lisa, entrenching inequity? 00:36:35.000 --> 00:36:35.900 @Tim. are you saying that the issues are not real or not Moodle specific? We've had three LMSs in my time and all have similar issues. The problem often revolves around the server getting overwhelmed - slowing down so much that there are long lags between submitting a response and the screen refreshing with the next question and the browser logging them out. I had just over 100 student logging in at the same time just to download an assessment and had requests from 12 who couldn't download and I had to email them instead. 00:37:06.000 --> 00:37:06.900 @Stefan team teaching and learning in program, course, module and assessment design and implementation and practice including diverse staff roles working together 00:37:53.000 --> 00:37:53.900 Who else has had exam imposters? 00:38:01.000 --> 00:38:01.900 @Noel - yes server capacity is a thing. hopefully less as things move to scalable cloud infrastructure. if the IT section is able to pre-order extra capacity on the cloud too. 00:38:04.000 --> 00:38:04.900 I think we can lose sight of the learning goals in the rush to get the process working? 00:38:05.000 --> 00:38:05.900 love this quote - and how often do we learn from exams - often it is hard to get feedback 00:38:27.000 --> 00:38:27.900 @Rae like that 00:38:31.000 --> 00:38:31.900 Vic St Crawl! 00:38:32.000 --> 00:38:32.900 @Rae πŸ˜€ 00:38:42.000 --> 00:38:42.900 Very valid point Noel. I have faced many issues around the same line on Blackboard 00:38:43.000 --> 00:38:43.900 a lovely part of the world. i have visited many years ago! 00:38:55.000 --> 00:38:55.900 love that @Rae πŸ‘ 00:38:59.000 --> 00:38:59.900 Ian Rankin is posting a lot of photos of Edinburgh at the moment 00:39:14.000 --> 00:39:14.900 oh I love Ian Rankin's books 00:39:27.000 --> 00:39:27.900 ☺ 00:39:34.000 --> 00:39:34.900 Look out, he has a new Rebus out in October 00:39:51.000 --> 00:39:51.900 noooooo 00:40:15.000 --> 00:40:15.900 oh that is great to know ! πŸ™‚ 00:40:46.000 --> 00:40:46.900 yes much prefer that kind of authentic practice I love open book exams. With a bit of thought put into the questions/task, you can get some real higher order thinking from the students. 00:41:01.000 --> 00:41:01.900 @Ghizzi Definitely. My experience (as educational designer) is that we are right in the transition from autonomous lecturers to team players (and it's not always smooth). Technology is an extra hurdle. 00:41:02.000 --> 00:41:02.900 if we called it a 'case study assignment' not an exam, might it be less 'scary'' to students? 00:41:04.000 --> 00:41:04.900 @Noel I am not denying that people have problems. Other people do exams in Moodle just fine - but they prepared properly. I am saying "Moodle is a disaster for exams" is not a helpful statement. 00:41:11.000 --> 00:41:11.900 Sounds like an authentic assessment Stuart! 00:41:26.000 --> 00:41:26.900 yes 00:41:27.000 --> 00:41:27.900 I like that idea- they can be prepared and then see how they adjust their thinking 00:41:33.000 --> 00:41:33.900 yes, that's been the style of the final exam for chartered accountancy in England/Scotland/Wales for a number of years 00:41:42.000 --> 00:41:42.900 great idea 00:41:47.000 --> 00:41:47.900 I like the idea of curve balls- some great skill assessment could come from that 00:41:49.000 --> 00:41:49.900 very interesting @laurine for those with specific anxieties definitely 00:41:56.000 --> 00:41:56.900 having done it myself, it's a good test of advanced skills and thinking on your feet 00:41:58.000 --> 00:41:58.900 That would work well in Nursing to see the student's level of critical thinking skills 00:42:09.000 --> 00:42:09.900 curve ball is good.. 00:42:17.000 --> 00:42:17.900 The helpful statement is "Just because you LMS the normal load of students studying, don't assume it will cope with a large synchronous exam" although I still have the occasional nightmare about it (sat mine in 2004!) 00:42:20.000 --> 00:42:20.900 Nice idea - and a real challenge for anyone who has paid a contract cheating co to prepare answers for them 00:42:35.000 --> 00:42:35.900 @Tim sure is right 00:42:36.000 --> 00:42:36.900 When i ran exams in moodle they were very successful - but agree with Tim - you do need to prepare well and cross T's and dot I's.. most of the problems were related to associated logistics, not Moodle itself. 00:42:53.000 --> 00:42:53.900 @Jo, yes indeed! 00:43:11.000 --> 00:43:11.900 @ Noel - any recommended resources for creating open book exams? 00:43:27.000 --> 00:43:27.900 'mess' will always happen! 00:43:35.000 --> 00:43:35.900 that is life! but ye we can learn. 00:43:46.000 --> 00:43:46.900 I promise I won't mention it again but if anyone needs help to see through the 'fog of war' https://www.linkedin.com/in/petergwestcott/ 00:44:10.000 --> 00:44:10.900 thanks Peter! 00:44:12.000 --> 00:44:12.900 events like this - to help capture and share! 00:44:41.000 --> 00:44:41.900 Very helpful to hear what everyone is doing. Thanks 00:44:43.000 --> 00:44:43.900 Also, where people have had 'disasters', and they do happen. what would be helpful is if we could learn from them. If you push the system to breaking point, then where exactly does it break? If we know, that would tell us where to focus our efforts to make it better. and open access publishing of results, tips, case studies etc. 00:44:58.000 --> 00:44:58.900 A great presentation Stuart! Thanks! 00:44:59.000 --> 00:44:59.900 @Allan Really interesting and useful research thank you for sharing with us all 00:45:09.000 --> 00:45:09.900 IESE Business School In Barcelona are very good at this 00:45:11.000 --> 00:45:11.900 A few modules (fully online modules) in our institution have used portfolios instead of sit-down exams for a while. Although I have not used it before it, I understood that it was successful but this is for modules with smaller student numbers. 00:45:16.000 --> 00:45:16.900 Yes @Tim Hunt failures are great to hear about UTS ran a failure workshop recently. 00:45:17.000 --> 00:45:17.900 Really interesting presentation. Gained a few ideas 00:45:28.000 --> 00:45:28.900 Thanks @Stuart 00:45:30.000 --> 00:45:30.900 Relatively small sample perhaps? 00:45:33.000 --> 00:45:33.900 @Tim good point 00:45:35.000 --> 00:45:35.900 Thank you 00:45:36.000 --> 00:45:36.900 Thank you! 00:45:37.000 --> 00:45:37.900 thanks, Stuart Great session 00:45:38.000 --> 00:45:38.900 Thank you 00:45:46.000 --> 00:45:46.900 Thank you! 00:45:51.000 --> 00:45:51.900 thank you 00:45:54.000 --> 00:45:54.900 Interested to hear how other people are implementing 'blended' STEM exams where students enter their answers on one platform and then upload handwritten working on another -- seems clunky but necessary for equations/diagrams? Thank you 00:46:02.000 --> 00:46:02.900 Thank you Thank you 00:46:06.000 --> 00:46:06.900 Are there any examples of significant assessment re-design resulted in better washback on students as well as teachers? Thanks, Stuart! Really enjoyed your talk, especially since I could visualize where you come from... 00:46:07.000 --> 00:46:07.900 @stuart it seems a small sample size? 00:46:08.000 --> 00:46:08.900 Thank you - really interesting session Many thanks Stuart. Very interesting. 00:46:09.000 --> 00:46:09.900 THANK YOU 00:46:16.000 --> 00:46:16.900 Thank you 00:46:17.000 --> 00:46:17.900 thank you 00:46:22.000 --> 00:46:22.900 please fill our feedback survey for the session ! http://taw.fi/feedback 00:46:23.000 --> 00:46:23.900 Have you thought about the likelihood of students having someone with them when they are actually sitting the exams for cheating purposes? How do you protect the integrity of such exam? 00:46:36.000 --> 00:46:36.900 thank you! 00:46:41.000 --> 00:46:41.900 how can we influence policy so we move away from the rigidness of GCSE? 00:46:49.000 --> 00:46:49.900 all of the above! 00:46:59.000 --> 00:46:59.900 Charles William - I have the same question. 00:47:00.000 --> 00:47:00.900 Many thanks for your presentation. It was very interesting πŸ‘ 00:47:01.000 --> 00:47:01.900 The discussion was about tehcincal issues with the system. 00:47:02.000 --> 00:47:02.900 we are often asked how to work the timing of exams ... I am sure there are resources out there but does anyone have a source they recommend? 00:47:05.000 --> 00:47:05.900 Lee-Von, definitely what we have in mind. 00:47:07.000 --> 00:47:07.900 Thank you, food for thought 00:47:11.000 --> 00:47:11.900 even paper exams can break!!! 00:47:25.000 --> 00:47:25.900 Actually, when we tested online exams, the weakest link was the WiFi. 00:47:32.000 --> 00:47:32.900 Thanks Back to my question: Are we assuming that all students have access to a computer? Online schooling, because of Covid-19, is showing the gap between students is widening, mostly because students from poorer families do not have access to the same resources. 00:47:34.000 --> 00:47:34.900 What do you think is the greatest barrier to getting academics to engage in exams to be transformative? 00:47:36.000 --> 00:47:36.900 @Lee-Von one way would be for students to upload mini videos of them working out the equations - we use Canvas and they can do that if we set questions as an Essay questions they then have access to all the tools like video 00:47:48.000 --> 00:47:48.900 We mustn't forget that there can be catastrophe in traditional exams, too. Power cuts, fire alarms, student health emergency, etc. 00:47:50.000 --> 00:47:50.900 if we try we may fail. If we do not try we shall certainly fail! 00:47:51.000 --> 00:47:51.900 Stuart, what do you think students expect? Most students expect conventional exams (MCQs, short answer) - have to re-work those expectations. 00:48:23.000 --> 00:48:23.900 yes definitely need to help students recognise the learning benefits of fails too 00:48:28.000 --> 00:48:28.900 Thank you Matthew and Stuart. 00:48:33.000 --> 00:48:33.900 Even if students can access laptops on campus, they will not have the same dexterity as pupils who have had the same resources since the age of 0. 00:48:35.000 --> 00:48:35.900 I suppos a point is: if possible, don't try this for the first time in a high stakes setting! (This may not be possible right now.) 00:48:41.000 --> 00:48:41.900 @Mathew, then we all reinvent the same broken wheel. 00:48:46.000 --> 00:48:46.900 before you go - please fill our feedback survey for the session ! http://taw.fi/feedback 00:48:49.000 --> 00:48:49.900 @Carla - that is a big issue - as well as access to a quiet space and reliable internet (esp for proctored exams) Stuart, did you have feedback from students who have additional needs, more time , rest breaks etc 00:48:58.000 --> 00:48:58.900 We had a mid sem exam crash because ITS were doing an update. Another had issues because they set up Blackboard in the wrong way (forced completion was on). But many have gone well and the comm with students are so important. 00:49:04.000 --> 00:49:04.900 yes, you'd want to run a formative/low stakes in the same tech so they could practise well ahead of time 00:49:21.000 --> 00:49:21.900 😭 not happening this semester 00:49:29.000 --> 00:49:29.900 Also, if you are using IT infrastructure for something critical, talk to IT first! 00:49:33.000 --> 00:49:33.900 @ Tim is time limiting essential in assessing students? 00:49:37.000 --> 00:49:37.900 On the plus side, Lee-Von, I think STEM has ways to generate similar-but-different questions algorithmically. 00:49:39.000 --> 00:49:39.900 @lice - yes! at least a no-stakes practice run 00:49:49.000 --> 00:49:49.900 students tend to be very risk averse and consequently anxious about change, in my experience. yes, we need to give space for those feelings and support students to effectively prepare for unfamiliar assessment types 00:49:55.000 --> 00:49:55.900 Ghizzi - time limits definitely not essential! 00:50:16.000 --> 00:50:16.900 @Pauline, that has wider applications too! Video of performing a technique etc 00:50:23.000 --> 00:50:23.900 we went about delivering laptop and data card for students who did not have WIFI or a working laptop.... 00:50:25.000 --> 00:50:25.900 Absolutely essential to provide practice - so that means being well ahead in our own preparation 00:50:26.000 --> 00:50:26.900 (Although occasionally time limits, or some time pressure, is authentic.) 00:50:46.000 --> 00:50:46.900 Thanks for the thought provoking presentation 00:50:52.000 --> 00:50:52.900 @May - sounds like a a great support 00:50:59.000 --> 00:50:59.900 Deakin have just moved a couple of schools to online (local and remote) with automated invigilation and it worked well with good student experience 00:51:05.000 --> 00:51:05.900 @Tim definitely would reduce student anxiety and create a more equitable assessment of cohorts learning 00:51:20.000 --> 00:51:20.900 Thank you for the presentation and the lively chat everyone 00:51:28.000 --> 00:51:28.900 Thank you for your time Stuart, very informative. 00:51:33.000 --> 00:51:33.900 We all worry too much 00:51:39.000 --> 00:51:39.900 thank you 00:51:54.000 --> 00:51:54.900 Also consideration for students whose homes are not conducive for online assessment, special permission were sought for them if they want / prefer to come into university for exam, very small handful of them 00:51:56.000 --> 00:51:56.900 in the cases where online exams have been used, do you know if the responses were checked using a plagiarism tool, or any other form of forensic analysis? 00:52:02.000 --> 00:52:02.900 Before you all head off - please fill our feedback survey for the session ! http://taw.fi/feedback (i will pass your anonymous feedback to the presenter too!) 00:52:08.000 --> 00:52:08.900 Thank you for the thought provoking presentation. 00:52:16.000 --> 00:52:16.900 @May yes definitely 00:52:39.000 --> 00:52:39.900 pre-release is authentic; how often are people locked in a room and made to solve a workplace problem, and how happy would the employer be with the outcome? Is 50% OK ? 00:52:49.000 --> 00:52:49.900 COVID19 00:53:06.000 --> 00:53:06.900 There's a big difference between students enrolling (deliberately) in an online course vs the current situation where students are plunged into online and working from their own bedroom. 00:53:18.000 --> 00:53:18.900 We asked academics to set up a practice but some didn't sadly... 00:53:28.000 --> 00:53:28.900 @stefan yes 00:53:35.000 --> 00:53:35.900 @ Charles Williams In another webinar on online assessment it was suggested that you use essay questions because those are more difficult to cheat. In my assessment I only ask application questions based on case studies (I developed 3 case studies on Covid-19 as I think all students will have "experience" on it). I also make it clear that No marks will be awarded for repetition of theory and this is similar to my formative assessment during semesters 00:53:39.000 --> 00:53:39.900 yes absolutely has been same for teaching staff to use new ptech and procedures 00:53:42.000 --> 00:53:42.900 @stefan nobody's chosen the current situation 00:53:47.000 --> 00:53:47.900 I find it amazing that they cant imagine things from student perspective 00:53:49.000 --> 00:53:49.900 @Stefan, Exactly and that relates to online assessment pre- and post-covid 00:53:54.000 --> 00:53:54.900 The practice session is very important and it does help determine whether the timing is accurate and the level of questions appropriate (not too easy or hard). 00:53:56.000 --> 00:53:56.900 Thanks Stuart, I would be interested to hear more examples of what kinds of innovations/creative assessments are possible within an online exams format when we avoid just migrating them. The participants/scholars quoted in the study alluded to this (receding) possibility, but it would be great hear more about what a transformed exam would look like. 00:54:07.000 --> 00:54:07.900 @Alice ☹ 00:54:34.000 --> 00:54:34.900 @Celina - we use ProctorU for exams that are invigilated online - they have keystroke analysis but I have to say I haven't seen where this has flagged issues - perhaps as the verification process is fairly full on 00:54:35.000 --> 00:54:35.900 @Zelna - sounds like great assessment 00:54:36.000 --> 00:54:36.900 have you Stuart, or anyone else, explored whitelisting resources so that students can access allowed online materials in secure online exams? 00:54:39.000 --> 00:54:39.900 certainly hope so 00:54:48.000 --> 00:54:48.900 @Zelna sounds good 00:54:49.000 --> 00:54:49.900 I am actually inspired creatively about writing new question styles. 00:54:57.000 --> 00:54:57.900 @rebecca yes it's widely used 00:55:03.000 --> 00:55:03.900 how?/ 00:55:04.000 --> 00:55:04.900 We've had one small pilot so far 00:55:07.000 --> 00:55:07.900 thank you 00:55:14.000 --> 00:55:14.900 @Rebecca, yes we also do that 00:55:23.000 --> 00:55:23.900 Thank you for this presentation. 00:55:38.000 --> 00:55:38.900 Are there any resources you can recommend for online assessment of computer literacy skills? - ie. testing a student's technical proficiencies in Microsoft Word/Excel/PowerPoint 00:55:42.000 --> 00:55:42.900 can anyone say an online proctoring software that is not network dependent? 00:55:50.000 --> 00:55:50.900 πŸ‘ 00:55:52.000 --> 00:55:52.900 can anyone share an online proctoring software that is not network dependent? 00:55:54.000 --> 00:55:54.900 @Pauline, hadn't considered video, thanks for suggesting. Another issue it raises is around the status of the video - is it just a security check? Will the video be 'marked' in any way? 00:56:01.000 --> 00:56:01.900 @rebecca We did something similar to make software available We have as assessment desktop which is deployed to clusters for online exams. 00:56:15.000 --> 00:56:15.900 I did biochem, we were allowed models so there's a bit of an open book tradition I think 00:56:38.000 --> 00:56:38.900 @Mathew YES!!!!!!! 00:56:51.000 --> 00:56:51.900 application, I often talk about 'shop windows' with my students - they are studying accounting 00:56:59.000 --> 00:56:59.900 @Diane that's interesting, something we haven't been able to explore yet but there's definitely demand. Think whitelisting (esp of applications/software) really opens up opportunities for more authentic assessment 00:57:00.000 --> 00:57:00.900 I agree Matthew - with good identity security. 00:57:08.000 --> 00:57:08.900 yes definitely much more authentic 00:57:09.000 --> 00:57:09.900 Yes they need to apply the learning in tests, make case studies, problem solving. Have the internet open like they would in real life 00:57:14.000 --> 00:57:14.900 exam as shop window of the skills Matthew is talking about 00:57:22.000 --> 00:57:22.900 we sending anatomical models (by mail) to podiatry students to base online responses for case studies as cant use real people in covid much more authentic to accounting practice 00:57:24.000 --> 00:57:24.900 @Pauline yes 00:57:36.000 --> 00:57:36.900 comes down to better design of assessment and considering purpose 00:57:42.000 --> 00:57:42.900 the main problem with online exams is not the tools it's the users 00:57:44.000 --> 00:57:44.900 Has anyone seen examples of assessing in VR? I agree - and the option of vivas afterwards if you suspect cheating 00:57:46.000 --> 00:57:46.900 Before you all head off - please fill our feedback survey for the session ! http://taw.fi/feedback I like the two responses that COVID-19 might generate - "temporary blip or permanent flip?" 00:57:57.000 --> 00:57:57.900 😌 00:58:02.000 --> 00:58:02.900 @carolyn permanent blip 00:58:09.000 --> 00:58:09.900 @Carolyn, me too 00:58:11.000 --> 00:58:11.900 @rebecca - but presumes on-campus, ie control of desktop? 00:58:15.000 --> 00:58:15.900 @bjblanch - wow that is amazing - has to be a logistical challenge! 00:58:18.000 --> 00:58:18.900 This style reduces plagiarism as you would see identical answers more easily, you can also give a handy hint for the assessment in every class, if you don't see them in the end case-study response you are alerted to the fact someone else may have done it 00:58:26.000 --> 00:58:26.900 Many thanks 00:58:31.000 --> 00:58:31.900 @Diane yes, new challenges now! 00:58:38.000 --> 00:58:38.900 thanks Stuart and Matthew 00:58:49.000 --> 00:58:49.900 small numbers Before you all head off - please fill our feedback survey for the session ! http://taw.fi/feedback 00:58:50.000 --> 00:58:50.900 One of the things we were talking about yesterday is getting academics to think about students creating items in their exams rather than simply respond in online exams. Anyone else doing this? Creating videos etc. 00:58:57.000 --> 00:58:57.900 Sorry I have to head off. thank you Stuart, lots of food for thought. Agree that it will be very interesting to explore what the current period shows us, once the smoke clears a little 00:59:02.000 --> 00:59:02.900 Thank you for the session. thanks, Stuart and Mathew 00:59:07.000 --> 00:59:07.900 Thanks Stuart and Mathew 00:59:10.000 --> 00:59:10.900 Thanks Matthew and Stuart Thank you for the insight into your thought provoking work. 00:59:11.000 --> 00:59:11.900 πŸ‘ 00:59:14.000 --> 00:59:14.900 Very happy to continue the conversation on Twitter too: https://twitter.com/OpenPlanStuart 00:59:20.000 --> 00:59:20.900 & thank you everyone for very interesting chat too 00:59:21.000 --> 00:59:21.900 Thanks stuart 00:59:22.000 --> 00:59:22.900 πŸ‘ πŸ‘ 00:59:23.000 --> 00:59:23.900 thank you for your time. Thank you! Thanks Mathew and Stuart! 00:59:25.000 --> 00:59:25.900 we trailing assessing nursing skills by submission of video Thanks Stuart and Mathew, and all chat participants. V interesting πŸ™‚ 00:59:26.000 --> 00:59:26.900 Thanks for the session, really useful 00:59:29.000 --> 00:59:29.900 Thanks Stuart and Mathew πŸ™‚ Before you all head off - please fill our feedback survey for the session ! http://taw.fi/feedback 00:59:32.000 --> 00:59:32.900 thank you 00:59:33.000 --> 00:59:33.900 thank you! 00:59:35.000 --> 00:59:35.900 Thank you. Great session 00:59:39.000 --> 00:59:39.900 πŸ‘ πŸ‘ πŸ‘ 00:59:42.000 --> 00:59:42.900 Thanks all another great session 00:59:48.000 --> 00:59:48.900 I had 22 years work experience outside academia and was frustrated to death with graduates that could not apply the "knowledge" (not theory) they accumulated. I love the idea of open internet seeing what sources students access during exams 00:59:51.000 --> 00:59:51.900 Many thanks, Stuart and Matthew and all participants. 01:00:00.000 --> 01:00:00.900 I'm interested in connecting with anyone who has had experience with online assessments for computer literacy - marilyn@lisof.co.za 01:00:04.000 --> 01:00:04.900 great session - many thanks! 01:00:10.000 --> 01:00:10.900 thank you Stuart 01:00:12.000 --> 01:00:12.900 valuable comment Zelna 01:00:16.000 --> 01:00:16.900 We used open internet in an exam where students were asked in a Sport Facilities Masters course to determine a location for a new wave pool for the surfing events at the Tokyo 2020 Olympics due to lackluster surf. Students had to research a location and solution for logistics and then compose a lobbying letter as their exam submission in the role of a consultant for a major international surfing organisation. Creative and a high level of academic integrity. 01:00:17.000 --> 01:00:17.900 stu.allan76@gmail.com 01:00:31.000 --> 01:00:31.900 Before you all head off - please fill our feedback survey for the session ! http://taw.fi/feedback 01:00:34.000 --> 01:00:34.900 Thanks Stuart and Mathew, great session 01:00:37.000 --> 01:00:37.900 great Danielle! 01:00:44.000 --> 01:00:44.900 thanks so much Stuart and Mathew