5:01 PM Hi I'm from Vancouver Ken Howah (CQU) 5:01 PM Hi, Ken Howah, I'm from CQU Melbourne campus Karen Dunn-Haley 5:01 PM Hi - I'm from California. Trevor Meek U of Bradford 5:01 PM Hi I'm from Bradford , UK Khameel Swinburne 5:01 PM Hi all, Khameel from Swinburne (Malaysia Campus) Mathew Hillier 5:01 PM http://www.uq.edu.au Iris Nube (IB) 5:01 PM Hi, I'm from The Hague in the Netherlands Geraldine #2 5:02 PM From Karratha North West Western Australia (Iron Ore country) Martin@mcqs.com 5:02 PM Hi , I'm martin from ireland Trevor Meek U of Bradford 5:02 PM Hi Khameel - please keep in contact t.meek@bradford.ac.uk as I would like to make contact re collaborative work in Malaysia... thanks. Jeffery 5:02 PM msia? Khameel Swinburne 5:03 PM Alright @ Trevor Jeffery 5:03 PM Hi, i'm from sunny Singapore! Sami 5:03 PM Hi, I'm from the UAE Tim Hunt 5:04 PM Hi, I'm from Milton Keynes in the UK. Thank you for staying up to talk to us! 5:04 PM Mathew Hillier 5:05 PM esp high stakes assessment! Tim Hunt 5:07 PM LOL Khameel Swinburne 5:07 PM Good one. Mathew Hillier 5:11 PM impressive! ;-) 5:11 PM Ghizzi 5:12 PM I think we all have a bit of rant about it occasionally!;) Khameel Swinburne 5:12 PM B Di Regan-Roots 1 5:12 PM b Trevor Meek U of Bradford 5:12 PM B Denis Duret 5:12 PM B Hele 5:12 PM b MARTINSCHRANZ 5:12 PM A Mathew Hillier 5:13 PM hi geoff ;-) Geoff Crisp 5:14 PM hi mathew, finally got here Ghizzi 5:15 PM absolutely Sami 5:15 PM That's a wonderful point with your interaction with faculty Mathew Hillier 5:16 PM wondering how video goes over wireless when all are downloading at once. ok 5:17 PM Tim Hunt 5:18 PM (Pity the answer boxes in this DB question cant be in-place in the image.) Mathew Hillier 5:18 PM Tim - limits of blackboard ;-) Ghizzi 5:19 PM haha mine too Mathew Hillier 5:20 PM great . getting students to construct responses rather then just recall :-D most welcome :-) 5:21 PM Tim Hunt 5:22 PM Can you quickly tell us verbally about some of the other q types? Mathew Hillier 5:22 PM this approach is what Geoff and I have been doing with the TA website examples Geoff Crisp 5:22 PM great to see possibilities for divergent responses rather than just convergent responses Trevor Meek U of Bradford 5:22 PM Rob, is there any error checking that all fields are completed prior to moving to the next question or is this left just to students as per instruction? Tim Hunt 5:22 PM Fair enough Thanks. Mathew Hillier 5:24 PM good ol java Ghizzi 5:25 PM hehehe Tim Hunt 5:26 PM Not locking down is only OK as long as all students are doing the exam on computer. If you want it to be a choice, then computer and paper have to be equivalent. Hence you need some sort of lock-down. :-( Mathew Hillier 5:27 PM did you provide a choice in the early days? - danny joined the Main Room. ( 5:27 PM ) - Mathew Hillier 5:28 PM invigilators here are staff members or contractors doing the job. - Nicky joined the Main Room. ( 5:29 PM ) - Trevor Meek U of Bradford 5:29 PM We have found the same that dedicated e-assessment invigilators are a good option. Jeffery 5:29 PM hmm dedicated e-assessment invigilators? Rob Peregoodoff 5:30 PM trusite Trevor Meek U of Bradford 5:31 PM Yes, they are a group that trained to be aware of the types of mal practice that can occur and also in the set-up and maintenance of the exam and its security. Mathew Hillier 5:31 PM what about mobile phone internet access via 3G network (not via lan or wireless). Alexander [FU Berlin] 5:31 PM sorry for being late. good morning :-) Mathew Hillier 5:31 PM hi alex Alexander [FU Berlin] 5:32 PM hey mathew Tobias Halbherr 5:32 PM are your exams high-stakes - i.e. do you have significant drop-out rates in your exams? Iris Nube (IB) 5:33 PM Do you check the key-loggers for all students, or only when there is suspicion? Would it be possible to check all students automatically? Laurine 5:34 PM do they have a practice of this format before the actual exam? Tobias Halbherr 5:34 PM How do you prevent ghostwriting if you give full access to Internet in exams? Geoff Crisp 5:34 PM that is what assessment should be like Rob - well done Ghizzi 5:34 PM yes agree MARTINSCHRANZ 5:35 PM what about an online exam done outside the classroom, how would you invigilate? Tim Hunt 5:36 PM Getting rid of invigilation is a completely different question, really, to replacing paper with typing. In my opinion. 5:36 PM Ghizzi 5:37 PM interesting yes improve quality of question designs 5:38 PM Alexander [FU Berlin] 5:38 PM absolutely! Ghizzi 5:38 PM Really struggle to get academic staff to focus on that sometimes Tim Hunt 5:39 PM I am failing to find trusite on Google. At the end of this talk, would you be able to give us a URL? Thanks. THanks. 5:39 PM Ghizzi 5:39 PM Fab would love to see that level of analytics on our assessments Iris Nube (IB) 5:40 PM Do you check the trusite key-loggers for all students, or only when there is suspicion? Would it be possible to check all students automatically? Ghizzi 5:41 PM love that title :) Tobias Halbherr 5:41 PM how many trained invigilators do you have/need, how many online-exams do you perform per semester MARTINSCHRANZ 5:41 PM what about the students privacy rights Iris Nube (IB) 5:41 PM thank you! Ken Howah (CQU) 5:43 PM wow Alexander [FU Berlin] 5:43 PM o.O Ghizzi 5:43 PM hehehe yup Tim Hunt 5:43 PM (I would not recommend that.) Well, OK if you had tested it. Ghizzi 5:44 PM a bit of guerrilla action could be fun ;) Mathew Hillier 5:45 PM good solid servers needed Ghizzi 5:45 PM oh dear wow could use that data hmmm 5:46 PM hehe 5:46 PM - Lesley Hamilton (UWS) joined the Main Room. ( 5:46 PM ) - Mathew Hillier 5:47 PM great use of the $! Ghizzi 5:47 PM oh I like that idea yes that's the problem 5:47 PM Laurine 5:47 PM do students have a practice using this format prior to the actual exam? Mathew Hillier 5:47 PM do you use e-feedback from e-marked exams? Trevor Meek U of Bradford 5:47 PM excellent idea for feedback. Laurine 5:48 PM great, that must reduce anxiety +++ Ghizzi 5:48 PM yes absolutely Mathew Hillier 5:48 PM agree with running the practice exams ok 5:48 PM :-) 5:48 PM Mathew Hillier 5:51 PM the 30 minute ramp up is an important tip here! we have maxed out the exam spaces on campus so scheduling is very tight already 5:51 PM Ghizzi 5:51 PM can you paste in links please? could you not leave extra 5 mins but not tell them? 5:52 PM Mathew Hillier 5:53 PM i noticed students handwriting little reminders to themselves on scrap paper in the e-exams i have ran too Ghizzi 5:54 PM yes pI would always want paper for notes, planning etc Tim Hunt 5:54 PM I don't mind if you overrun a bit. This is all great. MARTINSCHRANZ 5:54 PM future Ghizzi 5:54 PM Rant Mathew Hillier 5:54 PM b im a masochist :-P Di Regan-Roots 1 5:54 PM b Laurine 5:54 PM yeah me too, Mathew Rob Peregoodoff 5:55 PM True site Rob Peregoodoff 5:56 PM reddit caching Mathew Hillier 5:56 PM wireless is a single point of failure. i guess a risk there. wouldn’t risk this on our dodgy network :-P 5:56 PM Ken Howah (CQU) 5:56 PM you really need those backups incl the spare PC's room etc 5:57 PM Mathew Hillier 5:58 PM lol Mathew Hillier 5:58 PM good story! Tobias Halbherr 5:58 PM redundancy very important in online-exams Geoff Crisp 5:58 PM great presentation on how this can be done well Laurine 5:59 PM thanks, Rob! good to look to the future..... Trevor Meek U of Bradford 5:59 PM Did you try any other caching system Rob or did you stick with Reddit? Iris Nube (IB) 5:59 PM Thank you very much for this wonderful presentation! Tobias Halbherr 5:59 PM very interesting to see, how you organize your online-exams Tim Hunt 5:59 PM By the way, interesting doc: the requirements for networking at IETF meetings: https://iaoc.ietf.org/ietf-network-requirements.html Ghizzi 5:59 PM yes second that fab Tim Hunt 5:59 PM That was a great talk. Ken Howah (CQU) 5:59 PM many thanks Rob Alexander [FU Berlin] 5:59 PM Very interesting. Your overall workflow reminds me a lot of ours here in Berlin. Thanks a lot ! Tim Hunt 5:59 PM And, I still want to hear the rant :-) Geraldine #2 5:59 PM fab, this was great, very thought provoking Tobias Halbherr 5:59 PM thank you, goodbye :) Ghizzi 5:59 PM me too was looking forward to that Trevor Meek U of Bradford 6:00 PM thanks.. and lowly No! Nicky 6:00 PM thanks bye! Tim Hunt 6:00 PM So, you use your normal blackboard system for Exams, you don't have a separate install? Di Regan-Roots 1 6:00 PM really interesting thanks Tim Hunt 6:00 PM But just one instance? Tim Hunt 6:00 PM OK Tim Hunt 6:00 PM But then, all the hardware is available for the exams. Interesting trade-off. 6:01 PM Ghizzi 6:01 PM yes Tim Hunt 6:01 PM Yes. You need to stress-test. Trevor Meek U of Bradford 6:01 PM Can you reiterate that point re concurrent access numbers - what is the max you have managed? Tim Hunt 6:01 PM Can't you use JMeter or something for that sort of testing? Yes, please talk to your IT department. The are your friends. (Guess which department I work in) 6:02 PM Alexander [FU Berlin] 6:03 PM we used blackboard for exams in 2009 and the database just locked after 120 students tried to access the questions. did you optimize anything in blackboard? Tim Hunt 6:03 PM JMeter is a tool for simulating 100s of users hitting your system at the same time. https://jmeter.apache.org (tThere are other similar tools.) Denis Duret 6:03 PM Have you been asked to have EMI questions in a test with Blackboard? Trevor Meek U of Bradford 6:03 PM Thanks. Good idea re Fri eve's. Alexander [FU Berlin] 6:03 PM ok Denis Duret 6:04 PM Extended Matched Item (I think) Alexander [FU Berlin] 6:04 PM hahaha :-) Denis Duret 6:04 PM Popular with the medics Geoff Crisp 6:04 PM EMI common in medical and health science questions Denis Duret 6:05 PM It isn't really an option with Blackboard. We have used Jumbled Sentence but not ideal. Alexander [FU Berlin] 6:05 PM so I came a bit late. just to make sure: the answers are getting uploaded from the browser AFTER the student clicks a save button? Tim Hunt 6:05 PM Our experience was that the server-capacity was not an issue. Lockdown browser was a small issue for some. Reliable wifi was the biggest problem. Affected about 10% of students. Ghizzi 6:06 PM yes we'd have same problem, plus issues with our servers too :( Tim Hunt 6:06 PM But, the problem for us at the OU is that they are all distance students. Therefore, we have to hire exam halls all around the country, and so the WiFi is supplied by a 3rd party. :-( Ghizzi 6:06 PM oh yes can see how that would be a problem Ken Howah (CQU) 6:07 PM Same at CQU...we have exam sites around the WORLD Tim Hunt 6:08 PM The problem with laptop power is not just battery life or number of sockets. A laptop charger can be 70W. 70W * 600 students = 40kW. Can you power cables cope with that? Ken Howah (CQU) 6:09 PM good point about the total kW Ghizzi 6:10 PM hehe I think ours would probably blow! old old buildings Ken Howah (CQU) 6:11 PM perhaps this model is not ready for multi-site uni's Ghizzi 6:11 PM we FutureLearn MOOCs Tim Hunt 6:11 PM On the other hand, multisite might be easier for WiFi than 600 people in one hall. Ghizzi 6:12 PM lost sound Iris Nube (IB) 6:12 PM It might be possible for multi-site universities, if you change the content of the assessment. I think Rob talked about a 3-hour exam over 24-hour period which was as reliable as the lock-down exams were? Ghizzi 6:12 PM oops back Ken Howah (CQU) 6:12 PM A big problem would be ensuring infrastructure consistency from mini site to mini site Iris Nube (IB) 6:12 PM Thank you Trevor Meek U of Bradford 6:13 PM Not heard of in UK but is a good idea. Tim Hunt 6:13 PM Trevor, there are the people who provide access to the UK driving theory test. I think they were bought by Pearson, so it is not Pearson, I think. 6:14 PM Thanks for a great talk. 6:14 PM Trevor Meek U of Bradford 6:14 PM Thanks Tim.. I will try to find. Geraldine #2 6:14 PM Again thank you very interesting Geoff Crisp 6:14 PM thank you for staying up so late for us Rob, much appreciated Tim Hunt 6:14 PM http://www.learndirect.com Ken Howah (CQU) 6:15 PM Thanks again Rob...will keep in mind about your interest in doing a multi-site project one day, if we get that far! Ghizzi 6:15 PM Ooh very thank you much food for thought thank you:) Ken Howah (CQU) 6:15 PM Been great, good night! Ghizzi 6:15 PM sleep well! ;) Trevor Meek U of Bradford 6:16 PM Thanks - great session. Rob Peregoodoff 6:16 PM Good bye all Alexander [FU Berlin] 6:16 PM bye!!! and thanks!!! Geoff Crisp 6:16 PM Thanks Mathew Ghizzi 6:16 PM bye all c u tomoz danny 6:16 PM Thanks!