WEBVTT

00:00:24.000 --> 00:00:24.900
<v Harvey Woolf>
Hi from an overcast Dudley in the West Midlands.

00:05:56.000 --> 00:05:56.900
<v Priya Dutta>
Many congratulations, Peter!

00:09:17.000 --> 00:09:17.900
<v Tim Hunt (OU)>
Was there an issue coming up with a story that worked for students from many different cultures?

00:11:09.000 --> 00:11:09.900
<v Tim Hunt (OU)>
For those who don't know it: https://en.wikipedia.org/wiki/Common_European_Framework_of_Reference_for_Languages

00:11:53.000 --> 00:11:53.900
<v Mathew Hillier>
Thanks Tim 🙂 BTW questions will likely be addressed in the Q&A at the end.

00:16:17.000 --> 00:16:17.900
<v Mathew Hillier>
For those interested - there is a 60 sec demo video that you may want to watch after the session. https://youtu.be/-Ah5Pq7xzHE

00:17:17.000 --> 00:17:17.900
<v Jenny James - QUT>
👏 

00:18:44.000 --> 00:18:44.900
<v Rachel Forsyth Lund Uni>
looks great, @Peter Cormack  - I like the student-centred approach which isn't always foregrounded as much as you describe.

00:19:15.000 --> 00:19:15.900
<v Priya Dutta>
Many congratulations to John and the whole team!

00:26:06.000 --> 00:26:06.900
<v Tim Hunt (OU)>
I'm a Moodle developer. I want to know how you(r developers) chose to implement some of this technically, but that is a different talk.

00:26:59.000 --> 00:26:59.900
<v Mathew Hillier>
Similarly curious if the moodle plugin used is available for others to use? (if it is open source?). 

00:29:00.000 --> 00:29:00.900
<v Tim Hunt (OU)>
Interesting that the AI is being referred to as 'Tutor'. (Minor ethical question there ;-))

00:32:07.000 --> 00:32:07.900
<v Rachel Forsyth Lund Uni>
This is not a comment on this project, but a general one: we need a course on the use of AI assessment including ethical concerns. Is it a slippery slope from use in formative to summative assessment? Or is everyone OK with fully automated grading and feedback? 

00:32:47.000 --> 00:32:47.900
<v Tim Hunt (OU)>
Fully automated grading has been a thing for decades (right back to multiple choice on paper).

00:32:59.000 --> 00:32:59.900
<v Tim Hunt (OU)>
All that has changes is the range of things that can now be graded automatically.

00:33:13.000 --> 00:33:13.900
<v Rachel Forsyth Lund Uni>
yes, I know. But not for complex tasks. 

00:33:44.000 --> 00:33:44.900
<v Rachel Forsyth Lund Uni>
I mean where marking is usually needed

00:33:57.000 --> 00:33:57.900
<v Rachel Forsyth Lund Uni>
Argh where HOLISTIC marking is usually needed

00:34:54.000 --> 00:34:54.900
<v Tim Hunt (OU)>
Yes, and there is a big difference between "AI can replicate the human grades on these assessments submitted by students in the past" and "It is impossible to game the system for a higher score if you know this is going to be AI-marked".

00:36:25.000 --> 00:36:25.900
<v Matt Townsend - Cardiff University>
We're being approached by a few firms offering AI marking. Some staff jumping at the prospect without thinking of the ethics/student kick back. 

00:37:05.000 --> 00:37:05.900
<v Paul Oliveri #2>
It would be interesting to see how the AI feedback satisfaction score compared with the human feedback satisfaction score. Students don't find our feedback five star in many cases.

00:37:14.000 --> 00:37:14.900
<v Rachel Forsyth Lund Uni>
Me too - I have a lot of concerns and blogged about this - again, no reflection on the current project! - I just want us to talk about it more. https://assessmentinhe.wordpress.com/2024/05/26/should-i-use-ai-to-grade-assignments/

00:38:04.000 --> 00:38:04.900
<v Rachel Forsyth Lund Uni>
Sorry I was replying to Matt. @Paul Oliveri #2 absolutely, I think GenAI feedback statements can be really good - if selected by the tutor. Am I tying myself in knots here?

00:38:30.000 --> 00:38:30.900
<v Tim Hunt (OU)>
AI feedback can score much better on one dimension which makes a big difference to the effectiveness of feedback to learning: AI can give the feedback in minutes, while human feedback often only arrives days or weeks later. (But, that is just one dimension.)

00:39:31.000 --> 00:39:31.900
<v Paul Oliveri #2>
@Rachel Forsyth Lund Uni I think that's just our point in the Gen AI journey...tying ourselves in knots.

00:39:46.000 --> 00:39:46.900
<v Tim Hunt (OU)>
My question was really about different cultures all around the world where the British Council operates.

00:40:16.000 --> 00:40:16.900
<v Lai>
I often encounter discussions between rubrics (detailed, structured marking system) vs. holistic marking. I recognize the arguments and counter-arguments for both. In an AI grading world, I wonder whether building a detailed rubric is a precondition for communicating with AI to illustrate the assessment criteria and standards. of course, for complex tasks, sometimes rubric alone is also insufficient. maybe concrete examples of different levels of performance are also needed...

00:40:18.000 --> 00:40:18.900
<v Rachel Forsyth Lund Uni>
Also I have played with generating feedback statements which are feedforward, inclusive, encouraging, kind, etc and it is excellent for that (will add that to the blog soon) - sometimes it's hard to remember all that while grading at the same time, so a bank of really good statements is handy

00:40:48.000 --> 00:40:48.900
<v Tim Hunt (OU)>
Thanks

00:41:06.000 --> 00:41:06.900
<v Rachel Forsyth Lund Uni>
GenaI also good for generating complex holistic rubrics...

00:42:26.000 --> 00:42:26.900
<v Paul Oliveri #2>
Question for John. How did the AI feedback satisfaction score compare with previous human feedback in those courses/units

00:44:56.000 --> 00:44:56.900
<v Priya Dutta>
I think the "human in the loop" model is really powerful.

00:47:20.000 --> 00:47:20.900
<v Priya Dutta>
https://www.e-assessment.com/special-interest-groups/?_gl=1*1p2arkr*_up*MQ..*_ga*NjYyNjk0MDU4LjE3MjA1OTIxMjE.*_ga_M81C2LFSXK*MTcyMDU5MjEyMC4xLjAuMTcyMDU5MjEyMC4wLjAuMA..

00:47:46.000 --> 00:47:46.900
<v Lai>
sorry the link does not work

00:47:49.000 --> 00:47:49.900
<v Tim Hunt (OU)>
Link so cryptic that Bb Collaborate does not recognise it.

00:48:08.000 --> 00:48:08.900
<v Rachel Forsyth Lund Uni>
thanks, @Priya Dutta ! 

00:48:21.000 --> 00:48:21.900
<v Rachel Forsyth Lund Uni>
@Tim needs AI parsing (or something)

00:48:29.000 --> 00:48:29.900
<v Tim Hunt (OU)>
Cleaner link https://www.e-assessment.com/special-interest-groups/ works

00:48:35.000 --> 00:48:35.900
<v Priya Dutta>
Here it is again: https://www.e-assessment.com/special-interest-groups/

00:50:39.000 --> 00:50:39.900
<v Paul Oliveri #2>
Definitely Matt

00:52:32.000 --> 00:52:32.900
<v Tim Hunt (OU)>
Thanks. Interesting session.

00:52:41.000 --> 00:52:41.900
<v Rachel Forsyth Lund Uni>
Thanks very much @Peter Cormack and @John #2 

00:52:42.000 --> 00:52:42.900
<v Tim Hunt (OU)>
https://www.e-assessment.com/events/2024/10/beyond-multiple-choice-2024/ - Online, if I read that right.

00:53:10.000 --> 00:53:10.900
<v Lai>
I just wonder whether there is a chance to learn from the two cases concerning the technical aspects, such as how to implement similar AI applications in a Moodle environment (e.g. prompts for generate better AI -feedback to a specific assignment)

00:53:19.000 --> 00:53:19.900
<v Adam Taylor>
Link for registering for the AI SIG webinar available soon. Please check your member emails or follow our LinkedIn or X pages. Register your interest for BMC here: https://www.beyond-multiple-choice.app/

00:53:28.000 --> 00:53:28.900
<v Divya Srivastava>
Thanks for the interesting session

00:53:30.000 --> 00:53:30.900
<v Jenny James - QUT>
Thanks so much

00:53:32.000 --> 00:53:32.900
<v Matt Townsend - Cardiff University>
Cheers all.

00:53:43.000 --> 00:53:43.900
<v John #2>
Thanks all!

00:53:51.000 --> 00:53:51.900
<v Paul Oliveri #2>
Thanks everyone