WEBVTT 00:01:09.000 --> 00:01:09.900 Welcome to the session today. Today I am at home in eastern Sydney in Australia on lands of the Bidjigal and Gadigal peoples of the Eora nation. Looking forward to your discussions today! 00:08:32.000 --> 00:08:32.900 Primary students are using GenAI?? 00:08:37.000 --> 00:08:37.900 I'm so out of touch! 00:09:43.000 --> 00:09:43.900 Wouldn't it be possible to detect AI generated text by cross referencing it to piece of work known to have been written by the student? 00:10:07.000 --> 00:10:07.900 It takes a village to raise an assessment.... 00:10:58.000 --> 00:10:58.900 @Martin McMorrow University of Notre Dame Australia - some us have very large units with +700 students... 00:11:48.000 --> 00:11:48.900 Yes, good point, but I'm thinking of automated programmes which compare texts - similar to some of the programmatic assessment approaches highlighted in the last session 00:11:48.000 --> 00:11:48.900 Support the idea of developing AI literacy in both staff and students.! comments? 00:11:57.000 --> 00:11:57.900 😩 00:12:23.000 --> 00:12:23.900 A lot of parents are using to help their kids. They ask me about it... 00:12:26.000 --> 00:12:26.900 @Jacqui and often assessments pre the one in question are not sufficiently 'written' 00:12:29.000 --> 00:12:29.900 Martin, it would be difficult to make the argument of "generative AI use" vs legitimate reasons for changes in writing (e.g. writing coach, learning) 00:13:50.000 --> 00:13:50.900 I'm thinking of the uni accumulated sets of texts from students including cross-referencing, not a single criterion text 00:14:04.000 --> 00:14:04.900 my view is that detection reflects a 'catching people out' mentality. is it better to invest our efforts in using gen AI to value add to learning? 00:14:34.000 --> 00:14:34.900 You can train AI to mimic your idiomatic style of writing though. It doesn't take much input. 00:14:39.000 --> 00:14:39.900 ^^^Koshila 👍👍 00:14:53.000 --> 00:14:53.900 I think it's more important to teach students ethical and responsible use of GenAI 00:15:09.000 --> 00:15:09.900 For me it comes down to this: 1. What we really want to assess is the student, and their knowledge and skills. 2. For years (decades, centuries?) what we have actually assessed is some words on a bit of paper (which allegedly come from the student). How much that is an invalid proxy for the student, is being driven home to us. 00:15:14.000 --> 00:15:14.900 we will have bots answering the questions of other bots soon enough! 00:15:15.000 --> 00:15:15.900 Thank Matthew 00:15:26.000 --> 00:15:26.900 @Tim, 110% 00:15:40.000 --> 00:15:40.900 Great point Tim. 00:15:57.000 --> 00:15:57.900 @Jill LeBihan I was going to make the same point 00:16:59.000 --> 00:16:59.900 Do we need to look at the way we are framing assessments tho? Typically, they're seen as a stressful hurdle to get through....how can we reframe assessments so students see the value of completing these and....even get excited to do them? 00:18:15.000 --> 00:18:15.900 Sure. 00:18:29.000 --> 00:18:29.900 @Nikki. Nobel prize if you can solve that!😒 00:18:34.000 --> 00:18:34.900 LOL! 00:18:43.000 --> 00:18:43.900 "AI won't take your job but a person who can use AI will" a familiar quote now - we're obliged to teach students to use it responsibly, ethically and transparently 00:18:49.000 --> 00:18:49.900 Research project Laurine?? 00:19:08.000 --> 00:19:08.900 you betcha! 00:19:15.000 --> 00:19:15.900 🙂 00:19:16.000 --> 00:19:16.900 @Fiona Stroud yes indeed! 00:19:27.000 --> 00:19:27.900 👍 00:19:35.000 --> 00:19:35.900 Assessment IS teaching. We need to move to more formative assessment, away from high stakes summative tasks, but we might need to move away from classes with 700 students in them. And good luck with that. AI is going to be seen as a money saving tool. This all reflects our disdain for the complex work that teaching and assessment is. 00:20:04.000 --> 00:20:04.900 I feel it depends on what it is that we're aiming to assess. If we want to teach and assess students' autonomous writing abilities, then we need to ensure they're not getting it done by bots. It's a bit like athletics. We know athletes can run faster using blood doping and performance enhancers, but they're excluded because the purpose of those events is to assess their human abilities without those aids. I feel personally that there's room for both assisted and non-assisted writing at uni, but students are going to find the distinction confusing. 00:20:08.000 --> 00:20:08.900 oh that is SO true, Jacquie 00:20:16.000 --> 00:20:16.900 I agree with @Tim Hunt (OU) The reason ChatGPT is such a disruptor is that it mimics academic discourse well (if you encourage it) and we have previously used the mastery of academic discourse as a proxy measure for our students understanding what they have read. So instead of proxy measures through discourse, we have to understand how our students can apply their learning to novel problems in real time. 00:20:43.000 --> 00:20:43.900 Yes Jill!! 00:20:55.000 --> 00:20:55.900 I;m interested in how you see programmatic assessment in this, Matthew 00:21:00.000 --> 00:21:00.900 Well put Jill. 00:24:16.000 --> 00:24:16.900 @Jacquie - that is a $ issue which we have very little influence on & a workload issue in EBAs - agree though re formative rather than high stakes summative to assist with learning & assessment of... 00:24:31.000 --> 00:24:31.900 Agree with the above that academic discourse is over-privileged, but we still need discourses (those of our homes, family, friends, workplaces, etc..) to communicate meaningfully with other people and with AI. 00:25:36.000 --> 00:25:36.900 there are issues related to equity of access to the most advanced AI tools that are paid, so there will still be uneven playing field between students. 00:26:43.000 --> 00:26:43.900 sadly true, @Mathew Hillier 00:26:49.000 --> 00:26:49.900 we could also frame this from the perspective of authentic assessment - authentic assessment is now assessment that gets student to engage with gen AI 00:27:27.000 --> 00:27:27.900 @Koshila Kumar agree - AI is or soon will be a 'tool of the trade'. 00:27:32.000 --> 00:27:32.900 I think this is one of the biggest problems @Mathew Hillier We used to say to our students that they shouldn't be paying for extra services and that would help them evade the Essay Mill sharks. But now lots of students are paying for GrammarlyGo or even Co-pilot and the number of tools is spiralling. It's hard for us on academic salaries to keep up in this space too because we can't get institutional licenses for things. 00:28:07.000 --> 00:28:07.900 Mostly agree Koshila. - However, in maths and science - when it if it is valid to allow calculators in assessment has been an ongoing issue. It depends, and with AI it can only be less clear cut. 00:29:19.000 --> 00:29:19.900 You could still get great presentation using a mash-up of different GenAI tools. 00:29:36.000 --> 00:29:36.900 Without consistency across the higher education sector (which I guess is what TEQSA is looking for now) then students studying the same degree at different universities are also going to get vastly different experiences and results. 00:30:33.000 --> 00:30:33.900 @Jacquie: How does formative assessment solve the threat to academic integrity? Curious... 00:30:39.000 --> 00:30:39.900 @Tim Hunt (OU) IMHO there is a phase to go through. We want to know the students can fly the plan without the instruments, but also that they can use those instruments to fly more effectively too. 00:30:59.000 --> 00:30:59.900 Yes. That is the point Matthew 00:31:00.000 --> 00:31:00.900 @Tim Hunt (OU) what if the mark were awarded not for the 'answer' but the process of arriving at it using tools at your disposal. 00:31:19.000 --> 00:31:19.900 Yes, that's a great analogy, Matthew 00:31:41.000 --> 00:31:41.900 Koshila - I think the issue is more about what particular skill you are teaching and assessing right now. 00:32:04.000 --> 00:32:04.900 moving academics from "Content is King" and "More is Better" (ie learn all I know in one 12-week period) is in my bitter experience about the hardest step in any "Change plan" 00:32:17.000 --> 00:32:17.900 Maybe Level 5 is ChatGPT Tennis, where students share output from ChatGPT and then work on the prompts to change the output (better or worse). There is a short explanation of this here: https://monash.au.panopto.com/Panopto/Pages/Viewer.aspx?id=6efb0a55-bfed-4e71-9a8d-b02600201b6d 00:32:48.000 --> 00:32:48.900 We are encouraging a "show your workings" approach, and using AI transparently is part of that. But students need more help on how to document their AI use, I think. 00:32:52.000 --> 00:32:52.900 thanks, @Wendy! 00:33:11.000 --> 00:33:11.900 Thanks Wendy 00:33:44.000 --> 00:33:44.900 Agree @Jill LeBihan - this is my experience also. 00:35:12.000 --> 00:35:12.900 @Lotte Dyhrberg O'Neill #2 because formative assessment is done more often and as the student needs feedback to continue, it builds relationships with students so you know how they work and think, and is less high stakes, so the incentive to cheat is lessened as well. I like to use a UDL approach here, which also emphasises formative assessment. But again, huge classes makes this more difficult. The combination of the neoliberal university, continued funding cuts, a tool that people think can "do" teaching and assessment, and the lack of regulation are ingredients for a huge problem in this sector. IMHO 00:37:39.000 --> 00:37:39.900 assessment tasks/exams, esp if not invigilated, suffer from wholesale outsourcing (contract cheating). identify verification is still needed is it not? i.e many students are prepared to put their name on the work of others supplied from contract cheating providers. 00:38:15.000 --> 00:38:15.900 @Jacquie yes indeed... programmatic assessment utilising formative assessment and relationships (reducing transactional marking) might be one way. esp given $$ available go down never up..... 00:38:19.000 --> 00:38:19.900 What do you mean by "busy work"? We can just not give busy work... 00:38:29.000 --> 00:38:29.900 oh if only! 00:38:43.000 --> 00:38:43.900 @Jacquie: Yes, feasibility is the Achilles heel of formative assessment. In my experience, the assessment stakes do not have to be high for cheating. Sadly. 00:39:18.000 --> 00:39:18.900 @Jacquie I'm thinking that this is trying to short cut to the bit where the student needs to think! What is the point that they need to analyse, think, compare etc 00:39:26.000 --> 00:39:26.900 http://matthewwysel.com/linkedin 00:39:52.000 --> 00:39:52.900 Maybe this is the way forward instead regarding Authorship Authentication System https://norlab.ai/education 00:39:58.000 --> 00:39:58.900 Yes @Wendy Taleo That bit is missing entirely, and that's the point of education at this level. 00:40:07.000 --> 00:40:07.900 The ads for ChatGPT that are popping up on my Facebook make a big sales pitch out of its potential to 'write essays and reports for you' ... it doesn't come across as a product that's been designed and marketed with ethical and educational goals .. as shown also in the way they exploited other writers to create the programme. 00:40:14.000 --> 00:40:14.900 https://www.youtube.com/@Aichiever23 00:40:41.000 --> 00:40:41.900 thanks, Sinead 00:40:51.000 --> 00:40:51.900 Thanks Sinead 00:41:15.000 --> 00:41:15.900 Isn't it this one? https://www.youtube.com/@aiachievers 00:41:55.000 --> 00:41:55.900 Do we want students to read a novel or just use a crib............................ 00:41:59.000 --> 00:41:59.900 sorry, more than one out there 00:42:55.000 --> 00:42:55.900 @Michael fair question, but what do WE do? I skim and dip in and out at times too 00:43:36.000 --> 00:43:36.900 We are also missing the bit where Gen AI doesn't 'know' things, and we are asking students to use it to learn. I would be concerned that "less sub-par assignments' means that students can present work that reads better, but we still won't know if they have learned what they need to know. 00:43:45.000 --> 00:43:45.900 Equity - AI as the knowledgeable other - some of us have people that offer us this, for the others, they now have an option. 00:43:53.000 --> 00:43:53.900 I almost never read journal papers as one would a story book. reading with questions in mind means you do skim and jump to sections. 00:43:55.000 --> 00:43:55.900 Good tips Matthew! Important to embrace rather than resist it! 00:44:33.000 --> 00:44:33.900 Thanks Matthew - great to see examples of practice as well as raising and sharing questions 00:45:01.000 --> 00:45:01.900 @Beth, we have seen this too... 00:45:05.000 --> 00:45:05.900 But @Mathew Hillier when you do that you know what you are skimming and what you are looking for. If they don't learn that, they don't understand what they are reading or how to look for the useful parts of an article. 00:45:27.000 --> 00:45:27.900 Agreed Jacquie 00:45:37.000 --> 00:45:37.900 Generative AI will open so many new opportunities for students and industries. Looking forward to see how educators can collaborate with industries to upskill students on how we can leverage Generative AI to transform the learning experience for today and tomorrow! 🥰 . lha1[at]kpmg.com.au 00:45:45.000 --> 00:45:45.900 @Jacquie yes indeed. we need to teach how to do so. (reading and with ai) 00:46:03.000 --> 00:46:03.900 At this point, we can compare and judge Ai informed work. In the future, if the students of today won't be able to do that, they won't have those skills. 00:46:05.000 --> 00:46:05.900 @JAcquie, yes, so scaffolding - and modelling?- "why and how and when and WHAT to read" is important 00:46:17.000 --> 00:46:17.900 agree with you mathew 00:47:30.000 --> 00:47:30.900 Thanks for this session. Got to pedal to campus now so I'm going to log out. But thanks, everyone. 00:48:15.000 --> 00:48:15.900 Thank you - great session and lots of food for thought!! 00:48:22.000 --> 00:48:22.900 @Jacquie Agree. I sometimes find it more difficult/takes longer to mark because of this. It sounds good, but do they really understand how that theory applies? OR Sometimes, it's me asking myself, 'Does that theory apply here?' if ChatGPT has spun out something I'm not expecting or familiar with. 00:50:52.000 --> 00:50:52.900 Thank you. Have to leave for next meeting. Really interesting. 00:52:38.000 --> 00:52:38.900 Thanks very much 00:52:41.000 --> 00:52:41.900 a word guessing engine. 00:52:56.000 --> 00:52:56.900 not a database of facts. 00:52:59.000 --> 00:52:59.900 I have to leave now but thanks so much, Matthew, a really interesting presentation. 00:53:15.000 --> 00:53:15.900 useful vs right is a good frame 00:53:19.000 --> 00:53:19.900 Or a quality answer. And the student doesn't know the answer either - they didn't 'create' or 'find' the answer through their own thinking. 00:53:20.000 --> 00:53:20.900 Heuristic 00:53:35.000 --> 00:53:35.900 Agreed! Thinking about it this way is helpful for interacting with these tools - and makes them a lot less "SkyNet". 00:53:39.000 --> 00:53:39.900 Thank you so much Matthew. 00:53:45.000 --> 00:53:45.900 Thank you Matthew/everybody 👍 00:53:52.000 --> 00:53:52.900 There was a cool lecturer from UNE / Who introduced AI to me / He told us to use it / But never abuse it / And don't mess with probability (non AI generated limerick!) 00:53:54.000 --> 00:53:54.900 Thank you Matthew, really insightful 00:53:59.000 --> 00:53:59.900 Hi everyone, I suggest we continue this discussion in a post about this event I dropped today: https://www.linkedin.com/posts/matthewwysel_frenemies-unleashing-chatgpt-into-assessments-activity-7183688850718777344-G--l That way we can all move to a/synchronous and also keep the discussion up as we move across trains/buses etc. Matthew Wysel. PS. if you can't find that post, then search for me on LinkedIn, follow, and the post will probably shoot to the top of your feed. 👍🏼 00:54:02.000 --> 00:54:02.900 Thank you for a thought=provoking session. 00:54:15.000 --> 00:54:15.900 Thank you Matthew, very helpful. 00:54:18.000 --> 00:54:18.900 Thanks so much Matthew and Matt. This has been a really interesting presentation. 00:54:27.000 --> 00:54:27.900 Thank you for a very insightful session. 00:54:30.000 --> 00:54:30.900 Thanks all. Good to know we're thinking along the same lines here in Cardiff. 00:54:30.000 --> 00:54:30.900 thank you👏 00:54:31.000 --> 00:54:31.900 Matthew W , suggests we continue this discussion in a post about this event: https://www.linkedin.com/posts/matthewwysel_frenemies-unleashing-chatgpt-into-assessments-activity-7183688850718777344-G--l That way we can all move to a/synchronous and also keep the discussion up as we move across trains/buses etc. 00:54:36.000 --> 00:54:36.900 Thank you Matthew, thanks everyone 00:54:42.000 --> 00:54:42.900 Thank you 00:54:42.000 --> 00:54:42.900 This was terrific, thank you! Loved the questions and comments too 🙂 00:54:53.000 --> 00:54:53.900 Thank you. 00:54:56.000 --> 00:54:56.900 Excellent, thank you! 00:55:01.000 --> 00:55:01.900 👏 thank you for a wonderful session! 00:56:07.000 --> 00:56:07.900 Thanks very much. Great stuff! 00:56:07.000 --> 00:56:07.900 Thank you! 00:56:12.000 --> 00:56:12.900 👏 00:56:13.000 --> 00:56:13.900 thanks Matt! 00:56:14.000 --> 00:56:14.900 Thank you for an interesting discussion! 00:56:14.000 --> 00:56:14.900 Thank you - love these sessions! 00:56:14.000 --> 00:56:14.900 Thank you - this was really interesting 00:56:18.000 --> 00:56:18.900 This was excellent, thank you! 00:56:23.000 --> 00:56:23.900 thank you, really great session! 00:56:25.000 --> 00:56:25.900 Thank you! 00:56:38.000 --> 00:56:38.900 Thanks a lot Matthew and Mathew! 00:56:39.000 --> 00:56:39.900 and LOL Martin for the Limerick !