TPP. Case Study Three. AI as a Teaching Collaborator, Now and the Future.
Introduction.
The full extent of AI impending impact on education and academia is as of yet unknown (Dignum, V. 2021) . It is expected to impact all industries including education, and there are opportunities now to shape this outcome. We have begun to see tools such as ChatGPT (Chat GPT, 2024) and image generation platforms (MidJourney, 2024), change the way students work, and in turn institutions have begun to respond. This Case study is looking to speculate on how positively AI tools can enhance assessment and feedback models, specifically as a tutor, student collaborator in this process. MA Design for Art Direction (MADAD) assessment process will be mapped, to highlight opportunities for change.
Context.
MADAD course has six formative feedback points excluding resubmissions, and numerous summative feedback points. All six assessments are 100% second marked, and third marked. UAL requires only Unit05: Final Major Project to be 100% second marked, and the other units to be at least 10% second assessed.
Assessment Model %.
The reason we complete 100% second, and third marking is to enhance the parity, as from PTES (PTES, 2023) results it has shown this is an area of improvement.
PTES Free Comment Feedback on Assessment.
PTES MADAD 22/23 Freeform comments. (PTES, 2023).
PTES MADAD 22/23 Freeform comments. (PTES, 2023).
PTES MADAD.
PTES MADAD 22/23 Assessment and Feedback 78%. (PTES, 2023).
The PTES data is in contrast to our External Examiners report, that suggests that the assessment workload is excessive and that the MADAD team could do less.MADAD External Examiner Report 22/23. Eleni Ikon.
Assessment Structure MADAD.
MADAD60 students are enrolled on the MADAD course in 23/24, we have six tutors assessing each unit, with the exception of Unit05: Final Major Project which has 12. For the five units with six assessors, there is a simple formula used to ensure tutors are rotated, and the Unit Lead is never repeated.
MADAD Formula to ensure rotation of tutors. This is applied against an alphabetical list of student names.
Given this commitment to parity of assessment, students still query their grades and feedback. Which is responded to with an opportunity to speak with their assessor for clarification, this interaction usually resolves the query.
MADAD student email.
How Could this be done better?
I am interested in how AI could be integrated to improve this model. AI in assessment is often viewed through as a deficit model, and how it jeopardises the integrity model in education (Bouteraa, M. Bin-Nashwan, S, A. Al-Daihani, M. Dirie, K, D. Benlahcene, A. Sadallah, M. Zaki, H, O. Lada, S. Ansar, R. Fook, L, M. Chekima, B. 2024), which I am not interested in investigating here. I am more aligned with how it can be a collaborator and intermediate actor between tutor and student, and how it could potentially reinforce the parity of assessment (Ahmad, S, F. Alam, M, M. Mohd, K, R. Mubarik, M, S. Hyder, S, I. 2022). I will first map the MADAD assessment model, review possible opportunities, and then engage with basic AI on a simple task around Learning Outcomes and the UAL Matrix.
Diagram drawn in TPP Pgcert unit to map assessment.
Revised Assessment Diagram with AI opportunities plotted.
ChatGPT.
As an experiment I inputted basic prompts and information into ChatGPT to generate outcomes. These were subjective questions, and prompts around the UAL Matrix and Learning Outcomes.
ChatGPTs responses to subjective questions on AI’s involvement in assessment.
The five UAL’s Matrix was used as a prompt to generate ChatGPTs definitions
UAL’s Matrix definitions were compared to ChatGPTs. ChatCPTs overall were better.
Asking ChatGPT more complex prompts around Unit Learning Objectives.
Outcomes.
I was surprised by how useful this exercise proved to be. The expansion explanations on the UAL Matrix I intend to incorporate into some of the course literature and resources moving forward, as they explain more fully how these areas can be applied.
Expert Interview.
After briefly discussing the possible use of AI at UAL with Shelia Smith a Senior Digital Learning Advisor, they recommended I speak with Chris Rowell who is Digital Learning Producer Community, to discuss further. I organised an interview with Chris on this. Chris has been involved in numerous progressive AI initiatives at UAL. Some chosen highlight quotes below.
Summary.
There are many unknowns with emerging technology, I think products like the TeacherMatic (TeacherMatic, 2024) could be very impactful on holistic workloads. For assessment there are currently opportunities to use tools such as ChatGPT to strategically support. This could be in the format of checking tutors written feedback for spellings, grammar and clarity. Or supporting content expansion, such as the UAL Matrix experiment from this Case Study. In the future I would welcome AI as a more active collaborator to support parity.
Reference List.
Ahmad, S, F. Alam, M, M. Mohd, K, R. Mubarik, M, S. Hyder, S, I. (2022). ‘Academic and Administrative Role of Artificial Intelligence in Education’. MDPI. Available at https://www.mdpi.com/2071-1050/14/3/1101. Last Accessed 15th March.
Bouteraa, M. Bin-Nashwan, S, A. Al-Daihani, M. Dirie, K, D. Benlahcene, A. Sadallah, M. Zaki, H, O. Lada, S. Ansar, R. Fook, L, M. Chekima, B. (2024). ‘Understanding the diffusion of AI-generative (ChatGPT) in higher education: Does students’ integrity matter?’. Science Direct. Available at https://www.sciencedirect.com/science/article/pii/S2451958824000356 . Last Accessed 20th May.
Chat GPT (2024). OpenAI. Available at https://openai.com/index/chatgpt/ . Last accessed 15th March 2024.
Dignum, V. (2021). ‘The role and challenges of education for responsible AI’. London. Review of Education, 19 (1), 1, 1–11. https://doi.org/10.14324/LRE.19.1.01
MidJourney (2024), MidJourney, Discord. Available at https://www.midjourney.com/home . Last Accessed 15th March 2024.
PTES (2023), Dashboards, UAL. Available at https://dashboards.arts.ac.uk/dashboard/ActiveDashboards/DashboardPage.aspx?dashboardid=29c2e5b2-f8c5-4c3b-a5ad-5a27ad4ace4b&dashcontextid=638584675454108246 . Last Accessed 15th March.
TeacherMatic 2024. Available at https://teachermatic.com/ . Last Accessed 15th March.