Publications
DeepWrite - The AI corrector?
In this article by recode.law e.V., Olesja Kaltenecker and Jeremias Forssman take a closer look at project DeepWrite. Based on an interview with Christian Braun, Simon Alexander Nonn and Sarah Großkopf from the DeepWrite research project at the University of Passau, they examine the project itself – its strengths and opportunities – as well as the challenges that still exist. With careful preparation, AI can provide appropriate feedback, especially for shorter student solutions. Both the accuracy of the content and the (appraisal-)style as well as grammatical and lexical correctness are taken into account. However, the assessment of long solutions poses a challenge, not only in terms of the coherence of an argumentation throughout an entire exam.
Conceptualising of the artificial intelligence legal learning platform DeepWrite in Germany - Focused on the user interface based on the survey results
This paper by Yujin Kang in the Korean Design Forum (한국디자인포럼) is based on a "Survey to determine the UX needs of law students", which was conducted in the winter semester 2023/24 by the Department of Law at the University of Passau. In this study, the responses of the survey participants with regard to both the User Experience and the integration of Artificial Intelligence into such a learning platform. With regard to the User Experience, the focus is on the User Interface and the design system, which reflect the requirements and preferences of future users. The positive appearance of the learning platform allows users to use the platform over a longer period of time and to maintain the users' attention. In addition, the article deals with the theoretical consideration of the AI- and design process and highlights the importance of Human-Computer-Interaction (HCI) from the point of view of User Experience Design by comparing the user interfaces of the Large Language Model ChatGPT in the versions GPT-2 and GPT-3.5.
Can ChatGPT replace university lecturers?
In this article in the JuS (Juristische Schulung), research assistants Christian Braun, Sarah Großkopf and Simon A. Nonn discuss the question of whether ChatGPT can replace university lecturers - especially with regard to the teaching of legal reasoning skills and the "Gutachtenstil" using AI feedback.
In recent years, the Covid-19 pandemic and the associated rapid developments and advances in the field of digitalization have shown that didactics in higher education is currently undergoing change and that this process can and should be actively influenced in order to maintain the future viability and competitiveness of universities. A large part of this is the use of innovative technology and tools, such as artificial intelligence (AI), in particular large language models (LLM) and natural language processing (NLP), to create digital teaching and learning spaces for students of future generations.
Is GPT-4 a reliable rater? Evaluating consistency in GPT-4's text ratings
This study reports the Intraclass Correlation Coefficients of feedback ratings produced by OpenAI's GPT-4, a large language model (LLM), across various iterations, time frames, and stylistic variations. The model was used to rate responses to tasks related to macroeconomics in higher education (HE), based on their content and style. Statistical analysis was performed to determine the absolute agreement and consistency of ratings in all iterations, and the correlation between the ratings in terms of content and style. The findings revealed high interrater reliability, with ICC scores ranging from 0.94 to 0.99 for different time periods, indicating that GPT-4 is capable of producing consistent ratings. The prompt used in this study is also presented and explained.
Performance analysis of large language models in the domain of legal argument mining
In this study we (Abdullah Al Zubaer, Michael Granitzer and Jelena Mitrović) investigate the effectiveness of GPT-3.5 and GPT-4 for argument mining in the legal domain, focusing on prompt formulation and example selection using state-of-the-art embedding models from OpenAI and sentence transformers. Our experiments demonstrate that relatively small domain-specific models outperform GPT 3.5 and GPT-4 in classifying premises and conclusions, indicating a gap in these models' performance for complex legal texts. We also observe comparable performance between the two embedding models, with a slight improvement in the local model's ability for prompt selection. Our results indicate that the structure of prompts significantly impacts the performance of GPT models and should be considered when designing them.
How AI is used in higher education
In this interview series, the Federal Agency for Civic Education presents three projects funded by the Federal Ministry of Education, Science and Research. As part of this series, Veronika Hackl was able to introduce the project DeepWrite to readers.
Conference Report on the Second Passau June Conference
From 24-26 June 2022, ELSA-Passau organised the second June conference under the motto "Smart Law". This conference was to address nothing less than the future of law and the digitalisation of the legal profession. In the attached conference report, the project DeepWrite, among others, is discussed.