LLM fails. Gescheiterte Experimente mit Generativer KI und was wir daraus lernen können

Workshop am 8. und 9. April 2025

Organization:  Ngoc Duyen Tanja Tu (Abteilung Grammatik),  Christian Lang (Abteilung Grammatik), Annelen Brunner (Abteilung Lexik)

Important dates

Participation in the workshop is free of charge.

There will be a "Best short paper" award, sponsored by Freundeskreis des IDS (Friends of IDS).

Description

Failed experiments typically have no place in scientific discourse; they are discarded and not published. We believe this leads to a loss of potential knowledge. After all, a systematic reflection on the reasons for failure allows for the questioning and/or improvement of methods used. Furthermore, when previously failed experiments are repeated and succeed, explicit progress can be determined. Thus, the discussion and documentation of failures creates added value for the scientific community from the perspective of methodological reflection. This is even more relevant in a field like research into and with Generative Artificial Intelligence (AI), which cannot look back on decades of tradition and where best practices are still being negotiated.

This workshop focuses on linguistic and NLP experiments with Generative AI that did not yield the desired results, such as but not limited to:

• Using Generative AI as a Named-Entity Recognizer

• Using Generative AI for automatic transcription of spoken language data

• Using Generative AI for the creation of dictionary entries

• Using Generative AI for the detection of language change phenomena

The contributions focus on how this failure can contribute to knowledge gain regarding the work with Generative AI. If a contribution is accepted, there is an opportuniy to submit a short paper. The short papers will undergo double-blind peer review and will be published in a special issue of the Journal for Language Technology and Computational Linguistics and archived at ACL Anthology.

The workshop will take place from April 8-9, 2025 (Lunch-to-Lunch) at the Leibniz Institute for the German Language in Mannheim. The 20-minutes presentations will be given in English.