Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

AUTHORS: Andrea PedrottiMichele PapucciCristiano CiaccioAlessio Miaschi, Giovanni Puccetti, Felice Dell’Orletta, Andrea Esuli

WORK PACKAGE:

URL: https://aclanthology.org/2025.findings-acl.156.pdf

Keywords:

Abstract

Recent advancements in Generative AI and Large Language Models (LLMs) have enabled the creation of highly realistic synthetic content, raising concerns about the potential for malicious use, such as misinformation and manipulation. Moreover, detecting Machine-Generated Text (MGT) remains challenging due to the lack of robust benchmarks that assess generalization to real-world scenarios. In this work, we evaluate the resilience of state-of-the-art MGT detectors (e.g., Mage, Radar, LLM-DetectAIve) to linguistically informed adversarial attacks. We develop a pipeline that fine-tunes language models using Direct Preference Optimization (DPO) to shift the MGT style toward human-written text (HWT), obtaining generations more challenging to detect by current models. Additionally, we analyze the linguistic shifts induced by the alignment and how detectors rely on “linguistic shortcuts” to detect texts. Our results show that detectors can be easily fooled with relatively few examples, resulting in a significant drop in detecting performances. This highlights the importance of improving detection methods and making them robust to unseen in-domain texts. We release code, models, and data to support future research on more robust MGT detection benchmarks.




I regesti dei pontefici romani e l’intelligenza artificiale nel progetto REVErse Regesta

AUTHORS: Ilaria Sabbatini

WORK PACKAGE: WP 7- REVER

URL: https://iris.unipa.it/handle/10447/703959

Keywords:  Digital Humanities, Regesta, Papal registers, Medieval documents, Artificial intelligence, Handwritten Text Recognition (HTR), Information extraction, Summarization, Diplomatics, Corpus building

Abstract
The contribution presents the REVErse Regesta project, aimed at developing an automated regesta generation system for medieval papal documents through the integration of diplomatics, palaeography, and artificial intelligence. By building a corpus based on papal registers and employing HTR technologies and tagging processes, the project seeks to implement summarization and information extraction grounded in documentary structures, with the goal of producing structured regesta and supporting both research and teaching.




A digital archive for religious texts from the Nile Valley and beyond

AUTHORS: Federico Maria Avano, Angela Bosco, Andrea D’Andrea,
Gilda Ferrandino & Zied Mnasri

WORK PACKAGE: WP 10 – ReTINA

URL: https://unora.unior.it//retrieve/handle/11574/255100/263291/ANCIET_EGYPT.pdf

Keywords:  Artificial Intelligence; Egyptian religious texts; Metadata; Image processing; Deep and
Machine learning

Abstract
TWithin the project ITSERR, granted by Recovery Plan, the WP 10 ReTINA aims to create an
environment that defines and optimizes guidelines for the digitization of a diverse range of
sources, considering the challenges posed by very different languages and types of scriptural
media. The dataset includes religious texts from sites in the ancient world of the Nile Valley
and beyond. The contents of these texts are very diverse and varied: descriptions of religious
ceremonies, funerary texts, prayers, incantations, lists of offerings and temple personnel. So far,
several attempts have been made to achieve automatic textual analysis from digitized religious
texts. However, three main problems still hinder this task: firstly, the lack of sufficient material,
which results in small data sets, mostly related to a single site; secondly, the variety of languages
and writing forms; and finally, the lack of dedicated methods, intentionally developed to analyze
and process such ancient religious texts.
This paper intends to bridge the gap between the state of the art of linguistic text analysis on
the one hand and image processing applied to historical texts on the other, starting from a review
of AI and metadata projects in the field of Egyptian language text analysis. The paper aims to:
a) study the datasets, methods and tools that enable the restoration of missing text fragments
from scanned images of religious texts from the Nile Valley, including funerary inscriptions on
tombs and other types of related materials such as papyri, parchments, wood, etc.; b) apply image
processing and linguistic analysis to obtain transcription and possibly analysis and restoration of
other religious manuscripts, covering a wide range of languages. The survey intends to provide a
comprehensive review of the state of the art of this research topic, including three main aspects,
which will be the focus of attention during the implementation of the project: i) datasets, including
digitized images and manuscripts, ii) open data repositories, with associated metadata standards,
taxonomies and thesauri, iii) the state of the art projects that has already been realized, either
for Egyptological or other archaeological data annotation and 3D visualization and iv) artificial
intelligence methods and tools, mainly for image processing and linguistic analysis, commonly
used for this type of problems. Finally, a novel idea for applying artificial intelligence methods to
the related data types is proposed in the framework of the project ITSERR.