From toxicity to constructive dialogue: an LLM-driven detoxification approach with multi-source parallel data

Abstract

The surge of toxic interactions online poses a serious obstacle to cultivating constructive digital spaces. Current automated detoxification systems, though encouraging in theory, often stumble: they veer off-topic (semantic drift), strip away too much nuance (over-detoxification), and operate like black boxes, eroding trust and complicating real-world use. In response, this paper outlines a detoxification pipeline designed to tackle these issues head-on by weaving explicit reasoning into grounded generation. Our proposed pipeline particularly introduces a multi-task setup that finetunes a 7-billion-parameter language model in two stages: first, it produces a clear explanation of why a comment is toxic (drawing on the concept of Chain-of-Thought prompting), and then it rewrites the comment. What sets our approach apart is a three-step inference routine: the explanation guides a retrieval query that fetches relevant, benign examples from a curated knowledge base via Retrieval-Augmented Generation. The final rewrite leans on both the rationale and these examples, anchoring the output in a transparent reasoning path and verified data. To bolster robustness, we merge data from over 40,000 cases across three sources—synthetic examples with reasoning (DetoxLLM), human-crafted paraphrases (ParaDetox), and adversarial hate speech instances (ToxiGen)—using a fresh fusion strategy. Crucially, we make the fine-tuning doable on everyday hardware by employing 4-bit Quantized Low-Rank Adaptation (QLoRA). Experiments show that our reasoningplus- retrieval pipeline is a feasible and effective alternative at retaining the original meaning while achieving strong style-transfer metrics. By emphasising not only performance but also clarity and fidelity, this system charts a path toward more accountable, inspectable moderation tools.