What This Is
The companion repo for our lab What ‘Open’ Actually Means: Abliterating Gemma 4 in 24 Minutes. It contains everything needed to reproduce the experiment: running Heretic, a fully automatic LLM decensoring tool, against Google’s Gemma 4 E4B-it model, plus the comparison tooling and results.
This is what the “every experiment ships its repo” promise looks like in practice: the setup scripts, the run configuration, the evaluation harness, and the numbers we published.
Key Findings
| Metric | Original | Abliterated (Trial #157) |
|---|---|---|
| Refusals on harmful prompts | 97/99 (98%) | 47/99 (47.5%) |
| KL divergence from original | 0.0 | 0.1029 |
| Normal task capability | Baseline | Identical |
| Total time | n/a | 24 minutes |
| Adapter size | n/a | 3.4 MB |
Gemma 4 proved significantly more resistant to abliteration than Gemma 3 (47% refusals remaining vs 3%), suggesting Google hardened alignment between generations.
Quickstart
Requires an NVIDIA GPU with 16+ GB VRAM (tested on RTX 5090 Laptop), CUDA 12+, Python 3.10+, and Linux (native or WSL2).
git clone https://github.com/thenewguardai/tng-heretic.git
cd tng-heretic
# Creates .venv, installs Heretic + Gemma 4 deps
npm run setup
# Full run: 200 trials, ~24 min on RTX 5090
npm run hereticA shorter 50-trial variant is included for validating your setup before committing to the full run.
Why This Exists
“Open weights” means anyone can point an automated decensoring tool at a model within hours of release. This repo makes that claim concrete and testable instead of theoretical: run it yourself, compare your numbers to ours, and draw your own conclusions about what runtime safety needs to cover.