Eight Public AI-Music Detectors on One Test Set
Eight detectors, the same 2,104 audio files and their public decision thresholds. The useful result is not only who ranks first; it is how differently the systems trade missed AI tracks against real music falsely flagged as AI.
Result in one line: ArtifactNet v9.4 had the highest F1 in this run, while FST had the lowest false-positive rate. Those are different operating objectives; neither number makes a detector an automatic proof of authorship.
Update (2026-09-23): ArtifactBench v2 has been released — a lineage-aware, frozen-protocol evaluation suite of 828 tracks (605 AI and 223 real), split into calibration, validation and sealed-test before any comparison. On the 562-track paired sealed-test intersection, ArtifactNet v9.4 records AUROC 0.982 and balanced-accuracy 0.918, against the public Deezer detector 0.761 and 0.776. v2 is designed so that model scores cannot be silently compared across different file sets — see GitHub · Hugging Face · paper. These numbers are tied to the v2 cohort and calibration-only threshold policy and should not be compared as if scored on the v1 cohort above.
The controlled comparison
The run used the same restored ArtifactBench v1.1 files for every adapter: 1,388 AI-generated tracks and 716 real tracks. Scores were evaluated at each public adapter's fixed threshold of 0.5. Parameter counts, preprocessing adapters and per-source results are preserved in the public result record.
| Detector | Parameters | F1 | Precision | Recall | Real FPR |
|---|---|---|---|---|---|
| ArtifactNet v9.4 ONNX | 4.2M | 0.952 | 0.932 | 97.3% | 13.8% |
| AI-Music-Detection AST-60s | 90.8M | 0.840 | 0.848 | 83.1% | 28.9% |
| CLAM (MoM) | 194.3M | 0.787 | 0.711 | 88.3% | 69.7% |
| SpecTTTra α-120s | 18.7M | 0.777 | 0.880 | 69.5% | 18.4% |
| Deezer ISMIR fakeprint LR | 3.6K | 0.754 | 0.906 | 64.6% | 13.0% |
| FST (Mippia) | 174.4M | 0.735 | 0.984 | 58.7% | 1.8% |
| DeepFense EAT+Nes2Net | — | 0.650 | 0.589 | 72.4% | 97.8% |
| SpecTTTra β-5s | 18.7M | 0.563 | 0.884 | 41.3% | 10.5% |
Download this table: results-8way.csv (8 rows; one column per figure above, plus test set, file counts, threshold, run date 2026-07-04 and report date). Licensed CC BY-NC 4.0, the same licence as the ArtifactBench dataset.
Key figures for citation
Each sentence below is self-contained: subject, number, test set, threshold and date, so it can be quoted without the table. All figures: ArtifactBench v1.1, 2,104 files (1,388 AI-generated, 716 real), each detector at its public fixed threshold of 0.5, run by Intrect Research, published 2026-09-14.
- ArtifactNet v9.4 (4.2M parameters) scored the highest F1 of the eight detectors, 0.952, with 97.3% recall on AI tracks and a 13.8% false-positive rate on real music.
- FST (Mippia, 174.4M parameters) had the lowest false-positive rate, 1.8%, at the cost of 58.7% recall (F1 0.735).
- Two detectors flagged more than two thirds of the real partition as AI: CLAM (MoM) at 69.7% and DeepFense EAT+Nes2Net at 97.8%.
- Parameter count did not predict rank: the 3.6K-parameter Deezer fakeprint model (F1 0.754) outscored two models above 170M parameters.
- Update 2026-09-23, ArtifactBench v2 sealed test (562 paired tracks): ArtifactNet v9.4 AUROC 0.982 and balanced accuracy 0.918, against 0.761 and 0.776 for the public Deezer detector. v2 figures are a different cohort and are not comparable with the v1.1 table.
Cite as: Intrect Research (2026). Eight Public AI-Music Detectors on One Test Set (Artifact Observatory report 001, ArtifactBench v1.1). https://intrect.io/research/artifact-observatory/ai-music-detector-benchmark-2026/ — figures as of 2026-09-14; revisions are dated in the callouts above and the public result record is the source of truth.
Why the false-positive column matters
A label or distributor reviewing a catalog does not pay only for missed AI tracks. Every genuine release incorrectly escalated consumes review time and can harm a creator. In this run, two systems with superficially useful recall flagged more than two thirds of the real partition. FST made the opposite trade: the fewest false alarms, but missed more AI tracks. The correct threshold therefore depends on whether the detector is a triage signal, an investigative aid or one input to a human appeal process.
What this report does not prove
- The restored real subset contained 716 available files, not every row in the purged manifest. The public record explains the restoration boundary.
- A fixed threshold is reproducible, but it is not necessarily the best threshold for a particular catalog or error budget.
- Generator updates and new delivery codecs can change performance after the review date.
- Detection is not provenance. A score should not replace Content Credentials, platform records or a human review and appeal path.
A separate task: removing audible artifacts
Detection asks where a track may have come from. Restoration asks whether codec-like residue can be reduced without damaging the music. The two clips below demonstrate the second task only. They are the same public, curated Suno example used on the de-artifact product page; they are not extra detector test rows.
Original AI mix
Suno 032 · dry input
Cleaned mix
Same excerpt · de-artifact AI preset
Looking for a buyer's view instead of the methodology? See Best AI Music Detector (2026): eight tools compared.
Sources and revision policy
Primary sources: ArtifactBench dataset card, the 8-way result table, and the ArtifactNet paper. This is a dated snapshot: the v1.1 comparison was run July 4, 2026 and published here September 14, 2026, with the v2 sealed-test update noted September 23, 2026. There is no fixed re-run schedule — the table is refreshed only on a real re-measurement, and a changed partition or rerun will receive a new report or an explicit correction note instead of silently replacing these figures. Until a re-run happens, the numbers above remain as of their stated dates.
Test the maintained detector on your own file
The first analysis is free. Treat the result as forensic evidence to review, not a verdict to enforce automatically.
Run a free analysis →