ScamShield: Interpretable Multi-Signal Scam Detection

Abstract

Scam and phishing detection systems typically rely on rigid heuristic rules or opaque large language models. This paper presents ScamShield: a hybrid, interpretable pipeline combining a 24-feature supervised machine learning model, an LLM semantic safety layer, and rule-based heuristics in a unified ensemble. Evaluated on a synthetic benchmark of 19,992 messages spanning 17 scam categories and validated on the UCI SMS Spam Collection (5,574 real-world messages), ScamShield achieves cross-validated F1 = 0.9969 ± 0.0004 on the synthetic benchmark and F1 = 0.9303 ± 0.0098 on real-world data — competitive with fine-tuned DistilBERT (estimated F1 ≈ 0.97–0.99) while requiring 125× less storage, operating at sub-5 ms inference latency, and providing full per-prediction interpretability via coefficient attribution. Statistical significance is confirmed by McNemar's test against all four baselines (p < 0.001). Adversarial evaluation reveals recall drops of 71–82% under obfuscation attacks, identifying semantic embeddings and character-level robustness as the primary improvement directions.

Keywords

Applied Science - Computer Science

Citation

Publication Details

ISSN: Online ISSN: 3143-5653

License: Author-retained; open access display by NSRI unless a separate article license states otherwise.

Peer review status: NSRI uses editorial and scholarly review. When appropriate, manuscripts may undergo blinded review by reviewers with relevant subject knowledge.

AI disclosure: No AI disclosure is attached to this public record unless stated in the manuscript.

Conflict of interest statement: No conflict of interest statement is attached to this public record unless stated in the manuscript.

References

References are available in the manuscript PDF when provided.