TAICIThe AI Cybersecurity Institute

Research

Publications

Peer-reviewed work from our team measuring how far AI can automate social engineering — and how well it defends against it. Each study tests frontier models on real human subjects rather than on synthetic proxies, which is what makes the numbers usable as a baseline.

This research is the empirical foundation for the benchmarks we build: ScamBench, ManipulationBench, and the Scam Killchain.

  1. Expert Systems with Applications · 2026

    Evaluating large language models’ ability to automate spear phishing

    F. Heiding, S. Lermen, A. Kao, B. Schneier, A. Vishwanath

    A human-subject study measuring whether frontier models can run personalized spear phishing end to end — reconnaissance, targeting, and email generation — benchmarked against human experts.

    Key findings

    • Fully AI-automated emails matched human experts at a 54% click-through rate, against 12% for the arbitrary-phishing control group.
    • Automation cut the cost per target by roughly 92%, with economic analysis pointing to as much as a 50× increase in attacker ROI.
    • Models produced accurate, useful target profiles in 88% of cases from open sources alone.
    • The same capability cuts both ways: properly prompted models detected phishing intent with over 90% accuracy and few false positives.
    Read the paper Open-access preprint DOI: 10.48550/arXiv.2412.00586

  2. Expert Systems with Applications · 2026

    Evaluating AI models’ capability to automate voice phishing attacks

    F. Heiding, C. Mayrink Verdun, S. Lermen, A. Kao, V. Albiero, L. Deason, I.-E. Veliche, C. Lehane

    A large-scale survey experiment (N=4,100) plus qualitative interviews (N=12) testing how US adults respond to voice phishing calls generated by leading speech models, against human baselines.

    Key findings

    • Overall compliance with the phishing request reached 16.5% across five scam categories, rising to 36% for the “relative in distress” scenario.
    • Caller persuasiveness was the strongest predictor of compliance, and some models — Sesame most notably — rated on par with or slightly above human callers.
    • Human-operated vishing is unprofitable at US wages; several AI voice models make the same attack economically viable.
    • The threat is automation economics rather than novel persuasion tactics — which is what makes it scale.
    Read the paper DOI: 10.1016/j.eswa.2026.133620

  3. IEEE Access, vol. 12, pp. 42131–42146 · 2024

    Devising and Detecting Phishing Emails Using Large Language Models

    F. Heiding, B. Schneier, A. Vishwanath, J. Bernstein, P. S. Park

    The earlier study in this line of work: comparing LLM-generated phishing emails against human-crafted and V-Triad emails on real participants, then testing whether the same models can detect what they produce.

    Key findings

    • GPT-4-generated phishing emails performed comparably to hand-crafted expert emails on click-through, at a fraction of the effort.
    • LLMs showed strong but inconsistent detection performance, sensitive to how the prompt frames the task.
    • Established that offensive and defensive phishing capability rise together in the same models.
    Read the paper DOI: 10.1109/ACCESS.2024.3375882

For questions about this work, collaborations, or replication data, please .