The malicious use of AI is the deliberate use of artificial intelligence tools by hostile actors to plan, scale, or automate harmful activity. It now spans far more than cyber. The same models that help organisations detect vulnerabilities and write policy are being used to clone voices, generate convincing deepfakes, profile staff and facilities, and lower the skill required to run a sophisticated attack. For security and resilience teams, the practical point is straightforward: AI does not invent new threats so much as make familiar ones faster, cheaper, and harder to spot.
This shift matters because the barrier to entry has dropped. An actor who once needed technical skill, time, and money can now buy or prompt a tool that does much of the work. That changes the risk calculation for every organisation, not only those with a large cyber footprint.
Key Takeaways
- The malicious use of AI cuts across cyber, physical, and personal information security, so it belongs in an all-hazards security risk assessment, not only in the IT risk register.
- A wide range of actors use AI, including nation-states, organised crime, violent extremists, insiders, and low-skilled opportunists, each lowering their costs and raising their reach.
- Deepfakes and voice cloning are now a mainstream fraud and social-engineering method, with one Hong Kong case costing a single firm about 25 million US dollars after a fake video call.¹
- Whether attackers or defenders benefit more from AI remains genuinely uncertain, which is why layered controls and trained people matter more than any single tool.²
- The most effective response combines awareness training, strong verification procedures, and a security risk assessment that treats AI as a cross-cutting threat enabler.
Which threat actors are using AI, and how?
It helps to name the actors, because their goals differ and so do the controls that work against them.
Nation-states use AI for espionage and disruption. In November 2025, Anthropic reported what it described as the first documented large-scale cyber espionage campaign run mostly by an AI system, attributing it to a Chinese state-sponsored group and stating that the AI carried out an estimated 80 to 90 per cent of the operation with limited human direction.³ Some security researchers have questioned the framing, but the direction of travel is clear: state actors are testing how far automation can take an operation.
Organised crime uses AI to industrialise fraud. Criminal markets now sell purpose-built “dark” language models, stripped of safeguards, that write phishing emails, draft malware, and script scam calls at volume. These tools let offenders with little technical skill run operations that once needed a team.
Violent extremists and terrorist groups use generative AI for propaganda, recruitment, and operational planning. Counter-terrorism researchers have tracked a steady rise in AI-generated extremist content since 2023, including material aimed at younger audiences.
Insiders and infiltrators use AI to get inside the organisation in the first place. North Korean operatives have used AI-generated identities, altered headshots, and real-time deepfakes in video interviews to win remote roles at Western firms and funnel wages and stolen data back to the regime.⁴
Opportunistic fraudsters use whatever is cheapest. Voice cloning can be set up for a few dollars from a short audio sample, which is why “family in distress” and executive-impersonation scams have spread so quickly.
How is AI being used in cyber operations?
On the cyber side, AI assists at almost every stage. It speeds up reconnaissance and vulnerability discovery, drafts more convincing phishing lures in fluent local English, and helps generate or adapt malicious code. The Australian Signals Directorate assessed in its Annual Cyber Threat Report 2024 to 2025 that AI is “almost certainly” enabling malicious actors to launch attacks at greater scale and speed, against a backdrop of more than 1,200 incidents handled in the year and a rising average cost of cybercrime to business.⁵
The same capability cuts the other way for defenders, who use AI for faster detection, triage, and patching. That is the central tension of this topic, and it is unresolved.
What does AI mean for physical security?
This is where many organisations underestimate the exposure. AI is a powerful aid to hostile reconnaissance, the planning phase that precedes most deliberate physical attacks.
Open-source intelligence that once took days can be assembled in minutes. Models can aggregate public imagery, social media, corporate filings, and mapping data to profile a site, identify key staff, infer routines, and surface weak points in access control or guarding. Facial recognition and image tools can help an actor identify and track an individual from a single photograph. Social engineering then becomes more convincing: a cloned voice of a manager instructing a guard, a deepfaked email approving access, or a synthetic identity that passes a rushed screening check.
The defensive response is not new, but AI raises the stakes. Crime Prevention Through Environmental Design, disciplined visitor and contractor management, alertness to pretexting, and robust workforce screening all reduce the value an attacker can extract from AI-assisted planning. These belong in any current building and physical security review.
How are people and their personal information being targeted?
The personal information dimension is the fastest-moving of the three. Deepfakes and voice cloning have turned identity itself into an attack surface.
The clearest example remains the Hong Kong case in which a finance employee made fifteen transfers totalling about 25 million US dollars after joining a video call where the chief financial officer and several colleagues were all AI-generated fakes.¹ In the United States, the FBI reported that AI-related scams cost victims at least 893 million US dollars in 2025, the first year its national fraud report broke out AI as a category.⁶ In Australia, reported scam losses reached 2.18 billion dollars in 2025, up 7.8 per cent on the previous year, with AI-enabled voice and video deception a growing share.⁷
For an organisation, the lesson is procedural. Any instruction to move money, grant access, or release data needs a verification step that does not rely on a voice or a face alone, because both can now be faked convincingly.
Does AI favour attackers or defenders?
Honestly, it is too early to say. The International AI Safety Report concluded that whether attackers or defenders will benefit more from AI assistance remains uncertain.² What is clear is that AI lowers the barrier for criminals while also giving defenders genuine new capability. The organisations that fare best will be the ones that treat AI as a threat enabler across all three domains and adjust their controls accordingly, rather than waiting for certainty that will not arrive soon.
How Agilient Can Assist
The malicious use of AI does not call for a single product. It calls for a clear-eyed view of how an organisation could actually be targeted, and a set of controls matched to that picture.
Agilient helps organisations treat AI as a cross-cutting threat enabler within an all-hazards security risk assessment, so that cyber, physical, and personal-information exposures are considered together rather than in silos. For government and regulated clients, this connects directly to protective security and PSPF obligations around personnel, physical, and information security. Where the risk is people being deceived, security awareness training gives staff the verification habits that stop deepfake and voice-cloning fraud. And because disruption is a question of when, not if, Agilient’s business resilience programs keep operations running through an incident.
Agilient is independent and vendor neutral, and works with government, healthcare, aviation, defence, and other regulated sectors across Sydney, Melbourne, Brisbane, Adelaide, and Canberra.
A practical first step for any board is to ask one question: if someone cloned our chief executive’s voice tomorrow, which of our payment, access, and data-release processes would catch it?
Request a security risk assessment or book a short briefing with Agilient.
Frequently Asked Questions
What is the malicious use of AI?
The malicious use of AI is the deliberate use of artificial intelligence by hostile actors to plan, scale, or automate harmful activity. It includes generating phishing and malware, creating deepfakes and cloned voices, profiling people and facilities for reconnaissance, and lowering the skill needed to run sophisticated attacks.
Is AI only a cyber security problem?
No. While cyber is the most discussed domain, AI is equally relevant to physical security, where it aids hostile reconnaissance and social engineering, and to personal information security, where deepfakes and voice cloning enable fraud and impersonation. It is best addressed as a cross-cutting threat in an all-hazards risk assessment.
Which threat actors use AI maliciously?
Nation-states use it for espionage and disruption, organised crime for fraud at scale, violent extremists for propaganda and planning, insiders and infiltrators to gain access under false identities, and opportunistic fraudsters for low-cost scams such as voice cloning.
How can organisations defend against AI-enabled deepfake fraud?
The most effective control is procedural. Any instruction to move money, grant access, or release sensitive data should require verification through a separate, trusted channel that does not rely on a voice or video alone. This should be supported by staff awareness training and clear escalation rules.
Does AI favour attackers or defenders?
It remains uncertain. AI lowers the barrier to entry for attackers while also giving defenders stronger detection and response. The practical conclusion is to assume both sides are improving and to maintain layered controls rather than rely on any single tool.
Where should an organisation start?
Start with a security risk assessment that explicitly considers AI-enabled threats across cyber, physical, and personal-information domains, then prioritise the gaps it reveals, such as weak verification procedures, thin awareness training, or exposed reconnaissance data.
References
- CNN, Finance worker pays out $25 million after video call with deepfake chief financial officer, cnn.com
- International AI Safety Report 2026, Cyber offence and defence balance assessment, internationalaisafetyreport.org
- Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign, anthropic.com
- Microsoft Threat Intelligence, Jasper Sleet: North Korean remote IT workers’ evolving tactics to infiltrate organisations, microsoft.com
- Australian Signals Directorate, Annual Cyber Threat Report 2024-25, cyber.gov.au
- Federal Bureau of Investigation, Internet Crime Report 2025, Internet Crime Complaint Center (IC3), ic3.gov
- National Anti-Scam Centre, Targeting Scams: Report of the National Anti-Scam Centre 2025 (released March 2026), nasc.gov.au
