Authorized and safe
Use Microsoft’s platform, approved content, controlled domains and benign outcomes. Never request real credentials, install software, imitate highly sensitive personal crises, or target people outside authorization.
Authorized exercises, safe targeting, payloads, training, reporting, and remediation
Defender for Office 365 Attack Simulation Operations Guide
Operate phishing simulations as measured security exercises—not surprise punishment. Establish executive, legal, privacy, HR, communications, and help-desk governance; select techniques and payloads from current threats; validate target populations; randomize and localize delivery; assign meaningful training; interpret clicks and compromise signals carefully; reward reporting; and turn results into technical and process remediation.
Defender for Office 365 Plan 2, roles, audit, privacy, and approvalsTechniques, payloads, targets, region-aware delivery, languages, and trainingFalse clicks, reporting latency, repeat offenders, metrics, remediation, and evidence
Operating objective
Measure and improve reporting behavior without creating fear, deception harm, or misleading metrics
Attack Simulation Training in Microsoft Defender for Office 365 Plan 2 delivers benign social-engineering exercises and training. It can reveal control, process, and awareness gaps before a real campaign, but the data is not a complete measure of an employee’s security value. Email gateways, URL inspection, plugins, automated triage, forwarding, client behavior, reporting configuration, language, accessibility, and out-of-office status can materially affect results.
Use simulations to answer operational questions: Do users recognize and report likely phishing? Does reporting reach the right security queue? Are Safe Links, Safe Attachments, mail-flow, identity and endpoint controls behaving as designed? Does the organization respond to a simulated compromise quickly? Do high-risk workflows need technical safeguards? Does training reduce repeat behavior over comparable exercises?
Credential Harvest simulations do not collect, log, or store the information a user types; Microsoft records the click/compromise event and silently discards entered values. That protection does not remove the need for advance governance, minimum-access reporting, retention awareness, employee communications, accessibility, accommodations, labor/works-council or legal review where applicable, and a non-punitive escalation model.
Control statement: Every simulation must have written authorization, a safe business objective, technique/payload rationale, validated targets and exclusions, privacy and communications boundaries, incident/help-desk coordination, training plan, success measures, data-quality checks, stop conditions, evidence owner, retention decision, and remediation commitments.
Program governance
Build trust before measuring behavior
Privacy and fairness
Restrict result access, define the purpose, minimize exports and PII, document retention, support accessibility/language needs, and prohibit public shaming or unreviewed disciplinary use.
Operationally coordinated
Prepare SOC, help desk, communications, HR and leadership; distinguish simulation reports from real phishing; preserve a way to stop; and never suppress a genuine incident because an exercise is running.
Improvement focused
Reward correct reporting, assign role-relevant training, fix technical controls, compare like-for-like cohorts/techniques, and verify remediation. A click is a learning signal, not the final outcome.
Technique decision matrix
Choose the safest exercise that answers the real risk question
| Technique | What it can test | Special safeguards | Likely data-quality issues | Follow-up |
|---|---|---|---|---|
| Credential Harvest | Recognition of a sign-in lure, link handling, reporting and credential-protection behavior. | Use a clearly controlled login page; communicate that typed values are discarded; never collect secrets; consider passwordless/MFA context and employee sensitivity. | Security scanners can click links; password managers/client protections alter behavior; users may abandon before entry. | Strengthen phishing-resistant MFA, branded sign-in education, reporting and conditional access—not only training. |
| Malware Attachment | Attachment suspicion, open behavior, reporting, endpoint/mail control interaction and training. | Use only Microsoft-safe simulation payloads; coordinate Safe Attachments/endpoint tooling and incident queues; avoid executable harm. | Automated detonation or preview may create events; client support differs; forwarded messages receive normal protection. | Improve Safe Attachments, file-type/application controls, endpoint hardening and quarantine operations. |
| Link in Attachment | Layered lure behavior: opening a benign document then following a link to a controlled page. | Test supported clients, accessible content and safe landing page; coordinate both attachment and URL security telemetry. | Previewers, Safe Attachments and Safe Links can create signals; users may open but not click. | Correlate both stages and reinforce document/link verification plus technical controls. |
| Link to Malware | Recognition/reporting of a link that represents a malicious-download path. | No real malware; use platform-controlled payload; prepare endpoint/SOC telemetry and user support. | URL inspection, browser isolation and endpoint tools can click/block before the user. | Improve Safe Links, browser/endpoint policy, application control and download handling. |
| Drive-by URL / QR scenario | Link/QR skepticism, mobile context, destination verification and reporting outside a conventional anchor. | Use safe hosted destinations, test mobile/accessibility, avoid collecting device data beyond program need, and provide reporting guidance. | Mobile/email clients and QR scanners vary; proxy or camera preview may trigger network activity. | Improve mobile reporting, browser protection, QR education and high-risk transaction verification. |
| OAuth Consent Grant | Recognition of risky application-consent requests and escalation behavior. | No real illicit permission; align with Entra consent policy, admin workflow, approved apps and privacy review. | Tenant/user-consent settings can block the path before user behavior is observed. | Strengthen consent governance, verified publishers, app review, OAuth monitoring and incident response. |
| How-to Guide / training-only | Skills practice or targeted education without measuring deceptive message interaction. | Use when risk, accessibility, employee relations or recent incidents make a simulation inappropriate. | Completion does not prove future behavior, but it can prepare users for later measurement. | Verify understanding with role-based exercises, reporting drills and technical tests. |
Twelve-step operations runbook
Authorize, validate, launch, monitor, analyze, remediate, and retest
Define the risk hypothesis
State the behavior, workflow or control to evaluate; affected business process; comparable baseline; target improvement; prohibited outcomes; and why a simulation is safer/more useful than training-only or a technical test.
Confirm licensing and roles
Verify Defender for Office 365 Plan 2 eligibility, least-privileged simulation/report permissions, audit logging, active licensing needed for report details, data region/retention and approved administrators.
Complete governance review
Obtain security, leadership, HR/legal/privacy, communications, accessibility and support approvals appropriate to the organization. Define result access, retention, accommodations, escalation and non-punitive use.
Select technique and payload
Use threat-relevant, culturally appropriate, accessible content; avoid extreme personal/emotional lures; preview every link, attachment, login/landing page, language and notification; document source and custom content owner.
Validate targets
Resolve Entra groups or CSV/users, review membership at save time, remove guests/inactive/unlicensed/unsupported shared mailboxes, protect leave/termination/medical/incident cases, create control cohort and record the final population.
Design training and reinforcement
Choose Microsoft-recommended or role-relevant modules, due dates, training threshold, positive reinforcement for reporters, landing-page indicators and manager/help-desk support. Avoid repeatedly assigning the same training.
Schedule realistic delivery
Use region-aware delivery or randomized automation windows, multiple payloads, business-hour context and out-of-office awareness. Prevent simultaneous blasts that encourage users to warn each other or overload support.
Prepare operations
Test reporting add-ins/mailboxes, mail-flow rules, audit, SOC/help desk, alerts and escalation. Ensure reported simulation mail reaches Microsoft and distinguish a real incident occurring during the campaign.
Send test and authorize launch
Verify delivery, rendering, links, landing/training pages, notifications, localization, safe behavior and support script with a small approved test. Confirm stop/cancel authority and final signoff.
Monitor without contaminating
Watch delivery failures, unexpected targeting, complaints, data-quality anomalies, tooling clicks and real threats. Do not announce clues during the exercise unless safety or business impact requires intervention.
Analyze with context
Wait for reporting updates, separate scanner/tool activity, compare comparable cohorts, consider forwarding/replies/out-of-office/on-prem limitations, review report quality and interpret report behavior as strongly as click/compromise signals.
Remediate and retest
Deliver positive reinforcement and training, fix reporting/mail controls, strengthen identity/endpoint/app safeguards, assign owners/dates, verify completion and schedule a comparable future exercise to measure improvement.
Targeting, delivery, training, and data quality
Control variables before comparing people or campaigns
Target resolution
Guests and inactive Entra users are removed during validation; shared mailboxes are unsupported. Groups are expanded when the simulation/automation/training campaign is saved. Export and approve the resolved list, not the intended group name.
Localization and accessibility
Payload language, mailbox locale notifications, browser/account language landing/training pages and persistent training-language choices can differ. Test screen readers, mobile, cognitive load and accommodation workflows.
Randomization
Use region-aware delivery and automation scheduling, varied payloads and staggered timing to reduce peer warning and support spikes. Preserve schedule evidence so cohorts remain comparable.
Training threshold
The platform can suppress repeated assignment of the same training during the configured threshold. Confirm whether missing training is expected rather than treating it as a delivery failure.
Tool-generated clicks
Security gateways, Outlook plugins, endpoint tools and SOAR playbooks can inspect links/content and create fast click/compromise events. Review timestamps, IPs and patterns; do not punish users for automation.
Reporting latency and limits
Reports populate after launch and continue updating; on-premises mailbox reporting has reduced read/forward/delete/report details. Use built-in simulation reports as the primary source and document unavailable fields.
Top attack-simulation risks and misconfigurations
Failures that damage trust, expose sensitive data, or produce invalid conclusions
No written authorization
A campaign launches without accountable owner, HR/legal/privacy review, communications boundary or stop authority.
Simulation used as punishment
Results are published or used for discipline without context, fairness review, accommodations or improvement support.
Resolved target list not reviewed
Group changes, inactive users, sensitive cases, executives, new hires or unsupported recipients receive unintended content.
Payload causes real harm
A lure imitates trauma, payroll/benefits or urgent personal events beyond the approved risk and employee-support boundary.
Security-tool clicks blamed on users
Automated scanners, plugins or SOAR generate interaction signals that are treated as human compromise.
Raw click rate is the only KPI
Reporting, time-to-report, technical controls, cohort difficulty, out-of-office, language and remediation are ignored.
Real credentials requested
Custom content collects secrets or sends users to an uncontrolled destination instead of the platform’s discard behavior.
Training duplicates or does not fit
Threshold behavior, language, role and accessibility are ignored, producing fatigue rather than skill improvement.
Reporting pipeline is broken
Mail-flow, custom mailbox or non-Microsoft tooling prevents reported simulation messages from reaching Microsoft/reporting.
No remediation verification
Campaigns repeat, but identity, email, endpoint, reporting and workflow gaps remain unowned and untested.
Evidence and cadence
Measure learning, reporting, control performance, and remediation
Exercise evidence
- Authorization, objective, approvals and privacy boundary
- Technique, payload, pages, language and training
- Resolved targets/exclusions, schedule and control cohort
- Test results, launch/stop decisions and operations log
- Raw export, data-quality review and retention/deletion record
Outcome evidence
- Delivery, read, click, compromise and report behavior
- Time-to-report, reply/forward/delete/out-of-office
- Training assignment/completion and positive reinforcement
- Scanner/tool false-signal findings and on-prem limitations
- Technical/process remediation, owner, due date and verification
Useful metrics
- Report rate and median time to report
- Compromise/click rate normalized by comparable technique
- Tool-generated signal and delivery-failure rate
- Training completion and repeat behavior trend
- Remediation closure and retest improvement
Per exercise
Approve, validate, test, monitor, analyze, remediate, communicate and preserve evidence.
Monthly/quarterly
Review automations, target groups, content, training thresholds, report pipeline, permissions and remediation backlog.
Program review
Compare normalized trends, repeat offenders with context, technical gaps, accessibility/privacy and stakeholder trust.
Event-driven
Pause or adapt for real incidents, workforce events, organizational change, vendor/tool changes or data-quality anomalies.
Authoritative resources and related guidance
Connect simulations to reporting, training, email controls, and incident response
Microsoft implementation sources
Frequently asked questions
Defender for Office 365 Attack Simulation FAQ
Does a Credential Harvest simulation store passwords users enter?
No. Microsoft states that information entered on the simulation sign-in page is silently discarded; only the click is recorded as the compromise event. Custom content must never collect real secrets.
Can shared mailboxes be targeted?
No. Microsoft currently documents shared mailboxes as unsupported. Target user mailboxes or supported groups of user mailboxes and approve the resolved membership before launch.
Why do clicks sometimes appear immediately after delivery?
Email security gateways, Outlook plugins, endpoint tools or SOAR playbooks may inspect simulation links/content. Review click timestamp, IP and patterns before attributing the event to a person.
Does replying or forwarding remain inside the simulation protection path?
Microsoft says replies/forwards are treated like normal email. Forwarded messages can be detonated by Safe Links or Safe Attachments, and each simulation URL is tied to an individual user, so automated detonation can appear as that user’s click.
How long is simulation data retained?
Microsoft documents 18-month retention for simulation metadata, automation, tenant payloads/pages/notifications and user activity unless an administrator deletes the relevant object first; some recommended/global content has different retention.
What is a better success measure than click rate alone?
Use reporting rate and time-to-report, comparable technique/cohort trends, data-quality adjustments, training completion, technical-control findings, remediation closure and verified improvement in a later exercise.
Train, measure, remediate, and retest
Build an attack-simulation program that improves security without damaging trust
IT Perfection helps Orange County and Southern California organizations govern Defender for Office 365 Attack Simulation Training, validate targets, choose safe threat-relevant techniques, coordinate stakeholders, randomize/localize delivery, improve reporting and training, correct false signals, measure outcomes, and verify technical and process remediation.
Created by Ali Hassani, CISO — 25+ years of IT, cybersecurity, compliance, and infrastructure experience. This guide is for initial guidance only and does not replace a professional cybersecurity audit, compliance assessment, penetration test, incident investigation, legal/privacy/HR review, labor consultation, employee-relations advice, or tested incident-response program.
