Running impact analysis

What it answers

"If I move this policy from report-only to enforced (or change its conditions), who would have been blocked over the last N days?"

This is the question that should precede every CA enforcement change. Policytab computes it on-demand from Graph sign-in logs and caches the summary for 10 minutes per (policy × window).

How to run

1. Tenant detail → Impact.
2. Pick a policy from the dropdown (sourced from the latest snapshot).
3. Pick a window - 1, 7, 14, or 30 days. Microsoft caps sign-in retention at 30 days.
4. Click Run impact analysis. Policytab calls the graph-signin-impact Edge Function, which paginates through /v1.0/auditLogs/signIns filtered to this policy + window, aggregates in memory, and stores only the summary.

How to read the report

• Total sign-ins - every sign-in that touched the policy (regardless of result) in the window
• Unique users - distinct user IDs across those sign-ins
• Would block - sign-ins with result failure OR reportOnlyFailure. The critical number: this is the count of sign-ins that would have been blocked if the policy were enforced. Read this twice before flipping a policy from report-only to enforced.
• By result - full breakdown (success / failure / notApplied / reportOnlySuccess / reportOnlyFailure / etc.)
• Top affected users - ranked by would-block desc. Hover over the GUID for the full id.
• Top affected apps - same ranking, by app
• Top failure reasons - Microsoft's failure reason text aggregated across blocked events

What the cache buys you

A re-run within 10 minutes returns the cached result instantly. Click Re-run to force fresh. The cache key is impact:policy:<policy-id>:days:<window> so different windows are independently cached.

What we DON'T store

We never persist raw sign-in events. Only the aggregate is stored in signin_summary_cache and that row auto-expires after 10 minutes. No PII beyond what's in the aggregate (user GUID + display name).

Limits

• Microsoft caps auditLogs/signIns retention at 30 days. The 30-day window is the maximum.
• Tenants without Entra ID P1 cannot use the sign-in logs API - Impact will error out for those tenants. The UI surfaces this via the CapabilityGate.
• Very high-volume tenants (>10k sign-ins for one policy in a window) may take 10-30s to fetch. Cached subsequent runs are instant.