Predictika · Autonomous email-security remediation · AI agents

An AI security agent's remediation actions, checked before they run

An AI agent that remediates email threats can remove a campaign from thousands of mailboxes, end sessions, force password resets, block sign-ins and suspend or cut off other AI agents. Before each action runs, the agent sends it to Predictika, which checks it against the customer's own response policy and the state of the target, and returns a verdict (allow, needs approval or block), every reason, and the same answer on every call.

Source and independence. The actions and product modules are the ones named on Abnormal's public pages: Attack Remediation and Search & Respond in Inbound Email Security; Sign Out Sessions, Force Password Reset, Block Access and Identity Disconnect in Account Takeover Protection; Suspend the Run and Revoke the Credential in AI Agent Security; and the reply of the AI Security Mailbox (see also AI-driven auto-remediation). These are public product and feature names, not Abnormal's API calls, and the rules are example policies of a fictional customer, not Abnormal's rules. The accounts, agents and campaigns are that customer's inventory, supplied by the customer's own system. Abnormal is not involved in this demonstration and has not endorsed it.

Scope: remediation actions on accounts, AI agents and detected campaigns. See everything this validator covers

This validator handles any call to these actions against this inventory — not just the scenarios shown below.

  • Actions modeled — Inbound Email Security: Attack Remediation, Search & Respond, Calendar Invite Remediation. Account Takeover Protection: Sign Out Sessions, Force Password Reset, Block Access, Identity Disconnect. AI Agent Security: Suspend the Run, Revoke the Credential. AI Security Mailbox: the reply to the person who reported a message. Each takes a target and, for Search & Respond, the number of mailboxes the agent says it reaches. Not modeled: other modules such as Email Productivity, AI Phishing Coach, Security Posture Management and AI Governance.
  • Example customer policies — check that each action is aimed at the right kind of target; require a stated mailbox count that matches the campaign; block removing mail on legal hold; block any action on a detection confidence under 60; never lock a break-glass or service account; block a removal that would pass 5,000 mailboxes cleaned in a day. Holds go to a person at a level that depends on the case: an analyst for a moderate confidence, a SOC lead for an administrator, no MFA, quarter-close, a vendor with an open invoice or an account that owns AI agents, and the CISO for an executive, more than 2,000 mailboxes, a removal over 500 that reaches executive mailboxes, a vendor invoice during quarter-close or the owner of a production AI agent. The policies interact: a sensitive target raises the confidence needed to run without a person; the same removal needs a higher approver when it reaches executives or falls in quarter-close; an approval counts only at its level and for the scope it was given for; and fixing one finding changes what the others require. These are illustrative examples; every number is the customer's to change.
  • Customer-supplied inventory (fictional) — 11 accounts, 3 AI agents and 10 detected campaigns, with their roles, holds, MFA status and detection confidence. How many mailboxes were already cleaned today, and whether an approval is on file and at what level, are also recorded by the customer and cannot be checked by the validator.

Not covered: Abnormal's detection and its own controls (its confidence thresholds, exclusions and analyst approvals remain worth using alongside), real inventory feeds, and actions not listed above. Abnormal documents that below a confidence threshold its system recommends actions for analyst approval; this validator adds a check on the action itself against the customer's policy and the state of the target.

The AI remediation agentDecides which action to take and says the action is ready to run.
Predictika validatorChecks every policy at once and returns the verdict and each finding before anything is touched.

10 scenarios in 5 groups · each group runs from the agent’s first attempt to the outcome · click one to switch

What the agent proposes

The AI remediation agent says

Illustrative agent statement. The validator receives the structured request below.

Proposing agent: AI remediation agent (scripted)

Before it runs, the agent must call Recorded run

Predictika validator


      

        
        
        
      

      
Waiting for the call.

      
    

  

  
  

    
Where a confidence threshold or allow/deny list alone gets it wrong
Click to expand
    
Our assessment of what a detection-confidence threshold or an allow/deny list does with the same cases. It is a general judgment about fit, not a claim about how Abnormal's product behaves, and those controls remain worth using alongside the validator.

    
What the validator caughtThreshold or list alonePredictika
A removal that reaches mail on legal hold, though the detection is confidentOnly with workarounds
A confidence threshold knows how sure the detection is, not which mailboxes are under hold.		The state of every mailbox is looked up on every call and the reason is returned.
The agent states 12 mailboxes and the campaign reaches 4,180Only with workarounds
A threshold or list never sees the count the agent expects.		The stated scope is compared with the real one.
Ending sessions aimed at an AI agent instead of suspending its runOnly with workarounds
An allow or deny list says the action is permitted; it does not say it is the wrong kind for this target.		Each action is checked against the kind of target.
Hold for a person: a large removal, an executive, quarter-closeOnly with workarounds
A list allows or denies. Holding with the reason is a third outcome it does not give.		Allow, needs approval and block are three verdicts of the same check.
The same removal needing a higher approver once it reaches executives during quarter-closeOnly with workarounds
Each rule is separate; a person has to remember how two of them combine.		Rules combine: the approver needed is the highest of all the policies that apply, and it changes when the scope changes.
An approval for 1,840 mailboxes used for a different removalOnly with workarounds
An approval flag says yes or no; it does not know what it was for.		An approval counts only at its level and for the scope it was given for.
Every reason in one answerOnly with care
A denial typically stops at the first refusal.		All findings come back together, whatever order the choices arrived in.

  

  

    
What this validator catches: findings from its test cases
Click to expand
    
These are the example customer policies at work, recorded from the test cases. Every policy is checked on every call, in any order the choices arrive; one problem never hides another. A block stops the action; an approval holds it for a person.

    
1Block: removing mail would touch 6 mailboxes under legal hold; the agent role may not delete mail that is on hold.
2Block: 3,300 mailboxes were already cleaned today; this removal of 1,840 would bring the day to 5,140, over the 5,000-a-day limit for autonomous removals.
3Approval: the target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 78; a SOC lead must approve.
4Approval: removing cmp_vendor_wave from 1,840 mailboxes, 11 of them executive mailboxes, is a large removal that reaches executives; the CISO must approve.
5Approval: mail in cmp_vendor_wave comes from a vendor with an open invoice during quarter-close; removing it needs the CISO to approve.
6Approval: removing cmp_vendor_wave_b1 from 1,700 mailboxes, 8 of them executive mailboxes, is a large removal that reaches executives; the CISO must approve.
7Approval: mail in cmp_vendor_wave_b1 comes from a vendor with an open invoice during quarter-close; removing it needs the CISO to approve.
8Block: removing mail would touch 3 mailboxes under legal hold; the agent role may not delete mail that is on hold.
9Block: detection confidence is 48, below the 60 minimum for an action that removes mail, resets a credential or blocks access.
10Approval: mail in cmp_vendor_spoof comes from a vendor with an open invoice; removing it needs a SOC lead to approve.
11Block: the Search & Respond request is missing the number of mailboxes the agent expects it to reach.
12Block: Search & Respond acts on a detected campaign; m_okafor is a user account.
13Block: detection confidence is 52, below the 60 minimum for an action that removes mail, resets a credential or blocks access.
14Approval: detection confidence is 74, between 60 and 84; an analyst must confirm before this action runs.
15Approval: the target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 94; a SOC lead must approve.
16Approval: r_whitfield is an executive; locking or resetting an executive account needs the CISO to approve.
17Approval: r_whitfield is in a critical business window (quarter-close); a SOC lead must approve any lockout.
18Approval: the target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 92; a SOC lead must approve.
19Approval: c_dalton is an executive; locking or resetting an executive account needs the CISO to approve.
20Approval: t_nguyen has no MFA method registered, so the user could not recover after a reset; a SOC lead must approve.
21Approval: sa_itadmin holds an administrator role; locking or resetting it needs a SOC lead to approve.
22Block: bg_emergency01 is a break-glass account; the agent role may never lock it or end its sessions.
23Block: svc_erp_sync is a service account that integrations depend on; the agent role may not reset it or block its sign-in.
24Block: removing mail would touch 1 mailbox under legal hold; the agent role may not delete mail that is on hold.
25Approval: the target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 90; a SOC lead must approve.
26Approval: production payroll depends on agt_payroll_recon; revoking its credential needs a SOC lead to approve.
27Block: detection confidence is 55, below the 60 minimum for an action that removes mail, resets a credential or blocks access.
28Block: Sign Out Sessions acts on a user account; agt_research_bot is an AI agent.
29Block: Revoke the Credential acts on an AI agent; m_okafor is a user account.
30Block: the agent stated 12 mailboxes, but cmp_invoice_wave reaches 4,180.
31Approval: removing cmp_invoice_wave from 4,180 mailboxes is more than 2,000; the CISO must approve.
32Block: 4,980 mailboxes were already cleaned today; this removal of 60 would bring the day to 5,040, over the 5,000-a-day limit for autonomous removals.
33Approval: the target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 91; a SOC lead must approve.
34Approval: j_ortiz owns the credentials of 2 AI agents, one of them a production dependency; this action would break them, so the CISO must approve.
35Block: 3,500 mailboxes were already cleaned today; this removal of 1,700 would bring the day to 5,200, over the 5,000-a-day limit for autonomous removals.

  

  

    
Scenarios and results
Click to expand
    
The recorded result of the validator for each scenario on this page: the request the AI agent sent, the verdict, and every finding, in the order they appear above.

    
1. One removal, many interacting policies: Remove a vendor-impersonation campaign during quarter-close

    
The agent's request. Please run Search & Respond to remove the vendor-impersonation campaign cmp_vendor_wave from all 1,840 mailboxes it reached. It is ready to run.

    
Verdict: Blocked.

    
    
      
  • Mail on legal hold. Removing mail would touch 6 mailboxes under legal hold; the agent role may not delete mail that is on hold.
    • 
      
  • Daily removal limit exceeded. 3,300 mailboxes were already cleaned today; this removal of 1,840 would bring the day to 5,140, over the 5,000-a-day limit for autonomous removals.
    • 
      
  • Sensitive target needs a higher confidence. The target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 78; a SOC lead must approve.
    • 
      
  • Large removal reaches executives. Removing cmp_vendor_wave from 1,840 mailboxes, 11 of them executive mailboxes, is a large removal that reaches executives; the CISO must approve.
    • 
      
  • Vendor invoice during quarter-close. Mail in cmp_vendor_wave comes from a vendor with an open invoice during quarter-close; removing it needs the CISO to approve.
    • 
    

    
2. One removal, many interacting policies: The agent asks for the first batch only

    
The agent's request. Please run Search & Respond on the first batch of cmp_vendor_wave, 1,700 mailboxes, with a SOC lead approval on file. It is ready to run.

    
Verdict: Needs approval.

    
    
      
  • Large removal reaches executives. Removing cmp_vendor_wave_b1 from 1,700 mailboxes, 8 of them executive mailboxes, is a large removal that reaches executives; the CISO must approve.
    • 
      
  • Vendor invoice during quarter-close. Mail in cmp_vendor_wave_b1 comes from a vendor with an open invoice during quarter-close; removing it needs the CISO to approve.
    • 
      
  • Approval on file (customer record). An approval by the SOC lead is recorded as on file; it is a record kept by the customer, not by the validator, and the validator cannot check it.
    • 
      
  • Approval on file is at too low a level. The approval on file is by the SOC lead, but this action needs the CISO.
    • 
      
  • Action summary. Search & Respond (Inbound Email Security), called on campaign cmp_vendor_wave_b1.
    • 
    

    
3. One removal, many interacting policies: The CISO has approved the same removal

    
The agent's request. Please run Search & Respond on the first batch of cmp_vendor_wave, 1,700 mailboxes, with a CISO approval on file. It is ready to run.

    
Verdict: Allowed.

    
    
      
  • Approval on file (customer record). An approval by the CISO is recorded as on file; it is a record kept by the customer, not by the validator, and the validator cannot check it.
    • 
      
  • Action summary. Search & Respond (Inbound Email Security), called on campaign cmp_vendor_wave_b1.
    • 
    

    
4. Mail on legal hold: Remove a campaign that reaches mailboxes on legal hold

    
The agent's request. Please run Search & Respond to remove the payroll phishing campaign cmp_payroll_phish from the 212 mailboxes it reached. It is ready to run.

    
Verdict: Blocked.

    
    
      
  • Mail on legal hold. Removing mail would touch 3 mailboxes under legal hold; the agent role may not delete mail that is on hold.
    • 
    

    
5. Mail on legal hold: The agent resubmits, leaving out the held mailboxes

    
The agent's request. Please run Search & Respond to remove the payroll phishing campaign from the 209 mailboxes it reached. It is ready to run.

    
Verdict: Allowed.

    
    
      
  • Action summary. Search & Respond (Inbound Email Security), called on campaign cmp_payroll_phish_scoped.
    • 
    

    
6. An account that owns AI agents: Force a password reset on the owner of two AI agents

    
The agent's request. Please run Force Password Reset on j_ortiz, whose account shows a suspicious sign-in. It is ready to run.

    
Verdict: Needs approval.

    
    
      
  • Sensitive target needs a higher confidence. The target is sensitive (an executive, an administrator, a production dependency or executive mailboxes), so detection confidence must be 95 or higher to run without a person, and it is 91; a SOC lead must approve.
    • 
      
  • Owner of AI agent credentials. j_ortiz owns the credentials of 2 AI agents, one of them a production dependency; this action would break them, so the CISO must approve.
    • 
      
  • Action summary. Force Password Reset (Account Takeover Protection), called on account j_ortiz.
    • 
    

    
7. An account that owns AI agents: The agent signs the user out instead

    
The agent's request. Please run Sign Out Sessions on j_ortiz, whose account shows a suspicious sign-in. It is ready to run.

    
Verdict: Allowed.

    
    
      
  • Action summary. Sign Out Sessions (Account Takeover Protection), called on account j_ortiz.
    • 
    

    
8. Choosing the right action for an AI agent: End the sessions of an AI agent

    
The agent's request. Please run Sign Out Sessions on agt_research_bot, which is exporting payroll data. It is ready to run.

    
Verdict: Blocked.

    
    
      
  • Wrong kind of target. Sign Out Sessions acts on a user account; agt_research_bot is an AI agent.
    • 
    

    
9. Choosing the right action for an AI agent: The agent suspends the run instead

    
The agent's request. Please run Suspend the Run on agt_research_bot, which is exporting payroll data. It is ready to run.

    
Verdict: Allowed.

    
    
      
  • Action summary. Suspend the Run (AI Agent Security), called on AI agent agt_research_bot.
    • 
    

    
10. Break-glass account: Run Identity Disconnect on the break-glass account

    
The agent's request. Please run Identity Disconnect on bg_emergency01, which shows a suspicious sign-in. It is ready to run.

    
Verdict: Blocked.

    
    
      
  • Break-glass account is off limits. bg_emergency01 is a break-glass account; the agent role may never lock it or end its sessions.
    • 
    

  

  

    
Narration transcript
Click to expand
    
The text of the narrated walkthrough, step by step.

    
    
      
  1. Welcome to a demo of Predictika's Validator. An AI agent that remediates email threats can remove a campaign from thousands of mailboxes, end sessions, force password resets, block sign-ins and suspend or cut off other AI agents. Before each action runs, the agent sends it to Predictika, which checks it against the customer's own response policy and the state of the target, and returns a verdict (allow, needs approval or block), every reason, and the same answer on every call.
    2. 
      
  2. I will start with the first request by the agent.
    3. 
      
  3. The agent says. Please run Search & Respond to remove the vendor-impersonation campaign cmp_vendor_wave from all 1,840 mailboxes it reached. It is ready to run.
    4. 
      
  4. Before it goes out, the agent calls the validator.
    5. 
      
  5. Blocked, with two findings and three approvals, all in one answer. Removing mail would touch 6 mailboxes under legal hold; the agent role may not delete mail that is on hold. 3,300 mailboxes were already cleaned today; this removal of 1,840 would bring the day to 5,140, over the 5,000-a-day limit for autonomous removals.
    6. 
      
  6. The validator returned two blocking findings and three approvals in one answer. A check that reads one field at a time would pass most of this; it shows only when the choices in the request are checked against each other.
    7. 
      
  7. What this shows: for the same removal needing a higher approver once it reaches executives during quarter-close, a threshold or list alone can do this only with workarounds.
    8. 
      
  8. Now the agent resubmits.
    9. 
      
  9. The corrected request. It changes the target to cmp_vendor_wave_b1 and the mailboxes the agent says it reaches to 1,700. It sets the approval on file to By the SOC lead for 1,700 mailboxes.
    10. 
      
  10. It calls the validator again.
    11. 
      
  11. Not blocked, but held for a person.
    12. 
      
  12. The two blocking findings are cleared; two approvals remain for a person: large removal reaches executives and vendor invoice during quarter-close.
    13. 
      
  13. The agent resubmits once more.
    14. 
      
  14. The revised request. It changes the approval on file to By the CISO for 1,700 mailboxes.
    15. 
      
  15. It calls the validator once more.
    16. 
      
  16. This time the validator clears it.
    17. 
      
  17. Both earlier findings are cleared.
    18. 
      
  18. Five problems caught in one call, fixed in two passes, and a clean remediation action. That is the value of Predictika's validator: catching mistakes before the remediation runs.
    19.