OpenClaw Emergency Response Playbook: Incident Response for Compromise, High Bills, and Erratic Behavior

Why You Need an Incident Response Plan

If your OpenClaw agent starts misbehaving, you might have minutes to stop damage. Having a playbook means you react, not panic.

Common incidents:

• API bill spike to 10x normal
• Agent making requests to suspicious domains
• Repeated failed authentication attempts
• Agent ignoring SOUL.md boundaries
• Gateway not responding

Incident Response Playbook

Incident Type 1: Unexpected API Bill Spike

Symptoms:

• Daily API spend went from $5 to $50+
• Token usage drastically increased
• You notice it in the dashboard or provider email

Immediate Actions (within 5 minutes):

1. STOP the gateway immediately: openclaw stop
2. Check both provider dashboards:
   • Moonshot (Kimi): https://console.moonshot.cn/api-keys
   • Anthropic: https://console.anthropic.com/account/usage
   • OpenAI (if using): https://platform.openai.com/account/usage
3. Identify the spike: Was it in input tokens, output tokens, or API calls?
4. Check recent logs: openclaw logs --last 1h | grep -i error

Investigation (next 15-30 minutes):

1. Review session logs for loops:

   
   # Look for repetitive patterns
   openclaw logs --last 1h | grep -E "API call|error|retry"
   
   # Check for specific patterns
   - Same API call repeated 1000+ times
   - Error → retry → error loop
   - Exponential backoff not working
   

2. Check for injection or compromise:
   • Were recent messages unusual?
   • Are there API calls to unexpected endpoints?
   • Did the agent ignore tool policy restrictions?
3. Determine root cause:
   • Legitimate workflow change?
   • Prompt injection or compromise?
   • Runaway retry loop?
   • Bug in a recently added skill?

Recovery Actions:

1. Lower spending limits:

   
   openclaw config set moonshot.daily_limit '$5'
   openclaw config set anthropic.daily_limit '$10'
   

2. Disable problematic skills: If a new skill caused the spike, disable it
3. Restart with caution:

   
   openclaw start --monitor
   # Watch logs for 30 minutes before trusting it
   

Incident Type 2: Suspected Compromise or Prompt Injection

Symptoms:

• Agent ignoring SOUL.md boundaries
• Unexpected attempts to access blocked tools
• Sudden changes in behavior after a user message
• Agent trying to exfiltrate data or credentials

Immediate Actions (within 2 minutes):

1. STOP the gateway immediately: openclaw stop
2. Do NOT acknowledge the compromise to users yet — investigate first
3. Take a snapshot of logs and state:

   
   # Preserve evidence
   mkdir -p ~/incident-$(date +%s)
   cp -r ~/.openclaw/logs ~/incident-*/
   cp ~/.openclaw/config.yml ~/incident-*/
   docker ps -a > ~/incident-*/containers.txt
   

Investigation (next 30-60 minutes):

1. Review the suspicious message:
   • Does it contain prompt injection patterns?
   • Was it sent by a trusted user?
   • What was the agent's response?
2. Check execution logs:

   
   # What tools did the agent try to use?
   grep "tool_execution" ~/.openclaw/logs/session.log
   
   # What tool policies were violated?
   grep "policy_violation" ~/.openclaw/logs/session.log
   

3. Determine severity:
   • Low: Agent attempted blocked tool but was stopped by policy
   • Medium: Agent executed blocked tool, but limited data access
   • High: Agent exfiltrated credentials or sensitive data

Recovery Actions:

1. Revoke credentials (for HIGH severity only):

   
   # Revoke compromised credentials
   - API keys (Moonshot, Anthropic, OpenAI)
   - Telegram bot token
   - Matrix access tokens
   - OAuth tokens
   

2. Tighten SOUL.md boundaries: Add explicit rules about injection attempts
3. Tighten tool policies: Reduce allowed tools or hosts
4. Remove suspicious skills: If a skill was the vector, remove it
5. Restart and monitor:

   
   openclaw start --monitor
   # Watch for 2-3 hours of normal operation before trusting it
   

Incident Type 3: Erratic Behavior (Agent Acting Weird)

Symptoms:

• Agent responses are incoherent or contradictory
• Agent forgets context mid-conversation
• Agent makes logical errors it normally doesn't
• Agent is slower than usual

Immediate Actions:

1. Check system resources:

   
   # Is the machine running low on CPU, memory, or disk?
   top
   df -h
   iostat
   

2. Check gateway logs:

   
   openclaw logs --filter error | head -20
   

3. Check model connectivity:

   
   # Are API calls succeeding?
   grep "api_error|timeout|rate_limit" ~/.openclaw/logs/*.log
   

Common Causes & Solutions:

• Model is down/rate-limited: Check provider status page, wait 1-5 minutes
• Low disk space: Clean up logs, restart
• Memory leak in agent: Restart the gateway
• Network connectivity issue: Check firewall, routing, DNS

Incident Type 4: Gateway Won't Start

Symptoms:

• openclaw start hangs or immediately exits
• Port already in use
• Config file errors

Troubleshooting:

1. Check logs:

   
   openclaw logs --follow
   # Look for specific error messages
   

2. Verify config syntax:

   
   openclaw validate-config
   

3. Check port availability:

   
   lsof -i :3000
   # Kill the process if it's stuck
   kill -9 <PID>
   

4. Try a clean start:

   
   openclaw reset  # Warning: clears session history
   openclaw start
   

Incident Severity Matrix

Post-Incident Checklist

After any incident, follow this checklist:

1. Document what happened: Timeline, root cause, impact
2. Update SOUL.md and tool policies: Prevent the same incident
3. Review security posture: Was defense-in-depth effective?
4. Improve monitoring: Would earlier detection have helped?
5. Test fixes: Verify changes prevent the issue
6. Share learnings: Communicate to your team

Key Takeaways

1. Speed matters — Stop the gateway before damage scales
2. Preserve evidence — Save logs and state before restarting
3. Investigate root cause — Don't just restart and hope
4. Tighten controls after incidents — Use each incident to improve defenses
5. Test your playbook — Run drills quarterly so you know what to do

Resources & Automation

Consider automating incident response:

• Alerting: Set up alerts for bill spikes, tool policy violations, and error rates
• Auto-mitigation: Automatically lower spending limits if bill spikes
• Incident recording: Automatically save logs when incidents occur