Why environment separation matters
Deploying untested agents directly to production is a reliability and compliance risk. Environment separation (dev, staging, production) enables testing changes before they affect customers, validating accuracy with production-like data, and rolling back safely when issues occur.
Good environment management uses dev for building and experimentation, staging for validation with production-like data, and production for customer-facing workloads. Each environment has different credentials, spending limits, and approval rules.
Migration process
1. Build and test in dev
Use dev environment for building new agents and testing changes. Dev has relaxed approval rules, low spending limits, and uses test data or anonymized production data. Iterate quickly in dev without affecting production.
2. Promote to staging with environment-specific settings
Export configuration from dev and import to staging. Replace dev credentials with staging credentials, adjust spending limits to match expected staging usage, and configure approval rules to match production.
3. Validate in staging with production-like data
Run the agent in staging for at least 100 executions or one week. Use production-like data. Measure accuracy, approval rate, and failure rate. Target 95%+ accuracy before promoting to production.
4. Backup current production configuration
Before promoting to production, backup the current production configuration. This enables fast rollback if the new configuration causes issues.
5. Promote to production with monitoring
Export configuration from staging and import to production. Replace staging credentials with production credentials, configure production spending limits, and enable production approval rules. Monitor closely for the first 24 hours.
6. Maintain rollback capability
Keep the previous production configuration accessible for immediate rollback. If the new configuration causes issues, roll back to the previous version immediately.
Best practices
- Never deploy untested agents to production. Always test in dev and validate in staging before promoting to production.
- Use environment-specific settings. Different credentials, spending limits, and approval rules for each environment.
- Validate in staging with production-like data. Run for 100 executions or one week. Target 95%+ accuracy.
- Backup production configuration before migration. Enable fast rollback if issues occur.
- Monitor closely after production deployment. Watch approval rate, failure rate, and execution volume for the first 24 hours.
Frequently asked questions
Should we test agents in dev before deploying to production?
Yes. Build and test in dev, validate in staging with production-like data, then deploy to production. Never deploy untested agents directly to production.
Can we use the same configuration across all environments?
No. Use environment-specific settings for credentials, spending limits, and approval rules. Dev has low limits and relaxed approval, production has production limits and strict approval.
How long should we run agents in staging before promoting to production?
Run in staging for at least 100 executions or one week, whichever comes first. Validate 95%+ accuracy before promoting to production.
What if migration breaks production?
Maintain the previous production configuration as a rollback point. If migration causes issues, roll back to the previous version immediately.