This framework covers the full interview lifecycle for Principal AI Architect, Sr. AI Architect, and Quant Trading Platform Architect roles. Based on real interview patterns from Fortune 500 companies, FAANG, and financial institutions. The Hermes AI Trading Platform serves as the reference case study throughout.

Interview Stages

1. Design a multi-LLM gateway with failover, rate limiting, and cost tracking.
2. How do you choose between fine-tuning vs RAG for a given use case?
3. Design a prompt management system with versioning, A/B testing, and guardrails.
4. How would you architect an agent orchestration platform handling 1000+ concurrent conversations?
5. Design a model evaluation framework that catches regressions before deployment.
6. How do you handle context window limitations in long-running agent sessions?
7. Design a system for real-time monitoring of LLM output quality and cost.
8. How would you implement tool/function calling with strict schema validation?
9. Design a multi-agent system where agents delegate sub-tasks to each other.
10. How do you handle PII redaction across multiple LLM providers?
11. Design a caching strategy for LLM responses to reduce cost by 60%.
12. How would you architect a system that uses different LLMs for different tasks?
13. Design a system for continuous model improvement based on user feedback.
14. How do you handle model deprecation when a provider sunsets an API?
15. Design a system for secure multi-tenant AI agent access.
16. How would you implement streaming responses from an agentic workflow?
17. Design a knowledge base RAG system that handles 10M+ documents.
18. How do you evaluate if an AI system is production-ready?
19. Design a system for automated prompt engineering and optimization.
20. How would you handle hallucination detection and mitigation in production?
21. Design a credit-based usage system for internal AI API consumption.
22. How do you manage secrets and API keys across multiple LLM providers?
23. Design a system for cross-region failover of AI services.
24. How would you implement a human-in-the-loop approval workflow?
25. Design a feature store for ML models used across an organization.
26. How do you handle bias detection and fairness in AI systems?
27. Design a system for automated prompt injection detection.
28. How would you architect a system that combines structured and unstructured data?
29. Design a MCP (Model Context Protocol) server ecosystem strategy.
30. How do you measure and optimize AI system latency for real-time use cases?

1. Design a real-time trading system that processes market data and executes orders.
2. How do you architect a backtesting engine that supports walk-forward validation?
3. Design a risk management system that enforces position limits across multiple strategies.
4. How would you implement circuit breakers in an order execution pipeline?
5. Design a market data pipeline that handles 1M+ messages per second.
6. How do you handle broker API rate limits and outages?
7. Design a portfolio optimization system that tracks correlation across strategies.
8. How would you implement slippage modeling in a trading system?
9. Design a system that supports multiple brokers with failover.
10. How do you handle tax-aware trading (wash sales, tax-loss harvesting)?
11. Design a real-time P&L tracking system across multiple accounts.
12. How would you implement Kelly criterion position sizing?
13. Design a system that prevents duplicate orders across redundant systems.
14. How do you handle order book reconstruction from raw market data?
15. Design a system for strategy performance attribution.
16. How would you implement A/B testing of strategies in production?
17. Design a system for automated strategy discovery using ML.
18. How do you handle time synchronization in distributed trading systems?
19. Design a system for post-trade analytics and reporting.
20. How would you implement a trading simulator for what-if analysis?
21. Design a system that detects and prevents runaway trading.
22. How do you handle fractional share execution across different brokers?
23. Design a system for automated rebalancing of multi-strategy portfolios.
24. How would you implement dark pool vs lit exchange routing?
25. Design a system for regulatory reporting (MiFID II, SEC rules).

1. Design a multi-region active-active architecture for a trading platform.
2. How do you implement secret management at scale?
3. Design a CI/CD pipeline for ML models and trading strategies.
4. How would you architect a system for SOC2/HIPAA compliance?
5. Design a monitoring system that covers 500+ microservices.
6. How do you handle data retention and archival for regulatory compliance?
7. Design a cost governance system for multi-account AWS organizations.
8. How would you implement disaster recovery with RTO < 5 minutes?
9. Design a system for automated dependency scanning and CVE remediation.
10. How do you handle container image security in a Kubernetes cluster?
11. Design a logging strategy that costs < $1000/month for 100TB/month.
12. How would you implement canary deployments for trading strategies?
13. Design a system that prevents credential leakage in git repositories.
14. How do you handle database schema migrations in a zero-downtime system?
15. Design a system for automated backup verification and restore testing.
16. How would you implement rate limiting across a distributed system?
17. Design a network architecture that isolates trading environments.
18. How do you handle DNS resolution and service discovery in Kubernetes?
19. Design a system for automated capacity planning and scaling.
20. How would you implement a chaos engineering program for trading systems?

1. Tell me about a time you had to convince stakeholders to adopt a new technology.
2. Describe a situation where your architecture failed in production.
3. How do you balance speed of delivery with architectural quality?
4. Tell me about a time you mentored a team member who was struggling.
5. Describe a situation where you had to make a trade-off between cost and performance.
6. Tell me about a time you managed a production incident.
7. How do you stay current with rapidly evolving AI technology?
8. Describe a situation where you had to say no to a stakeholder request.
9. Tell me about a time you influenced a team to adopt better engineering practices.
10. How do you handle disagreements with your manager about technical direction?
11. Describe a project you led that required cross-team collaboration.
12. Tell me about a time you had to learn a new technology quickly.
13. How do you approach technical debt in a fast-moving environment?
14. Describe a situation where you had to deliver under an aggressive timeline.
15. Tell me a time you identified and fixed a critical security vulnerability.

Requirements: Must support 1000+ strategies, process market data in real-time, deliver alerts via Telegram/Discord/SMS with < 1-second latency, handle broker API rate limits, survive broker outages.

Key Decisions:

• Data flow: Market data → strategy engine → signal processor → risk gate → executor → broker
• Back-pressure: Each stage has bounded queues. If broker is slow, signals queue up instead of dropping.
• Circuit breaker: If 3 consecutive orders fail, pause that strategy for 5 minutes.
• Delivery: Priority queue — critical alerts (position liquidations) bypass rate limits.
• Failover: If primary broker is unreachable, route to secondary within 2 seconds.

┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
│ Market   │→→→│ Strategy │→→→│ Risk     │→→→│ Executor │→→→│ Broker   │
│ Data     │   │ Engine   │   │ Gate     │   │          │   │ API      │
└──────────┘   └──────────┘   └──────────┘   └────┬─────┘   └──────────┘
                                                   │
                                            ┌──────▼──────┐
                                            │  Telegram   │
                                            │  Discord    │
                                            │  SMS        │
                                            └─────────────┘

Requirements: Run in 2+ cloud regions, support 5+ LLM providers, auto-scale to 10K concurrent users, cost < $50K/month, PII redaction, audit trail.

Key Decisions:

• Gateway: Stateless agent gateway behind a regional load balancer. Each request is independent.
• Model routing: Priority: DeepSeek (cheap) → Gemini (vision) → Anthropic (complex reasoning). Auto-failover if primary returns errors.
• Cost control: Per-tenant budgets, daily spending limits per model, cache identical prompts.
• PII: Redact at ingress before reaching LLM. Reconstruct in response.
• Compliance: Every interaction logged to immutable audit store with user ID, model, tokens, cost.

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Tenant A   │     │  Tenant B   │     │  Tenant C   │
│  (Trading)  │     │  (Research) │     │  (Support)  │
└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
       │                   │                   │
       ▼                   ▼                   ▼
┌─────────────────────────────────────────────────────┐
│                API Gateway (Kong/Tyk)                │
│  Rate limiting · Auth · Tenant isolation · Audit     │
└────────────────────────┬────────────────────────────┘
                         │
┌────────────────────────▼────────────────────────────┐
│              LLM Router / Load Balancer              │
│  Provider health · Cost routing · Fallback chain     │
├─────────────┬──────────────┬──────────────┬─────────┤
│  DeepSeek   │   Gemini     │  Anthropic   │  Local  │
│  (primary)  │  (vision)    │ (fallback)   │ (cache) │
└─────────────┴──────────────┴──────────────┴─────────┘
                         │
┌────────────────────────▼────────────────────────────┐
│              Observability Stack                     │
│  LangFuse traces · Cost tracking · Alerting          │
└─────────────────────────────────────────────────────┘

Key considerations:

• Provider failover must be sub-second — pre-warm connections to fallback
• Cache layer: identical prompts (with TTL) skip LLM entirely — 60% cost reduction
• Tenant isolation: separate DB schemas, no cross-tenant data leakage
• Rate limiting: per-tenant, per-model, per-minute sliding window
• Audit: every request logged with tenant_id, model, tokens_in, tokens_out, latency, cost

The Hermes AI platform is built on a single-instance agent gateway architecture with multi-platform delivery. Unlike traditional microservice AI platforms, Hermes runs as a monolithic agent with plugin-based extensibility.

Architecture Layers

Single vs Multi-Agent Pattern

Key Orchestration Patterns

• Fan-out: batch delegate_task for parallel research (e.g., check all 4 trading envs)
• Chain: cron job A collects data → cron job B processes it via context_from
• Human-in-the-loop: clarify tool for approval gates on trading actions
• Watchdog: keepalive scripts restart failed services autonomously

Provider Chain

DeepSeek v4 Flash (primary)
  └─▸ Error / Rate limit → Gemini 2.5 Flash (fallback, via OpenRouter)
       └─▸ Vision tasks → Gemini 2.5 Flash (vision, forced)

Cost Optimization

Memory Layers

┌─────────────────────────────────────────────────────────┐
│                    Multi-Environment Pipeline             │
├─────────────────────────────────────────────────────────┤
│  DEV  (testing, 58 strats)                              │
│   ↓  strategy promotion (if PF > 1.0)                   │
│  QA   (validation, 5 strats, budget $4K)                │
│   ↓  cross-environment scoring                          │
│  PROD (staging, 5 strats, budget $4K)                   │
│   ↓  20 consecutive profitable days                     │
│  LIVE (real money, DISABLED)                            │
└─────────────────────────────────────────────────────────┘

Key Design Decisions

• 4 environments — isolation prevents bad strategies from reaching live money
• Identical configs — same budget, same limits, same symbols (so results are comparable)
• Alpaca broker — paper API for testing, live API for real trading. Same code path.
• Bracket orders — every entry has attached stop-loss + take-profit. No naked positions.
• Budget gate — hard cap on total exposure. Prevents runaway trading.

Strategy Discovery Pipeline

1. Data Collection (5-min bars, 20 symbols)
   ↓
2. Feature Engineering (RSI, MACD, VWAP, autocorr, vol ratio, ATR)
   ↓
3. XGBoost Training (predict next-bar direction)
   ↓
4. Rule Extraction (decision tree paths → strategy rules)
   ↓
5. Backtesting (10-30 days, track PnL, win rate, profit factor)
   ↓
6. Ranking (composite score = weighted PnL + WR + PF)
   ↓
7. Selection (top N = max_enabled_strategies, swap if new beats worst)

Strategy Performance Metrics

Risk Gates (in order of enforcement)

1. Budget Gate — hard $4K cap on total deployed capital
2. Daily Loss Limit — halt all trading if P&L hits -$10
3. Position Cap — max 5 concurrent positions
4. Entry Limit — max 4 entries per symbol per day
5. Spread Filter — reject trades where spread > 0.08%
6. Direction Filter — long-only enforced
7. Cooldown — 300s between re-entries on same symbol
8. Profit Lock — lock gains above $30 threshold, floor at $12

Every order uses bracket order pattern:

Entry (limit order)
  ├─ Stop-Loss (1.0% below entry for standard strats)
  └─ Take-Profit (0.5% above entry for standard strats)

If price hits SL → market sell (limit loss)
If price hits TP → limit sell (lock profit)
If neither by EOD → market close (< 15:00 ET cutoff)

Credential Management

Current State

Full System Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                        PUBLIC INTERNET                               │
├─────────────────────────────────────────────────────────────────────┤
│  Tailscale Funnel  │  ttyd :4860  │  Telegram Bot  │  Email/SMTP     │
└──────┬──────────────┴──────┬──────┴───────┬─────────┴────────────────┘
       │                     │              │
┌──────▼─────────────────────▼──────────────▼─────────────────────────┐
│                      REVERSE PROXY (port 80)                        │
│  routes: /dashboard → :9119 · /cloud → :8082 · /api → :8766         │
│          /handbook → HTML · /aidlc-handbook → HTML · /arch-review   │
└──────┬──────────────────────────┬───────────────────────────────────┘
       │                          │
┌──────▼──────────┐    ┌─────────▼───────────────────────────────────┐
│  HERMES GATEWAY  │    │              TRADING BOT FLEET               │
│  :8642           │    │                                               │
│  Telegram ───────┤    │  ┌──────────┐  ┌──────────┐  ┌──────────┐    │
│  Skills/Memory   │    │  │   DEV    │  │   QA     │  │  PROD    │    │
│  MCP Servers     │    │  │ paper    │  │ paper    │  │ paper    │    │
│  Cron Jobs       │    │  │ 58 strats│  │ 5 strats │  │ 5 strats │    │
│  SOUL.md         │    │  │ $4K      │  │ $4K      │  │ $4K      │    │
└─────────────────┘    │  └────┬─────┘  └────┬─────┘  └────┬─────┘    │
                       │       │             │             │          │
                       │  ┌────▼─────────────▼─────────────▼────┐     │
                       │  │         SHARED SERVICES              │     │
                       │  │  ┌──────┐ ┌──────┐ ┌─────────┐      │     │
                       │  │  │Alpaca│ │PG    │ │Grafana  │      │     │
                       │  │  │Paper │ │:32772│ │Cloud+   │      │     │
                       │  │  │API   │ │n8n   │ │Local    │      │     │
                       │  │  └──────┘ │Dify  │ │:8742    │      │     │
                       │  │           │Swamp │ └─────────┘      │     │
                       │  │           │Lang  │                  │     │
                       │  │           │fuse  │                  │     │
                       │  │           └──────┘                  │     │
                       │  └─────────────────────────────────────┘     │
                       └──────────────────────────────────────────────┘