SRS: UC_9.9 — Flow 31: Aggregate Exhaustion Monitor (Sev-1)
| Field | Value |
|---|---|
| BA in Charge | linh.nguyen34 |
| Date Created | 2026-09-03 |
| Version | v1.3 |
| Document References | Zapier Integration V7.pdf (§Flow 31 lines 2471–2479) · RFQ_ Stack Trading Prop Tech V7.pdf (§Calculate Volume Pacing lines 1460–1464; §Merchant_Routing_Config lines 847–855) |
UC Index
| UC_ID | Use Case Name | Business Description |
|---|---|---|
| UC_9.9 | Aggregate Exhaustion Monitor (Sev-1) | Daily 11:00 PM CST: Zapier Flow 31 calls GET /calculate-volume-pacing. If projected end-of-month volume across all active MIDs exceeds total aggregate capacity, fires SEV-1 Slack alert to Operations. |
<a id="UC_9.9"></a>
UC_9.9 — Flow 31: Aggregate Exhaustion Monitor (Sev-1)
1. Overview
| Field | Content |
|---|---|
| ID | UC_9.9 |
| Use Case | Aggregate Exhaustion Monitor (Sev-1) |
| Description | Macro-level predictive defense system. Every day at 11:00 PM CST, Zapier Flow 31 calls GET /calculate-volume-pacing to compute the projected end-of-month volume across all active MIDs. If the projection exceeds total aggregate capacity, a SEV-1 CRITICAL alert is sent to the Operations Slack channel. Complements UC_9.8 (Flow 30) which monitors individual MIDs — this flow monitors the system-wide aggregate trend to catch volume spikes before month-end exhaustion occurs. |
| Zapier Flow | Flow 31 (Aggregate Exhaustion Monitor) |
| Zapier Table | Merchant_Routing_Config (PostgreSQL — queried by Middleware, not directly by Zapier) |
| 3rd Party | Slack (SEV-1 alert delivery) |
2. Trigger
Type: Scheduled — Zapier Cron Job.
Schedule: Daily at 11:00 PM CST.
(Source: Zapier Integration V7.pdf §Flow 31 line 2473)
3. Basic Flow
Step 1 — Scheduled Trigger (Zapier)
Zapier's built-in scheduler fires at 11:00 PM CST daily and calls GET /calculate-volume-pacing on Middleware.
(Source: Zapier Integration V7.pdf §Flow 31 lines 2473–2474)
Step 2 — Pacing Calculation (GET /calculate-volume-pacing)
Middleware executes the following algorithm against Merchant_Routing_Config:
2a. Aggregate totals (active MIDs only):
SELECT
SUM(current_mtd_volume) AS total_mtd_volume,
SUM(monthly_limit) AS total_aggregate_capacity
FROM merchant_routing_config
WHERE is_active = TRUE;2b. Compute Average Daily Volume (ADV):
N = current day of month (e.g. 15)
ADV = total_mtd_volume / N2c. Project End-of-Month Volume:
T = total days in current calendar month (e.g. 30 for September)
Projected_EOM_Volume = ADV × T2d. Exhaustion check:
IF Projected_EOM_Volume >= total_aggregate_capacity:
projected_exhaustion = true
ELSE:
projected_exhaustion = false(Source: RFQ_ Stack Trading Prop Tech V7.pdf §Calculate Volume Pacing lines 1460–1464; Zapier Integration V7.pdf §Flow 31 lines 2474–2475)
Step 3 — Zapier Filter
Zapier evaluates the response. Only proceeds if projected_exhaustion = true. If false, the flow ends silently — no Slack message sent.
(Source: Zapier Integration V7.pdf §Flow 31 line 2475)
Step 4 — SEV-1 Slack Alert
If projected_exhaustion = true, send a SEV-1 CRITICAL alert to #ops-treasury (or configured Operations channel).
Slack message content:
🚨 [SEV-1 CRITICAL] AGGREGATE VOLUME PACING EXHAUSTION DETECTED
Current Day: [current_day_of_month] / [total_days_in_month]
Total MTD Volume: $[total_mtd_volume]
Average Daily Volume: $[average_daily_volume]/day
Projected EOM Volume: $[projected_eom_volume]
Total Aggregate Capacity: $[total_aggregate_capacity]
CRITICAL: Aggregate pacing indicates all merchant accounts will exhaust
capacity before month end. Immediate intervention required.(Source: Zapier Integration V7.pdf §Flow 31 lines 2476–2479)
4. API Endpoint
GET /calculate-volume-pacing
| Field | Value |
|---|---|
| Method | GET |
| Auth | Zapier Webhook Signature / HMAC (internal — not public) |
| Caller | Zapier Flow 31 (scheduled daily) |
| Input | None |
Response payload (normal path):
{
"calculation_date": "2026-09-03",
"current_day_of_month": 3,
"total_days_in_month": 30,
"total_mtd_volume": 45000.00,
"total_aggregate_capacity": 450000.00,
"average_daily_volume": 15000.00,
"projected_eom_volume": 450000.00,
"projected_exhaustion": true,
"active_mids_count": 4
}Response payload (Edge Case 2 — no active MIDs):
{
"projected_exhaustion": false,
"total_aggregate_capacity": 0,
"active_mids_count": 0,
"status_code": "NO_ACTIVE_MIDS_WARNING",
"message": "All payment MIDs are currently inactive. Ops Config Warning dispatched."
}| Field | Type | Description |
|---|---|---|
calculation_date | DATE | Date the calculation was run |
current_day_of_month | INT | N — denominator for ADV calculation |
total_days_in_month | INT | T — multiplier for EOM projection |
total_mtd_volume | DECIMAL | Sum of current_mtd_volume for all is_active = TRUE MIDs |
total_aggregate_capacity | DECIMAL | Sum of monthly_limit for all is_active = TRUE MIDs |
average_daily_volume | DECIMAL | total_mtd_volume / current_day_of_month |
projected_eom_volume | DECIMAL | average_daily_volume × total_days_in_month |
projected_exhaustion | BOOLEAN | true if projected_eom_volume >= total_aggregate_capacity; always false when guard clause fires |
active_mids_count | INT | Count of is_active = TRUE MIDs at calculation time; 0 triggers guard clause |
status_code | STRING | Present only on guard clause path: "NO_ACTIVE_MIDS_WARNING" |
message | STRING | Human-readable explanation, present only on guard clause path |
(Source: RFQ_ Stack Trading Prop Tech V7.pdf §Calculate Volume Pacing lines 1460–1464)
5. Safety Gates & Edge Cases
Edge Case 1 — Early-month noise suppression (3-day calendar grace period)
On the 1st of each month, current_mtd_volume values have just been reset to 0.00. ADV will be near-zero and Projected_EOM_Volume will be minimal — the math naturally suppresses false positives. However, the first few days of a month can produce misleading projections due to low-data ADV denominators.
Confirmed behavior (3-day calendar grace period — Option B):
For the first 3 calendar days of each month (days 1, 2, 3), Middleware MUST bypass the Flow 31 pacing calculations entirely and return projected_exhaustion = false unconditionally. On day 4 onward, standard pacing math resumes.
IF current_day_of_month <= 3:
→ return { projected_exhaustion: false } // Grace period — skip pacing math
ELSE:
→ Execute standard ADV / Projected_EOM_Volume calculationRationale: A time-based gate is more predictable than a volume floor. It removes ambiguity about whether a low-volume day is genuine suppression or a configuration problem, and allows day 4 to be the clean start of reliable trending data.
(Source: client confirmed 2026-09-18)
Edge Case 2 — All MIDs inactive (total_aggregate_capacity = 0)
If no MIDs have is_active = TRUE, total_aggregate_capacity = 0. Without a guard clause, two failure modes occur:
- Division-by-zero / NaN / Infinity when computing the volume ratio.
- False-positive SEV-1:
Projected_EOM_Volume (≥ 0) >= 0is alwaystrue→ fires a spurious CRITICAL alert. This misrepresents the incident as a volume exhaustion when the actual cause is a configuration/gateway-disable issue.
Confirmed behavior (Guard Clause — server-side):
Step 1: Query active MIDs
activeMIDs = SELECT * FROM Merchant_Routing_Config WHERE is_active = TRUE
Step 2: Compute total_aggregate_capacity
total_aggregate_capacity = SUM(activeMIDs.monthly_limit)
GUARD CLAUSE:
IF activeMIDs.length = 0 OR total_aggregate_capacity = 0:
→ log WARN: "All payment MIDs inactive — suppressing SEV-1 exhaustion alert"
→ return { projected_exhaustion: false, status_code: "NO_ACTIVE_MIDS_WARNING",
active_mids_count: 0, total_aggregate_capacity: 0 }
→ Middleware MUST additionally dispatch a separate Ops Config Warning
to **`#ops-treasury`** (distinct from SEV-1, NOT routed to Freshdesk):
"🚨 CRITICAL CONFIGURATION ERROR: All MIDs are currently marked inactive."
Step 3 onwards: Standard pacing math — only executes when active_mids_count > 0Alert scenario matrix:
| Scenario | Active MIDs | total_aggregate_capacity | projected_exhaustion | System action |
|---|---|---|---|---|
| Normal | ≥ 1 | > 0 | false | No alert |
| Volume pacing high | ≥ 1 | > 0 | true | SEV-1 to #ops-treasury |
| Edge Case 2 | 0 | 0 | false | Suppress SEV-1. Log WARNING. Dispatch Ops Config Warning to #ops-treasury (not SEV-1, not Freshdesk). |
(Source: client confirmed 2026-09-17)
Edge Case 3 — Endpoint unavailable
If GET /calculate-volume-pacing returns non-200 or times out, Flow 31 executes a two-phase exception handler:
Phase 1 — Automatic Retry (3-strike backoff): Zapier retries automatically 3 times, 5-minute intervals between attempts (total window: 15 minutes). Self-recovers on transient network errors or Middleware restarts.
Phase 2 — SEV-2 Slack Alert (after 3 failed retries):
If all 3 retries fail, Zapier triggers a SEV-2 WARNING to #ops-treasury and #dev_ops.
SEV-2 message content:
🟡 [SEV-2 WARNING] Volume Pacing Monitoring Endpoint Unavailable
• Event: Zapier Flow 31 failed to fetch MID volume pacing metrics.
• Target Endpoint: GET /calculate-volume-pacing
• Status: Failed after 3 retries (Non-200 / Timeout).
• Impact: Daily aggregate merchant capacity check was SKIPPED tonight.
• Action Required: @dev_ops please inspect Middleware Hub logs in AWS CloudWatch.Flow 31 error branching:
Zapier Flow 31: GET /calculate-volume-pacing
│
├─ HTTP 200 OK
│ ├─ projected_exhaustion = true → SEV-1 Slack Alert (#ops-treasury)
│ └─ projected_exhaustion = false → Pass (no alert)
│
└─ Non-200 / Timeout
└─ Auto-retry 3×, 5 min intervals
└─ Persistent fail (all 3 retries exhausted)
└─ SEV-2 Slack Alert (#ops-treasury + #dev_ops)(Source: client confirmed 2026-09-17)
6. Business Rules
| Rule ID | Rule | Source |
|---|---|---|
| BR_9.9.1 | Flow 31 firesdaily at 11:00 PM CST via Zapier scheduled trigger. It is NOT triggered by payment events and NOT a Middleware cron. | Zapier Integration V7.pdf §Flow 31 line 2473 |
| BR_9.9.2 | ADV denominator is current day of month at time of calculation. For the first 3 calendar days of each month (current_day_of_month <= 3), Middleware MUST bypass pacing calculations entirely and return projected_exhaustion = false unconditionally — 3-day calendar grace period prevents false-positive SEV-1 alerts from noise-level ADV denominators at the start of a new cycle. Standard pacing math resumes from day 4 onward. | RFQ_ Stack Trading Prop Tech V7.pdf §Calculate Volume Pacing lines 1460–1464; client confirmed 2026-09-18 |
| BR_9.9.3 | Only MIDs with is_active = TRUE are included in both total_mtd_volume and total_aggregate_capacity aggregations. | RFQ_ Stack Trading Prop Tech V7.pdf §Merchant_Routing_Config lines 847–855 |
| BR_9.9.4 | Zapier MUST suppress Slack output when projected_exhaustion = false. The SEV-1 alert fires only when projected_exhaustion = true. | Zapier Integration V7.pdf §Flow 31 line 2475 |
| BR_9.9.5 | The SEV-1 alert is informational only. It does NOT automatically disable MIDs, re-route traffic, or modify Merchant_Routing_Config. Remediation is manual by Ops. | Zapier Integration V7.pdf §Flow 31 lines 2476–2479 |
| BR_9.9.6 | Production Zapier Flow 31 MUST call the Production Middleware endpoint only. Cross-environment calls (prod → staging DB) are prohibited. | Client spec 2026-09-03 |
| BR_9.9.7 | Monthly reset (shared with UC_9.8 / BR_9.8.4):current_mtd_volume reset to 0.00 for all MIDs at 00:00 UTC on the 1st of each month. This resets the ADV baseline for Flow 31's next calculation cycle. | RFQ_ Stack Trading Prop Tech V7.pdf §Merchant_Routing_Config line 222 |
| BR_9.9.8 | Guard Clause — Zero Aggregate Capacity: If active_mids_count = 0 OR total_aggregate_capacity = 0, GET /calculate-volume-pacing MUST return projected_exhaustion = false immediately without executing pacing math. Prevents (a) division-by-zero errors and (b) false-positive SEV-1 alerts caused by the always-true comparison Projected_EOM_Volume >= 0. Middleware MUST log a WARNING and dispatch a separate Ops Config Warning to #ops-treasury Slack (NOT SEV-1, NOT Freshdesk). Alert content: "🚨 CRITICAL CONFIGURATION ERROR: All MIDs are currently marked inactive." Response includes status_code: "NO_ACTIVE_MIDS_WARNING". The root cause (all MIDs inactive) is a configuration issue, not a volume exhaustion event — the alert type MUST differ accordingly. | Client confirmed 2026-09-17; channel confirmed 2026-09-18 |
| BR_9.9.9 | Flow 31 Endpoint Failure — Retry + SEV-2: If GET /calculate-volume-pacing returns non-200 or times out, Zapier MUST retry 3 times at 5-minute intervals (Phase 1). If all 3 retries fail, Zapier triggers a SEV-2 WARNING to #ops-treasury and #dev_ops (Phase 2). SEV-2 message must include: endpoint name, failure status, impact statement (daily check skipped), and @dev_ops action request with AWS CloudWatch reference. | Client confirmed 2026-09-17 |
7. Relationship to UC_9.8 (Flow 30)
| Dimension | UC_9.8 — Flow 30 | UC_9.9 — Flow 31 |
|---|---|---|
| Scope | Individual MID | All active MIDs (aggregate) |
| Trigger | Real-time, per payment | Scheduled, daily 11:00 PM CST |
| Threshold | 80% of single MID's monthly_limit | 100% of total aggregate capacity (projected EOM) |
| Alert type | Warning | SEV-1 CRITICAL |
| Purpose | Catch individual MID running hot | Catch system-wide volume spike before month-end |
| Remediation | Contact processor for single MID limit increase | Emergency: contact all processors + onboard reserve MIDs + adjust round-robin weights |
8. Acceptance Criteria
| ID | Scenario | Expected Result |
|---|---|---|
| AC-9.9-01 | Day 15 of a 30-day month;total_mtd_volume = 300,000; total_aggregate_capacity = 400,000 → ADV = 20,000; projected EOM = 600,000 | projected_exhaustion = true; SEV-1 Slack alert sent to #ops-treasury |
| AC-9.9-02 | Day 15 of a 30-day month;total_mtd_volume = 150,000; total_aggregate_capacity = 400,000 → ADV = 10,000; projected EOM = 300,000 | projected_exhaustion = false; no Slack alert sent |
| AC-9.9-03 | Day 1 of month; total_mtd_volume = 1,000 (just reset); current_day_of_month = 1 | projected_exhaustion = false (within 3-day grace period — pacing math bypassed); no Slack alert |
| AC-9.9-04 | total_aggregate_capacity = 0 (all MIDs inactive — active_mids_count = 0) | Guard clause fires: projected_exhaustion = false; status_code = "NO_ACTIVE_MIDS_WARNING"; WARNING logged; Ops Config Warning dispatched (not SEV-1); no 500 error; pacing math NOT executed |
| AC-9.9-05 | projected_exhaustion = true | Slack message contains all 7 fields:current_day_of_month, total_days_in_month, total_mtd_volume, average_daily_volume, projected_eom_volume, total_aggregate_capacity and SEV-1 label |
| AC-9.9-06 | Cron fires at 11:00 PM CST on a given day | Flow 31 executes; Production endpoint called (not staging) |
| AC-9.9-07 | Day 20;projected_exhaustion = false | No Slack message; Zapier filter halts flow silently |
9. Open Questions
| ID | Priority | Question | Assumption |
|---|---|---|---|
Resolved 2026-09-18: Option B confirmed — 3-day calendar grace period. current_day_of_month <= 3 bypasses pacing math entirely. No volume floor. BR_9.9.2 + Edge Case 1 + AC-9.9-03 updated. | |||
total_aggregate_capacity = 0 (all MIDs inactive)? | Resolved 2026-09-17: Guard clause — return projected_exhaustion = false + status_code: "NO_ACTIVE_MIDS_WARNING" + log WARN + dispatch Ops Config Warning to #ops-treasury. Pacing math suppressed. BR_9.9.8 added. | ||
GET /calculate-volume-pacing returns non-200 or times out? | Resolved 2026-09-17: 3-retry backoff (5-min intervals) → SEV-2 to #ops-treasury + #dev_ops. BR_9.9.9 added. Edge Case 3 documented. |
10. Changelog
| Date | Version | Summary | Before | After | Source |
|---|---|---|---|---|---|
| 2026-09-18 | v1.2 → v1.3 | Edge Case 1 rewritten (3-day grace period replaces $5,000 floor); Edge Case 2 Ops Config Warning channel pinned to #ops-treasury; BR_9.9.2 updated; BR_9.9.8 updated with channel + alert content; AC-9.9-03 updated; Q1 + Q3 closed in Open Questions; version bump | Edge Case 1: $5,000 volume floor, [OPEN — Q1]. BR_9.9.2: volume floor logic. AC-9.9-03: "below $5,000 floor". Edge Case 2 channel: "TBD by Ops team". Q1 + Q3: open. | Edge Case 1: 3-day calendar grace period — current_day_of_month <= 3 bypasses pacing math entirely. BR_9.9.2: time-based gate. AC-9.9-03: grace period logic. Edge Case 2 channel: #ops-treasury (NOT Freshdesk). Q1 + Q3: resolved. | Client confirmed 2026-09-18 |
| 2026-09-17 | v1.0 → v1.1 | Edge Case 2 + BR_9.9.8 + AC-9.9-04 expanded + response payload + Q2 closed | Edge Case 2: [OPEN — Q2] — guard behavior unspecified. AC-9.9-04: minimal stub. Response payload: 8 fields. | Edge Case 2: full guard clause spec — active_mids_count = 0 check before pacing math, NO_ACTIVE_MIDS_WARNING status code, Ops Config Warning (not SEV-1), pseudocode + alert matrix added. BR_9.9.8 added. AC-9.9-04: expanded with guard clause detail. Response payload: 11 fields (active_mids_count, status_code, message added). Q2 closed. | Client confirmed 2026-09-17 |
| 2026-09-03 | v1.0 | Initial document creation | — | — | Zapier Integration V7.pdf §Flow 31; RFQ_ Stack Trading Prop Tech V7.pdf §Calculate Volume Pacing |