Designing Self-Healing Microservices with Agentic Watchdogs: Runtime eBPF Telemetry, LLM Root-Cause Analysis, and Automated Canary Rollbacks in .NET 10

+---------------------------------------------------------------------------------------------------------+
|                                    THE TRADITIONAL 3:00 AM INCIDENT TIMELINE                            |
+---------------------------------------------------------------------------------------------------------+
|                                                                                                         |
|   03:14:02  [Canary v2.14.2 Deployed: 10% Traffic]                                                      |
|   03:14:38  [Connection Leak Begins Sapping Socket Pool]                                                |
|   03:16:15  [TCP RST Storm Begins in Kernel]                                                            |
|   03:18:00  [Prometheus Static Alert: "p99 Latency > 1000ms" Fires]                                     |
|   03:18:30  [PagerDuty Alert Storm: 17 Microservices Fire Alerts]                                       |
|   03:22:10  [On-Call Engineers Paged, Wake Up, Authenticate VPN] ◄── 8 Min Human Wakeup Delay          |
|   03:26:45  [War Room Assembled; Triaging Cascading Symptoms]   ◄── Alert Fatigue & Symptom Confusion   |
|   03:34:20  [SRE Isolates Root Cause: Tariff Engine Canary]     ◄── 16 Min Distributed Trace Correl.    |
|   03:40:15  [Manual ArgoCD Rollback Executed & Traffic Drained] ◄── Total MTTD + MTTR: 26m 13s          |
|                                                                                                         |
|   Result: £18,400 SLA Breach Penalties, Dropped Ingestion Events, Severely Burned-Out Engineers.        |
+---------------------------------------------------------------------------------------------------------+

+=========================================================================================================+
|                                   THREE PATHOLOGIES OF STATIC INCIDENT TRIAGE                           |
+=========================================================================================================+
|                                                                                                         |
|  1. THE SYMPTOM VS. ROOT-CAUSE BLIND SPOT                                                               |
|  [Ingress Gateway (504 Gateway Timeout)]  ◄── High Latency / Error Symptom                             |
|        │                                                                                                |
|        ▼ (Cascading Backpressure)                                                                       |
|  [Orchestrator Service (Thread Pool Depletion)]                                                         |
|        │                                                                                                |
|        ▼ (Socket Starvation)                                                                            |
|  [Database Service (Healthy Connection Pool)]                                                           |
|        ▲                                                                                                |
|        └── [Canary Service (Leaking 1 Connection / Retry)] ◄── TRUE ROOT CAUSE HIDDEN UNDER SYMPTOMS    |
|                                                                                                         |
|  2. THE DESTRUCTIVE AUTOSCALING FEEDBACK LOOP                                                           |
|  Slow Database Query ──► Request Queuing ──► CPU Spike ──► HPA Scales Pods 3x ──► DB Conns Explode 3x  |
|                                                     ▲                                    │              |
|                                                     └──────── Complete DB Collapse ◄─────┘              |
|                                                                                                         |
|  3. THE 20-MINUTE COGNITIVE BOTTLENECK                                                                  |
|  Human On-Call Engineer: Context Switch ──► Log In ──► Filter Red Herrings ──► Correlate Diff ──► Action |
|  Mean Time To Diagnose (MTTD): 15-25 Minutes while customer revenue evaporates.                         |
+=========================================================================================================+

+======================================================================================================================+
|                                    AUTONOMOUS CLOSED-LOOP SELF-HEALING ARCHITECTURE                                  |
+======================================================================================================================+
|                                                                                                                      |
|   +--------------------------------------------------------------------------------------------------------------+   |
|   | 1. KERNEL TELEMETRY PLANE (eBPF Probes in Linux Kernel 6.8+)                                                 |   |
|   |                                                                                                              |   |
|   |   [kprobe:tcp_v4_connect]    [tracepoint:tcp_receive_reset]    [kprobe:sched_switch (Thread Starvation)]     |   |
|   |              │                                │                                │                             |   |
|   |              ▼                                ▼                                ▼                             |   |
|   |   +──────────────────────────────────────────────────────────────────────────────────────────────────────+   |   |
|   |   |                       High-Performance BPF Ring Buffer (Zero-Allocation Kernel Memory)               |   |   |
|   |   +──────────────────────────────────────────────────────────────────────────────────────────────────────+   |   |
|   +--------------------------------------------------------------------------------------------------------------+   |
|                                                      │ Sub-Millisecond Kernel Event Push (<0.5% CPU Tax)             |
|                                                      ▼                                                               |
|   +--------------------------------------------------------------------------------------------------------------+   |
|   | 2. REAL-TIME TELEMETRY INGESTION BUS (.NET 10 Runtime Engine)                                                |   |
|   |                                                                                                              |   |
|   |   System.Threading.Channels.Channel<EbpfSocketMetric> (Bounded Buffer, WaitToReadAsync)                      |   |
|   |   Zero-Allocation ReadOnlyMemory<byte> Binary Deserializer                                                   |   |
|   |   Statistical Anomaly Detector (Sliding Window Exponential Moving Average + EWMA Latency Spikes)             |   |
|   +--------------------------------------------------------------------------------------------------------------+   |
|                                                      │ Anomaly Envelope Dispatched (AnomalyScore > 0.88)             |
|                                                      ▼                                                               |
|   +--------------------------------------------------------------------------------------------------------------+   |
|   | 3. DELIBERATIVE AGENTIC WATCHDOG (Semantic Kernel + Frontier Reasoning Model)                                |   |
|   |                                                                                                              |   |
|   |   Multi-Modal Incident Context Assembly:                                                                     |   |
|   |   ├── eBPF Kernel Telemetry (TCP RST rates, Socket Queue Depth)                                              |   |
|   |   ├── OpenTelemetry Distributed Trace Spans (Error Roots, p99 Latency Paths)                                 |   |
|   |   ├── ArgoCD / Kubernetes Deployment Manifests (Recent Canary Rollouts, Image Tags)                         |   |
|   |   └── GitHub Commit Diff Metadata (Code changes in suspect microservice)                                    |   |
|   |                                                                                                              |   |
|   |   Deliberative LLM Root-Cause Synthesis:                                                                     |   |
|   |   "Tariff-Engine Canary v2.14.2 is exhausting DB connections on /calculate-quote. Diff shows missing using   |   |
|   |    statement on line 142 of TariffCalculator.cs. Root Cause: Resource Leak. Blast Radius: 10% Canary."       |   |
|   +--------------------------------------------------------------------------------------------------------------+   |
|                                                      │ Structured JSON Remediation Action Plan                       |
|                                                      ▼                                                               |
|   +--------------------------------------------------------------------------------------------------------------+   |
|   | 4. DETERMINISTIC REMEDIATION ACTUATOR (Policy Guarded & Idempotent)                                         |   |
|   |                                                                                                              |   |
|   |   [Safety Guard Verification: Blast Radius <= Max Allowed (15%) & Action In Whitelist]                       |   |
|   |   ├── 1. Dynamic Envoy / Traefik Traffic Shedding (Instant 0% Canary Traffic)                                |   |
|   |   ├── 2. Automated ArgoCD Canary Rollback (Patch Deployment to v2.14.1)                                      |   |
|   |   ├── 3. Distributed Circuit Breaker Trip (Isolate Upstream Retry Cascades)                                  |   |
|   |   └── 4. Post-Rollback Health Verification (Monitor eBPF TCP RST & Latency Normalize)                        |   |
|   +--------------------------------------------------------------------------------------------------------------+   |
|                                                      │ Closed-Loop Verification: Healthy Metrics Confirmed           |
|                                                      ▼                                                               |
|   +--------------------------------------------------------------------------------------------------------------+   |
|   | 5. INCIDENT RESOLUTION AUDIT TRAIL (Slack / MS Teams / PagerDuty Resolved Automatically)                     |   |
|   |    "Incident auto-remediated in 41.2 seconds. Canary v2.14.2 rolled back. Zero customer billing impact."    |   |
|   +--------------------------------------------------------------------------------------------------------------+   |
+======================================================================================================================+

flowchart TD
    subgraph KernelSpace ["Linux Kernel Space (eBPF)"]
        K1["kprobe:tcp_v4_connect"] --> RB["BPF Perf Ring Buffer"]
        K2["tracepoint:tcp_receive_reset"] --> RB
        K3["kprobe:sched_switch"] --> RB
    end

    subgraph Ingestion ["Ingestion Plane (.NET 10)"]
        RB -->|"RingBufferPollAsync()"| CH["Channel<EbpfSocketMetric>"]
        CH --> AD["Sliding Window Anomaly Detector"]
    end

    subgraph Deliberation ["Deliberative Agentic Brain"]
        AD -->|"Anomaly Detected (p > 0.90)"| AG["Agentic Watchdog (Semantic Kernel)"]
        OTEL["OpenTelemetry Distributed Traces"] --> AG
        GIT["Git Commit & Deployment Diffs"] --> AG
        K8S["Kubernetes Pod & Service State"] --> AG
        AG -->|"Structured Reasoning"| DIAG["Root Cause Diagnosis & Remediation Plan"]
    end

    subgraph Execution ["Remediation Plane"]
        DIAG --> SG{"Safety Guard Check<br/>Blast Radius &le; 15%?"}
        SG -->|Approved| ACT["Deterministic Actuator"]
        SG -->|Violated| ESC["Escalate to Human SRE"]
        ACT --> TS["Shed Canary Traffic (Envoy API)"]
        ACT --> RBK["Rollback Deployment (ArgoCD API)"]
        ACT --> CB["Trip Circuit Breakers"]
    end

    subgraph Verification ["Closed-Loop Verification"]
        RBK --> MON["Verify eBPF Socket Health"]
        MON -->|Normalized| AUD["Publish Incident Post-Mortem"]
    end

    classDef kernel fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc;
    classDef ingest fill:#0f172a,stroke:#34d399,stroke-width:2px,color:#f8fafc;
    classDef agent fill:#1e1b4b,stroke:#a855f7,stroke-width:2px,color:#f8fafc;
    classDef execute fill:#450a0a,stroke:#f87171,stroke-width:2px,color:#f8fafc;
    classDef verify fill:#064e3b,stroke:#10b981,stroke-width:2px,color:#f8fafc;

    class K1,K2,K3,RB kernel;
    class CH,AD ingest;
    class AG,OTEL,GIT,K8S,DIAG agent;
    class SG,ACT,TS,RBK,CB,ESC execute;
    class MON,AUD verify;

namespace Centralogic.SelfHealing.Core.Contracts;

using System;
using System.Collections.Generic;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;

/// <summary>
/// Telemetry metric captured directly from the Linux kernel eBPF ring buffer.
/// </summary>
public readonly record struct EbpfSocketMetric(
    ulong TimestampNanoseconds,
    uint SourceIp,
    uint DestinationIp,
    ushort SourcePort,
    ushort DestinationPort,
    uint ProcessId,
    byte Protocol,
    byte EventType, // 1 = SYN_SENT, 2 = ESTABLISHED, 3 = TCP_RST, 4 = RETRANSMIT
    uint LatencyMicroseconds,
    string PodName,
    string ServiceNamespace
);

/// <summary>
/// Structured incident context dispatched to the Agentic Watchdog.
/// </summary>
public sealed record AnomalyIncidentEnvelope
{
    public required Guid IncidentId { get; init; } = Guid.NewGuid();
    public required DateTimeOffset DetectedAt { get; init; } = DateTimeOffset.UtcNow;
    public required string PrimaryAffectedService { get; init; }
    public required double AnomalyScore { get; init; }
    public required IReadOnlyList<EbpfSocketMetric> CorrelatedSocketMetrics { get; init; }
    public required IReadOnlyDictionary<string, string> ActiveDeployments { get; init; }
    public required double P99LatencyDeltaPercent { get; init; }
}

/// <summary>
/// Deterministic root-cause diagnosis emitted by the LLM reasoning core.
/// </summary>
public sealed record DiagnosticAssessment
{
    [JsonPropertyName("rootCauseCategory")]
    public required string RootCauseCategory { get; init; } // e.g., "CONNECTION_LEAK", "DOWNSTREAM_TIMEOUT", "THREADPOOL_STARVATION"

    [JsonPropertyName("suspectService")]
    public required string SuspectService { get; init; }

    [JsonPropertyName("suspectCommitHash")]
    public string? SuspectCommitHash { get; init; }

    [JsonPropertyName("confidenceScore")]
    public required double ConfidenceScore { get; init; }

    [JsonPropertyName("causalExplanation")]
    public required string CausalExplanation { get; init; }

    [JsonPropertyName("recommendedRemediation")]
    public required RemediationActionType RecommendedRemediation { get; init; }

    [JsonPropertyName("targetDeploymentTag")]
    public string? TargetDeploymentTag { get; init; }

    [JsonPropertyName("trafficShedPercentage")]
    public int TrafficShedPercentage { get; init; }
}

public enum RemediationActionType
{
    None = 0,
    RollbackCanaryDeployment = 1,
    ShedCanaryTraffic = 2,
    TripCircuitBreaker = 3,
    RestartFailingPods = 4,
    EscalateToHumanOnCall = 5
}

/// <summary>
/// High-level interface contract governing the autonomous watchdog lifecycle.
/// </summary>
public interface IAgenticWatchdog
{
    ValueTask IngestKernelTelemetryAsync(EbpfSocketMetric metric, CancellationToken cancellationToken = default);
    Task<DiagnosticAssessment> DiagnoseIncidentAsync(AnomalyIncidentEnvelope incident, CancellationToken cancellationToken = default);
    Task<bool> ExecuteRemediationAsync(DiagnosticAssessment diagnosis, CancellationToken cancellationToken = default);
}

namespace Centralogic.SelfHealing.Telemetry;

using System;
using System.Buffers;
using System.Buffers.Binary;
using System.IO.MemoryMappedFiles;
using System.Runtime.CompilerServices;
using System.Threading;
using System.Threading.Channels;
using System.Threading.Tasks;
using Centralogic.SelfHealing.Core.Contracts;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

/// <summary>
/// Reads binary eBPF ring-buffer frames with zero memory allocations,
/// pushing structured records into high-performance .NET 10 channels.
/// </summary>
public sealed class EbpfTelemetryCollector : BackgroundService
{
    private readonly Channel<EbpfSocketMetric> _telemetryChannel;
    private readonly IAgenticWatchdog _watchdog;
    private readonly ILogger<EbpfTelemetryCollector> _logger;
    private const int RingBufferEventSizeBytes = 32;

    public EbpfTelemetryCollector(
        IAgenticWatchdog watchdog,
        ILogger<EbpfTelemetryCollector> logger)
    {
        _watchdog = watchdog;
        _logger = logger;

        // Bounded channel with DropOldest backpressure policy to protect watchdog memory
        _telemetryChannel = Channel.CreateBounded<EbpfSocketMetric>(new BoundedChannelOptions(50_000)
        {
            FullMode = BoundedChannelFullMode.DropOldest,
            SingleWriter = true,
            SingleReader = true
        });
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Initializing eBPF Zero-Allocation Ring Buffer Reader...");

        // Launch reader and ingestion workers concurrently
        var readerTask = Task.Run(() => PollKernelRingBufferAsync(_telemetryChannel.Writer, stoppingToken), stoppingToken);
        var consumerTask = Task.Run(() => ConsumeTelemetryPipelineAsync(_telemetryChannel.Reader, stoppingToken), stoppingToken);

        await Task.WhenAll(readerTask, consumerTask).ConfigureAwait(false);
    }

    private async Task PollKernelRingBufferAsync(ChannelWriter<EbpfSocketMetric> writer, CancellationToken cancellationToken)
    {
        // Simulated access to /sys/kernel/debug/tracing or custom BPF ringbuf memory map
        byte[] rentBuffer = ArrayPool<byte>.Shared.Rent(4096);
        try
        {
            while (!cancellationToken.IsCancellationRequested)
            {
                // In production, this binds directly to bpf_ring_buffer__poll via P/Invoke
                int bytesRead = ReadSimulatedKernelFrame(rentBuffer.AsSpan(0, RingBufferEventSizeBytes));
                if (bytesRead >= RingBufferEventSizeBytes)
                {
                    ReadOnlySpan<byte> span = rentBuffer.AsSpan(0, RingBufferEventSizeBytes);
                    
                    // Zero-allocation binary decoding using BinaryPrimitives
                    ulong timestamp = BinaryPrimitives.ReadUInt64LittleEndian(span[0..8]);
                    uint srcIp = BinaryPrimitives.ReadUInt32LittleEndian(span[8..12]);
                    uint dstIp = BinaryPrimitives.ReadUInt32LittleEndian(span[12..16]);
                    ushort srcPort = BinaryPrimitives.ReadUInt16LittleEndian(span[16..18]);
                    ushort dstPort = BinaryPrimitives.ReadUInt16LittleEndian(span[18..20]);
                    uint pid = BinaryPrimitives.ReadUInt32LittleEndian(span[20..24]);
                    byte proto = span[24];
                    byte eventType = span[25];
                    uint latency = BinaryPrimitives.ReadUInt32LittleEndian(span[26..30]);

                    var metric = new EbpfSocketMetric(
                        TimestampNanoseconds: timestamp,
                        SourceIp: srcIp,
                        DestinationIp: dstIp,
                        SourcePort: srcPort,
                        DestinationPort: dstPort,
                        ProcessId: pid,
                        Protocol: proto,
                        EventType: eventType,
                        LatencyMicroseconds: latency,
                        PodName: "tariff-engine-canary-7df4f68b",
                        ServiceNamespace: "billing"
                    );

                    while (!writer.TryWrite(metric))
                    {
                        await writer.WaitToWriteAsync(cancellationToken).ConfigureAwait(false);
                    }
                }

                // Micro-yield to prevent thread starvation on core
                await Task.Yield();
            }
        }
        finally
        {
            ArrayPool<byte>.Shared.Return(rentBuffer);
            writer.Complete();
        }
    }

    private async Task ConsumeTelemetryPipelineAsync(ChannelReader<EbpfSocketMetric> reader, CancellationToken cancellationToken)
    {
        while (await reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false))
        {
            while (reader.TryRead(out var metric))
            {
                await _watchdog.IngestKernelTelemetryAsync(metric, cancellationToken).ConfigureAwait(false);
            }
        }
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    private static int ReadSimulatedKernelFrame(Span<byte> destination)
    {
        // Mock kernel frame: TCP_RST event (type = 3) on PostgreSQL port 5432
        BinaryPrimitives.WriteUInt64LittleEndian(destination[0..8], (ulong)DateTime.UtcNow.Ticks);
        BinaryPrimitives.WriteUInt32LittleEndian(destination[8..12], 0xC0A80001); // 192.168.0.1
        BinaryPrimitives.WriteUInt32LittleEndian(destination[12..16], 0xC0A80002); // 192.168.0.2
        BinaryPrimitives.WriteUInt16LittleEndian(destination[16..18], 48922);
        BinaryPrimitives.WriteUInt16LittleEndian(destination[18..20], 5432); // Postgres
        BinaryPrimitives.WriteUInt32LittleEndian(destination[20..24], 4812);
        destination[24] = 6; // IPPROTO_TCP
        destination[25] = 3; // TCP_RST event
        BinaryPrimitives.WriteUInt32LittleEndian(destination[26..30], 85000); // 85ms latency
        return RingBufferEventSizeBytes;
    }
}

namespace Centralogic.SelfHealing.Agent;

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Centralogic.SelfHealing.Core.Contracts;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

public sealed class AgenticWatchdogEngine : IAgenticWatchdog
{
    private readonly Kernel _kernel;
    private readonly IChatCompletionService _chatService;
    private readonly IRemediationActuator _actuator;
    private readonly ILogger<AgenticWatchdogEngine> _logger;
    private readonly List<EbpfSocketMetric> _recentAnomalies = new(1000);
    private readonly object _lock = new();

    public AgenticWatchdogEngine(
        Kernel kernel,
        IRemediationActuator actuator,
        ILogger<AgenticWatchdogEngine> logger)
    {
        _kernel = kernel;
        _chatService = kernel.GetRequiredService<IChatCompletionService>();
        _actuator = actuator;
        _logger = logger;
    }

    public ValueTask IngestKernelTelemetryAsync(EbpfSocketMetric metric, CancellationToken cancellationToken = default)
    {
        // Check for TCP_RST or latency spikes > 50,000 microseconds
        if (metric.EventType == 3 || metric.LatencyMicroseconds > 50_000)
        {
            lock (_lock)
            {
                _recentAnomalies.Add(metric);
                if (_recentAnomalies.Count >= 25)
                {
                    // Trigger evaluation asynchronously
                    var anomalyBatch = _recentAnomalies.ToArray();
                    _recentAnomalies.Clear();

                    _ = Task.Run(async () =>
                    {
                        var envelope = new AnomalyIncidentEnvelope
                        {
                            PrimaryAffectedService = "tariff-engine",
                            AnomalyScore = 0.94,
                            CorrelatedSocketMetrics = anomalyBatch,
                            ActiveDeployments = new Dictionary<string, string>
                            {
                                ["stable"] = "v2.14.1",
                                ["canary"] = "v2.14.2"
                            },
                            P99LatencyDeltaPercent = 380.0
                        };

                        var diagnosis = await DiagnoseIncidentAsync(envelope, CancellationToken.None).ConfigureAwait(false);
                        await ExecuteRemediationAsync(diagnosis, CancellationToken.None).ConfigureAwait(false);
                    }, CancellationToken.None);
                }
            }
        }
        return ValueTask.CompletedTask;
    }

    public async Task<DiagnosticAssessment> DiagnoseIncidentAsync(AnomalyIncidentEnvelope incident, CancellationToken cancellationToken = default)
    {
        _logger.LogWarning("Watchdog deliberating on Incident {IncidentId} across service {Service}...", 
            incident.IncidentId, incident.PrimaryAffectedService);

        // Fetch simulated git commit diff for canary version
        string gitDiff = """
            diff --git a/src/Billing/TariffCalculator.cs b/src/Billing/TariffCalculator.cs
            index a4f91b..b8921c 100644
            --- a/src/Billing/TariffCalculator.cs
            +++ b/src/Billing/TariffCalculator.cs
            @@ -139,4 +139,6 @@ public async Task<TariffQuote> CalculateAsync(Guid customerId)
            -    await using var conn = await _dataSource.OpenConnectionAsync();
            +    var conn = _dataSource.OpenConnection(); // BUG: Removed await using and async pooling
                 return await conn.QuerySingleAsync<TariffQuote>(sql, new { customerId });
            """;

        var chatHistory = new ChatHistory();
        chatHistory.AddSystemMessage("""
            You are Centralogic's Autonomous SRE Watchdog. Analyze the provided Linux eBPF kernel socket metrics,
            distributed traces, active deployments, and git diff. Formulate a precise root cause diagnosis and 
            select the safest deterministic remediation action.
            
            Strictly output your response as valid JSON conforming to the following structure:
            {
              "rootCauseCategory": "CONNECTION_LEAK",
              "suspectService": "tariff-engine",
              "suspectCommitHash": "a4f91b",
              "confidenceScore": 0.98,
              "causalExplanation": "Detailed technical analysis...",
              "recommendedRemediation": 1,
              "targetDeploymentTag": "v2.14.1",
              "trafficShedPercentage": 100
            }
            Remediation enum: 0=None, 1=RollbackCanaryDeployment, 2=ShedCanaryTraffic, 3=TripCircuitBreaker, 4=RestartFailingPods, 5=EscalateToHumanOnCall.
            """);

        chatHistory.AddUserMessage($"""
            INCIDENT TELEMETRY:
            - Service: {incident.PrimaryAffectedService}
            - Detected At: {incident.DetectedAt:O}
            - p99 Latency Surge: +{incident.P99LatencyDeltaPercent}%
            - eBPF TCP RST Events Captured: {incident.CorrelatedSocketMetrics.Count}
            - Active Deployments: Stable={incident.ActiveDeployments["stable"]}, Canary={incident.ActiveDeployments["canary"]}
            - Suspect Pod: tariff-engine-canary-7df4f68b
            
            GIT DIFF OF CANARY v2.14.2:
            {gitDiff}
            """);

        var response = await _chatService.GetChatMessageContentAsync(chatHistory, cancellationToken: cancellationToken).ConfigureAwait(false);
        string rawJson = response.Content?.Trim() ?? "{}";

        // Clean any markdown code blocks
        if (rawJson.StartsWith("

### D. The Deterministic Remediation Actuator

To guarantee stability, the `RemediationActuator` enforces safety boundaries: verifying that the target is indeed a canary deployment, asserting that blast radius is strictly bounded ($\le 15\%$), and coordinating zero-downtime traffic draining and canary rollback.

---

## 4. Enterprise Performance & Benchmark Matrix

To quantify the operational impact of replacing manual on-call triage with an autonomous agentic watchdog backed by eBPF telemetry, we measured 120 synthetic and production incident simulations across our enterprise microservice cluster over a 90-day benchmark evaluation.