[{"data":1,"prerenderedAt":2595},["ShallowReactive",2],{"post-\u002Fblog\u002Fwhy-i-built-cloudhttp":3},{"id":4,"title":5,"body":6,"book":2584,"date":2585,"description":2586,"extension":2587,"meta":2588,"navigation":586,"path":2589,"seo":2590,"stem":2591,"tags":2592,"__hash__":2594},"blog\u002Fblog\u002Fwhy-i-built-cloudhttp.md","Spreading HttpClient connections across Kubernetes pods",{"type":7,"value":8,"toc":2565},"minimark",[9,14,24,27,39,45,52,68,72,75,155,162,165,168,182,185,189,196,409,429,626,636,647,714,718,723,734,803,807,814,919,922,926,933,1019,1022,1033,1036,1040,1043,1046,1049,1053,1062,1116,1119,1273,1280,1325,1342,1355,1359,1370,1377,1630,1637,1660,1663,1677,1767,1777,1781,1787,1801,1821,1828,2041,2044,2048,2051,2057,2127,2130,2135,2239,2244,2377,2380,2384,2387,2423,2426,2430,2433,2450,2453,2467,2471,2474,2520,2523,2531,2537,2545,2549,2552,2555,2561],[10,11,13],"h2",{"id":12},"background","Background",[15,16,17,18,23],"p",{},"Back in 2024 I wrote ",[19,20,22],"a",{"href":21},"\u002Fblog\u002Fload-balancing-long-lived-connections-in-kubernetes","Load Balancing Long Lived Connections in Kubernetes",". The short version: long-lived HTTP\u002F2 connections from a .NET service to a Kubernetes Service do not spread across the upstream pods the way most people assume. DNS is consulted once. kube-proxy picks a backend. After that, every request rides the same TCP connection to the same pod until the handler decides to recycle.",[15,25,26],{},"That post ended with four workarounds: a client-side load balancer written by hand, a service mesh, scaling down so it stops mattering, or shortening the pooled connection lifetime. Useful, but the first is a lot of code to maintain, and the others involve tradeoffs that not every team wants.",[15,28,29,30,34,35,38],{},"The lightweight option I always reached for in real services was the one I never properly packaged: register N named ",[31,32,33],"code",{},"HttpClient"," instances for the same logical upstream, give each its own ",[31,36,37],{},"SocketsHttpHandler"," and its own connection pool, then round-robin (or weight, or health-check) across them. Each pool independently does its own DNS lookup, opens its own TCP connection, and lands on whichever backend kube-proxy hands it. Not perfect, but each pool gets a fresh chance at the load balancer.",[15,40,41,42,44],{},"That hand-written pattern is what CloudHttp is now. Plus the ",[31,43,37],{}," defaults I keep typing in every new service, plus a few small ergonomic helpers around it.",[15,46,47],{},[19,48,49],{"href":49,"rel":50},"https:\u002F\u002Fgithub.com\u002Fhaiilong\u002FCloudHttp",[51],"nofollow",[53,54,59],"pre",{"className":55,"code":56,"language":57,"meta":58,"style":58},"language-sh shiki shiki-themes one-light one-dark-pro","dotnet add package haiilong.http.extensions\n","sh","",[31,60,61],{"__ignoreMap":58},[62,63,66],"span",{"class":64,"line":65},"line",1,[62,67,56],{},[10,69,71],{"id":70},"the-problem-more-concretely","The problem more concretely",[15,73,74],{},"A typical .NET service in Kubernetes calls another service through a cluster DNS name:",[53,76,80],{"className":77,"code":78,"language":79,"meta":58,"style":58},"language-csharp shiki shiki-themes one-light one-dark-pro","services.AddHttpClient\u003CInventoryClient>(c =>\n{\n    c.BaseAddress = new Uri(\"https:\u002F\u002Finventory.svc.cluster.local\");\n});\n","csharp",[31,81,82,112,118,149],{"__ignoreMap":58},[62,83,84,88,92,96,99,103,106,109],{"class":64,"line":65},[62,85,87],{"class":86},"s7GmK","services",[62,89,91],{"class":90},"s5ixo",".",[62,93,95],{"class":94},"sAdtL","AddHttpClient",[62,97,98],{"class":90},"\u003C",[62,100,102],{"class":101},"sC09Y","InventoryClient",[62,104,105],{"class":90},">(",[62,107,108],{"class":86},"c",[62,110,111],{"class":90}," =>\n",[62,113,115],{"class":64,"line":114},2,[62,116,117],{"class":90},"{\n",[62,119,121,124,126,129,133,136,139,142,146],{"class":64,"line":120},3,[62,122,123],{"class":86},"    c",[62,125,91],{"class":90},[62,127,128],{"class":86},"BaseAddress",[62,130,132],{"class":131},"sknuh"," =",[62,134,135],{"class":90}," new ",[62,137,138],{"class":101},"Uri",[62,140,141],{"class":90},"(",[62,143,145],{"class":144},"sDhpE","\"https:\u002F\u002Finventory.svc.cluster.local\"",[62,147,148],{"class":90},");\n",[62,150,152],{"class":64,"line":151},4,[62,153,154],{"class":90},"});\n",[15,156,157,158,161],{},"DNS resolves the Service to the kube-proxy ClusterIP. kube-proxy probabilistically picks one backend pod. The TCP connection is set up. ",[31,159,160],{},"IHttpClientFactory"," keeps the handler alive for two minutes by default. With HTTP\u002F2 multiplexing, every request from this caller pod rides the same TCP connection for those two minutes, and every one of those requests lands on the same backend pod.",[15,163,164],{},"Even if the upstream Service has ten replicas, this caller-pod-to-upstream-replica edge is sticky.",[15,166,167],{},"Two reasonable workarounds at the connection layer:",[169,170,171,179],"ol",{},[172,173,174,175,178],"li",{},"Shorten the pool lifetime so the connection rotates more often, which gives kube-proxy a fresh chance to pick a different backend. This is what ",[31,176,177],{},"PooledConnectionLifetime = 2 minutes"," does.",[172,180,181],{},"Run several connection pools in parallel so different requests from the same caller pod can ride different connections. Three pools means three chances at the load balancer instead of one.",[15,183,184],{},"CloudHttp does the second, and pulls in the first for free via the cloud-tuned defaults.",[10,186,188],{"id":187},"wiring-it-up","Wiring it up",[15,190,191,192,195],{},"The entry point is ",[31,193,194],{},"DistributedHttpClient",". Register it like a normal HTTP client, but with a count:",[53,197,199],{"className":77,"code":198,"language":79,"meta":58,"style":58},"builder.Services.AddDistributedHttpClient(\n    name: \"inventory\",\n    configureOptions: opts =>\n    {\n        opts.Mode = DistributionMode.RoundRobin;\n        opts.ClientCount = 4;\n    },\n    configureClient: c =>\n    {\n        c.BaseAddress = new Uri(\"https:\u002F\u002Finventory.svc.cluster.local\");\n        c.DefaultRequestHeaders.Accept.Add(new(\"application\u002Fjson\"));\n    },\n    configureBuilder: clientBuilder =>\n    {\n        clientBuilder.AddStandardResilienceHandler();\n    });\n",[31,200,201,219,233,245,250,274,292,298,310,315,337,366,371,384,389,403],{"__ignoreMap":58},[62,202,203,206,208,211,213,216],{"class":64,"line":65},[62,204,205],{"class":86},"builder",[62,207,91],{"class":90},[62,209,210],{"class":86},"Services",[62,212,91],{"class":90},[62,214,215],{"class":94},"AddDistributedHttpClient",[62,217,218],{"class":90},"(\n",[62,220,221,224,227,230],{"class":64,"line":114},[62,222,223],{"class":86},"    name",[62,225,226],{"class":90},": ",[62,228,229],{"class":144},"\"inventory\"",[62,231,232],{"class":90},",\n",[62,234,235,238,240,243],{"class":64,"line":120},[62,236,237],{"class":86},"    configureOptions",[62,239,226],{"class":90},[62,241,242],{"class":86},"opts",[62,244,111],{"class":90},[62,246,247],{"class":64,"line":151},[62,248,249],{"class":90},"    {\n",[62,251,253,256,258,261,263,266,268,271],{"class":64,"line":252},5,[62,254,255],{"class":86},"        opts",[62,257,91],{"class":90},[62,259,260],{"class":86},"Mode",[62,262,132],{"class":131},[62,264,265],{"class":86}," DistributionMode",[62,267,91],{"class":90},[62,269,270],{"class":86},"RoundRobin",[62,272,273],{"class":90},";\n",[62,275,277,279,281,284,286,290],{"class":64,"line":276},6,[62,278,255],{"class":86},[62,280,91],{"class":90},[62,282,283],{"class":86},"ClientCount",[62,285,132],{"class":131},[62,287,289],{"class":288},"sAGMh"," 4",[62,291,273],{"class":90},[62,293,295],{"class":64,"line":294},7,[62,296,297],{"class":90},"    },\n",[62,299,301,304,306,308],{"class":64,"line":300},8,[62,302,303],{"class":86},"    configureClient",[62,305,226],{"class":90},[62,307,108],{"class":86},[62,309,111],{"class":90},[62,311,313],{"class":64,"line":312},9,[62,314,249],{"class":90},[62,316,318,321,323,325,327,329,331,333,335],{"class":64,"line":317},10,[62,319,320],{"class":86},"        c",[62,322,91],{"class":90},[62,324,128],{"class":86},[62,326,132],{"class":131},[62,328,135],{"class":90},[62,330,138],{"class":101},[62,332,141],{"class":90},[62,334,145],{"class":144},[62,336,148],{"class":90},[62,338,340,342,344,347,349,352,354,357,360,363],{"class":64,"line":339},11,[62,341,320],{"class":86},[62,343,91],{"class":90},[62,345,346],{"class":86},"DefaultRequestHeaders",[62,348,91],{"class":90},[62,350,351],{"class":86},"Accept",[62,353,91],{"class":90},[62,355,356],{"class":94},"Add",[62,358,359],{"class":90},"(new(",[62,361,362],{"class":144},"\"application\u002Fjson\"",[62,364,365],{"class":90},"));\n",[62,367,369],{"class":64,"line":368},12,[62,370,297],{"class":90},[62,372,374,377,379,382],{"class":64,"line":373},13,[62,375,376],{"class":86},"    configureBuilder",[62,378,226],{"class":90},[62,380,381],{"class":86},"clientBuilder",[62,383,111],{"class":90},[62,385,387],{"class":64,"line":386},14,[62,388,249],{"class":90},[62,390,392,395,397,400],{"class":64,"line":391},15,[62,393,394],{"class":86},"        clientBuilder",[62,396,91],{"class":90},[62,398,399],{"class":94},"AddStandardResilienceHandler",[62,401,402],{"class":90},"();\n",[62,404,406],{"class":64,"line":405},16,[62,407,408],{"class":90},"    });\n",[15,410,411,412,415,416,419,420,422,423,425,426,428],{},"Under the hood this creates four named clients (",[31,413,414],{},"inventory#0"," through ",[31,417,418],{},"inventory#3","), each with its own ",[31,421,37],{}," and its own connection pool. The ",[31,424,194],{}," itself is registered as a keyed singleton, and you inject it where you would normally inject ",[31,427,33],{},":",[53,430,432],{"className":77,"code":431,"language":79,"meta":58,"style":58},"public sealed class InventoryService(\n    [FromKeyedServices(\"inventory\")] DistributedHttpClient http)\n{\n    public Task\u003CStockLevel?> GetStockAsync(string sku, CancellationToken ct)\n    {\n        var path = HttpRouteBuilder.BuildPath(\n            \"\u002Fstock\u002F{sku}\",\n            new Dictionary\u003Cstring, object?> { [\"sku\"] = sku });\n\n        return http.GetAsync\u003CStockLevel>(path, ct);\n    }\n}\n",[31,433,434,451,474,478,516,520,541,548,582,588,616,621],{"__ignoreMap":58},[62,435,436,440,443,446,449],{"class":64,"line":65},[62,437,439],{"class":438},"sLKXg","public",[62,441,442],{"class":438}," sealed",[62,444,445],{"class":438}," class",[62,447,448],{"class":101}," InventoryService",[62,450,218],{"class":90},[62,452,453,456,459,461,463,466,468,471],{"class":64,"line":114},[62,454,455],{"class":90},"    [",[62,457,458],{"class":101},"FromKeyedServices",[62,460,141],{"class":90},[62,462,229],{"class":144},[62,464,465],{"class":90},")] ",[62,467,194],{"class":101},[62,469,470],{"class":86}," http",[62,472,473],{"class":90},")\n",[62,475,476],{"class":64,"line":120},[62,477,117],{"class":90},[62,479,480,483,486,488,491,494,497,499,502,505,508,511,514],{"class":64,"line":151},[62,481,482],{"class":438},"    public",[62,484,485],{"class":101}," Task",[62,487,98],{"class":90},[62,489,490],{"class":101},"StockLevel",[62,492,493],{"class":90},"?> ",[62,495,496],{"class":94},"GetStockAsync",[62,498,141],{"class":90},[62,500,501],{"class":438},"string",[62,503,504],{"class":86}," sku",[62,506,507],{"class":90},", ",[62,509,510],{"class":101},"CancellationToken",[62,512,513],{"class":86}," ct",[62,515,473],{"class":90},[62,517,518],{"class":64,"line":252},[62,519,249],{"class":90},[62,521,522,525,529,531,534,536,539],{"class":64,"line":276},[62,523,524],{"class":438},"        var",[62,526,528],{"class":527},"sz0mV"," path",[62,530,132],{"class":131},[62,532,533],{"class":86}," HttpRouteBuilder",[62,535,91],{"class":90},[62,537,538],{"class":94},"BuildPath",[62,540,218],{"class":90},[62,542,543,546],{"class":64,"line":294},[62,544,545],{"class":144},"            \"\u002Fstock\u002F{sku}\"",[62,547,232],{"class":90},[62,549,550,553,556,558,560,562,565,568,571,574,577,579],{"class":64,"line":300},[62,551,552],{"class":90},"            new ",[62,554,555],{"class":101},"Dictionary",[62,557,98],{"class":90},[62,559,501],{"class":438},[62,561,507],{"class":90},[62,563,564],{"class":438},"object",[62,566,567],{"class":90},"?> { [",[62,569,570],{"class":144},"\"sku\"",[62,572,573],{"class":90},"] ",[62,575,576],{"class":131},"=",[62,578,504],{"class":527},[62,580,581],{"class":90}," });\n",[62,583,584],{"class":64,"line":312},[62,585,587],{"emptyLinePlaceholder":586},true,"\n",[62,589,590,593,595,597,600,602,604,606,609,611,614],{"class":64,"line":317},[62,591,592],{"class":438},"        return",[62,594,470],{"class":86},[62,596,91],{"class":90},[62,598,599],{"class":94},"GetAsync",[62,601,98],{"class":90},[62,603,490],{"class":101},[62,605,105],{"class":90},[62,607,608],{"class":527},"path",[62,610,507],{"class":90},[62,612,613],{"class":527},"ct",[62,615,148],{"class":90},[62,617,618],{"class":64,"line":339},[62,619,620],{"class":90},"    }\n",[62,622,623],{"class":64,"line":368},[62,624,625],{"class":90},"}\n",[15,627,628,629,632,633,635],{},"Every call to ",[31,630,631],{},"http.GetAsync\u003CT>(...)"," picks one of the four underlying named clients based on the distribution mode, sends the request, and returns. From the caller's perspective, this looks identical to a normal ",[31,634,33],{}," call.",[15,637,638,639,642,643,646],{},"The four underlying handlers each get the cloud-tuned defaults from ",[31,640,641],{},"ConfigureForCloud()",". If you want to tweak them per pool, there is a ",[31,644,645],{},"configurePrimaryHandler"," callback that runs once for each handler, after the defaults:",[53,648,650],{"className":77,"code":649,"language":79,"meta":58,"style":58},"configurePrimaryHandler: handler =>\n{\n    handler.ConnectTimeout = TimeSpan.FromSeconds(3);\n    handler.MaxConnectionsPerServer = 200;\n}\n",[31,651,652,663,667,694,710],{"__ignoreMap":58},[62,653,654,656,658,661],{"class":64,"line":65},[62,655,645],{"class":527},[62,657,226],{"class":90},[62,659,660],{"class":86},"handler",[62,662,111],{"class":90},[62,664,665],{"class":64,"line":114},[62,666,117],{"class":90},[62,668,669,672,674,677,679,682,684,687,689,692],{"class":64,"line":120},[62,670,671],{"class":86},"    handler",[62,673,91],{"class":90},[62,675,676],{"class":86},"ConnectTimeout",[62,678,132],{"class":131},[62,680,681],{"class":86}," TimeSpan",[62,683,91],{"class":90},[62,685,686],{"class":94},"FromSeconds",[62,688,141],{"class":90},[62,690,691],{"class":288},"3",[62,693,148],{"class":90},[62,695,696,698,700,703,705,708],{"class":64,"line":151},[62,697,671],{"class":86},[62,699,91],{"class":90},[62,701,702],{"class":86},"MaxConnectionsPerServer",[62,704,132],{"class":131},[62,706,707],{"class":288}," 200",[62,709,273],{"class":90},[62,711,712],{"class":64,"line":252},[62,713,625],{"class":90},[10,715,717],{"id":716},"the-three-distribution-modes","The three distribution modes",[719,720,722],"h3",{"id":721},"round-robin","Round-robin",[15,724,725,726,729,730,733],{},"This is the default mode. An atomic counter increments on every call and the index is ",[31,727,728],{},"counter % ClientCount",". The increment is lock-free via ",[31,731,732],{},"Interlocked.Increment",", with no allocation and no tuning required. Round-robin is the right choice when the upstream pods are interchangeable and roughly equal in capacity.",[53,735,737],{"className":77,"code":736,"language":79,"meta":58,"style":58},"services.AddRoundRobinDistribution(\n    name: \"payments\",\n    clientCount: 4,\n    configureClient: c => c.BaseAddress = new Uri(\"https:\u002F\u002Fpayments.svc.cluster.local\"));\n",[31,738,739,750,761,773],{"__ignoreMap":58},[62,740,741,743,745,748],{"class":64,"line":65},[62,742,87],{"class":86},[62,744,91],{"class":90},[62,746,747],{"class":94},"AddRoundRobinDistribution",[62,749,218],{"class":90},[62,751,752,754,756,759],{"class":64,"line":114},[62,753,223],{"class":86},[62,755,226],{"class":90},[62,757,758],{"class":144},"\"payments\"",[62,760,232],{"class":90},[62,762,763,766,768,771],{"class":64,"line":120},[62,764,765],{"class":86},"    clientCount",[62,767,226],{"class":90},[62,769,770],{"class":288},"4",[62,772,232],{"class":90},[62,774,775,777,779,781,784,786,788,790,792,794,796,798,801],{"class":64,"line":151},[62,776,303],{"class":86},[62,778,226],{"class":90},[62,780,108],{"class":86},[62,782,783],{"class":90}," => ",[62,785,108],{"class":86},[62,787,91],{"class":90},[62,789,128],{"class":86},[62,791,132],{"class":131},[62,793,135],{"class":90},[62,795,138],{"class":101},[62,797,141],{"class":90},[62,799,800],{"class":144},"\"https:\u002F\u002Fpayments.svc.cluster.local\"",[62,802,365],{"class":90},[719,804,806],{"id":805},"weighted","Weighted",[15,808,809,810,813],{},"Weighted distribution is useful for canary deployments or mixed-capacity pools. Each client index gets a relative weight, and selection is ",[31,811,812],{},"Random.Shared.NextDouble() * totalWeight"," plus a binary search into a sorted cumulative ladder.",[53,815,817],{"className":77,"code":816,"language":79,"meta":58,"style":58},"services.AddWeightedDistribution(\n    name: \"search\",\n    weights: new Dictionary\u003Cint, double> { [0] = 9, [1] = 1 },\n    configureClient: c => c.BaseAddress = new Uri(\"https:\u002F\u002Fsearch.svc.cluster.local\"));\n",[31,818,819,830,841,890],{"__ignoreMap":58},[62,820,821,823,825,828],{"class":64,"line":65},[62,822,87],{"class":86},[62,824,91],{"class":90},[62,826,827],{"class":94},"AddWeightedDistribution",[62,829,218],{"class":90},[62,831,832,834,836,839],{"class":64,"line":114},[62,833,223],{"class":86},[62,835,226],{"class":90},[62,837,838],{"class":144},"\"search\"",[62,840,232],{"class":90},[62,842,843,846,849,851,853,856,858,861,864,867,869,871,874,877,880,882,884,887],{"class":64,"line":120},[62,844,845],{"class":86},"    weights",[62,847,848],{"class":90},": new ",[62,850,555],{"class":101},[62,852,98],{"class":90},[62,854,855],{"class":438},"int",[62,857,507],{"class":90},[62,859,860],{"class":438},"double",[62,862,863],{"class":90},"> { [",[62,865,866],{"class":288},"0",[62,868,573],{"class":90},[62,870,576],{"class":131},[62,872,873],{"class":288}," 9",[62,875,876],{"class":90},", [",[62,878,879],{"class":288},"1",[62,881,573],{"class":90},[62,883,576],{"class":131},[62,885,886],{"class":288}," 1",[62,888,889],{"class":90}," },\n",[62,891,892,894,896,898,900,902,904,906,908,910,912,914,917],{"class":64,"line":151},[62,893,303],{"class":86},[62,895,226],{"class":90},[62,897,108],{"class":86},[62,899,783],{"class":90},[62,901,108],{"class":86},[62,903,91],{"class":90},[62,905,128],{"class":86},[62,907,132],{"class":131},[62,909,135],{"class":90},[62,911,138],{"class":101},[62,913,141],{"class":90},[62,915,916],{"class":144},"\"https:\u002F\u002Fsearch.svc.cluster.local\"",[62,918,365],{"class":90},[15,920,921],{},"That example sends roughly 10% of traffic to the second pool. Useful when you want to dip into a different upstream gradually: a different cluster, a different version of a service, a different region.",[719,923,925],{"id":924},"health-aware","Health-aware",[15,927,928,929,932],{},"Health-aware mode is round-robin with a temporary degraded list. When a pool returns a transient status code or throws a transient exception, CloudHttp records ",[31,930,931],{},"now + HealthDegradedTimeout"," for that pool and skips it on subsequent picks until the timestamp expires. The default timeout is 30 seconds.",[53,934,936],{"className":77,"code":935,"language":79,"meta":58,"style":58},"services.AddHealthAwareDistribution(\n    name: \"inventory\",\n    clientCount: 4,\n    degradedTimeout: TimeSpan.FromSeconds(30),\n    configureClient: c => c.BaseAddress = new Uri(\"https:\u002F\u002Finventory.svc.cluster.local\"));\n",[31,937,938,949,959,969,991],{"__ignoreMap":58},[62,939,940,942,944,947],{"class":64,"line":65},[62,941,87],{"class":86},[62,943,91],{"class":90},[62,945,946],{"class":94},"AddHealthAwareDistribution",[62,948,218],{"class":90},[62,950,951,953,955,957],{"class":64,"line":114},[62,952,223],{"class":86},[62,954,226],{"class":90},[62,956,229],{"class":144},[62,958,232],{"class":90},[62,960,961,963,965,967],{"class":64,"line":120},[62,962,765],{"class":86},[62,964,226],{"class":90},[62,966,770],{"class":288},[62,968,232],{"class":90},[62,970,971,974,976,979,981,983,985,988],{"class":64,"line":151},[62,972,973],{"class":86},"    degradedTimeout",[62,975,226],{"class":90},[62,977,978],{"class":86},"TimeSpan",[62,980,91],{"class":90},[62,982,686],{"class":94},[62,984,141],{"class":90},[62,986,987],{"class":288},"30",[62,989,990],{"class":90},"),\n",[62,992,993,995,997,999,1001,1003,1005,1007,1009,1011,1013,1015,1017],{"class":64,"line":252},[62,994,303],{"class":86},[62,996,226],{"class":90},[62,998,108],{"class":86},[62,1000,783],{"class":90},[62,1002,108],{"class":86},[62,1004,91],{"class":90},[62,1006,128],{"class":86},[62,1008,132],{"class":131},[62,1010,135],{"class":90},[62,1012,138],{"class":101},[62,1014,141],{"class":90},[62,1016,145],{"class":144},[62,1018,365],{"class":90},[15,1020,1021],{},"Recovery is purely time-based. There is no positive \"this pool is healthy again\" signal that immediately reinstates it. Concurrent in-flight requests complete out of order, and a stale healthy marker should not overwrite a newer degraded one.",[15,1023,1024,1025,1028,1029,1032],{},"The clock is ",[31,1026,1027],{},"Environment.TickCount64"," rather than ",[31,1030,1031],{},"DateTime.UtcNow",", so NTP nudges to the wall clock do not corrupt the degradation state.",[15,1034,1035],{},"If every pool ends up degraded at the same time, the selector falls back to round-robin instead of failing outright. Calling a possibly-degraded pool is better than refusing to call anything.",[10,1037,1039],{"id":1038},"a-reality-check-on-what-this-can-do","A reality check on what this can do",[15,1041,1042],{},"Several sentences in this post say \"more chances at the load balancer\" rather than \"always different backends\". That hedging is real and worth being honest about.",[15,1044,1045],{},"The N-pool setup gives kube-proxy N opportunities to pick different backends. If the upstream Service has only two replicas and you have four pools, by the pigeonhole principle at least two pools share a backend. If kube-proxy's hash function happens to map two pools to the same backend, you live with that for the next connection lifetime.",[15,1047,1048],{},"A proper service mesh sidecar (Envoy under Istio, the Linkerd2-proxy) does L7 client-side load balancing and will distribute each request reliably. If you have a mesh, prefer the mesh. CloudHttp is the workaround for when you do not have one, cannot write a custom client-side load balancer, and still want better odds than a single TCP connection's worth of luck.",[10,1050,1052],{"id":1051},"cloud-tuned-handler-defaults","Cloud-tuned handler defaults",[15,1054,1055,1056,1058,1059,1061],{},"The other thing the library does is bundle up the ",[31,1057,37],{}," defaults I have ended up typing into every cloud service for the last few years. They are applied automatically inside ",[31,1060,215],{},", but you can also use them on their own with a normal named client:",[53,1063,1065],{"className":77,"code":1064,"language":79,"meta":58,"style":58},"services.AddHttpClient(\"orders\", c => c.BaseAddress = new Uri(\"https:\u002F\u002Forders.svc\"))\n    .ConfigureForCloud();\n",[31,1066,1067,1106],{"__ignoreMap":58},[62,1068,1069,1071,1073,1075,1077,1080,1082,1084,1086,1088,1090,1092,1094,1096,1098,1100,1103],{"class":64,"line":65},[62,1070,87],{"class":86},[62,1072,91],{"class":90},[62,1074,95],{"class":94},[62,1076,141],{"class":90},[62,1078,1079],{"class":144},"\"orders\"",[62,1081,507],{"class":90},[62,1083,108],{"class":86},[62,1085,783],{"class":90},[62,1087,108],{"class":86},[62,1089,91],{"class":90},[62,1091,128],{"class":86},[62,1093,132],{"class":131},[62,1095,135],{"class":90},[62,1097,138],{"class":101},[62,1099,141],{"class":90},[62,1101,1102],{"class":144},"\"https:\u002F\u002Forders.svc\"",[62,1104,1105],{"class":90},"))\n",[62,1107,1108,1111,1114],{"class":64,"line":114},[62,1109,1110],{"class":90},"    .",[62,1112,1113],{"class":94},"ConfigureForCloud",[62,1115,402],{"class":90},[15,1117,1118],{},"The values it picks, and why:",[1120,1121,1122,1138],"table",{},[1123,1124,1125],"thead",{},[1126,1127,1128,1132,1135],"tr",{},[1129,1130,1131],"th",{},"Property",[1129,1133,1134],{},"Value",[1129,1136,1137],{},"What it gets you",[1139,1140,1141,1155,1167,1179,1196,1209,1222,1235,1248,1261],"tbody",{},[1126,1142,1143,1149,1152],{},[1144,1145,1146],"td",{},[31,1147,1148],{},"PooledConnectionLifetime",[1144,1150,1151],{},"2 minutes",[1144,1153,1154],{},"DNS refresh on the cadence of rolling deploys; the .NET default is infinite, which means a connection opened on day 1 still hits the same pod on day 30.",[1126,1156,1157,1161,1164],{},[1144,1158,1159],{},[31,1160,676],{},[1144,1162,1163],{},"5 seconds",[1144,1165,1166],{},"Fail fast on broken routes. The .NET default is also infinite, which is the wrong shape for cluster traffic.",[1126,1168,1169,1173,1176],{},[1144,1170,1171],{},[31,1172,702],{},[1144,1174,1175],{},"100",[1144,1177,1178],{},"Bounded concurrency per origin, with enough headroom for bursty service traffic.",[1126,1180,1181,1186,1189],{},[1144,1182,1183],{},[31,1184,1185],{},"AutomaticDecompression",[1144,1187,1188],{},"All",[1144,1190,1191,1192,1195],{},"gzip, deflate, brotli (and zstd on .NET 10). Adds ",[31,1193,1194],{},"Accept-Encoding"," automatically.",[1126,1197,1198,1203,1206],{},[1144,1199,1200],{},[31,1201,1202],{},"EnableMultipleHttp2Connections",[1144,1204,1205],{},"true",[1144,1207,1208],{},"Lets the handler open another HTTP\u002F2 connection when stream limits saturate.",[1126,1210,1211,1216,1219],{},[1144,1212,1213],{},[31,1214,1215],{},"InitialHttp2StreamWindowSize",[1144,1217,1218],{},"128 KiB",[1144,1220,1221],{},"Larger per-stream flow-control window. Fewer round trips on non-trivial response bodies.",[1126,1223,1224,1229,1232],{},[1144,1225,1226],{},[31,1227,1228],{},"KeepAlivePingDelay",[1144,1230,1231],{},"30 seconds",[1144,1233,1234],{},"Detect dead connections proactively. The default is infinite, meaning no pings at all.",[1126,1236,1237,1242,1245],{},[1144,1238,1239],{},[31,1240,1241],{},"KeepAlivePingTimeout",[1144,1243,1244],{},"10 seconds",[1144,1246,1247],{},"How long to wait for a pong before declaring the connection dead.",[1126,1249,1250,1255,1258],{},[1144,1251,1252],{},[31,1253,1254],{},"KeepAlivePingPolicy",[1144,1256,1257],{},"WithActiveRequests",[1144,1259,1260],{},"Only ping while requests are in flight. The default pings idle connections too, which is wasted work.",[1126,1262,1263,1268,1270],{},[1144,1264,1265],{},[31,1266,1267],{},"ResponseDrainTimeout",[1144,1269,1163],{},[1144,1271,1272],{},"Bound the time spent draining an unread response body when a request is disposed. The .NET default is 2 seconds; 5 is more pool-friendly.",[15,1274,1275,1276,1279],{},"The builder version (",[31,1277,1278],{},"IHttpClientBuilder.ConfigureForCloud()",") adds two more knobs that only make sense at the factory layer:",[1120,1281,1282,1292],{},[1123,1283,1284],{},[1126,1285,1286,1288,1290],{},[1129,1287,1131],{},[1129,1289,1134],{},[1129,1291,1137],{},[1139,1293,1294,1306],{},[1126,1295,1296,1301,1303],{},[1144,1297,1298],{},[31,1299,1300],{},"HttpClient.Timeout",[1144,1302,1231],{},[1144,1304,1305],{},"Bounded total per-request time. The .NET default is 100 seconds, which is far too long for cluster-internal calls.",[1126,1307,1308,1311,1316],{},[1144,1309,1310],{},"Factory handler lifetime",[1144,1312,1313],{},[31,1314,1315],{},"Timeout.InfiniteTimeSpan",[1144,1317,1318,1319,1321,1322,1324],{},"Stops ",[31,1320,160],{}," from rotating handlers on its own schedule. ",[31,1323,1148],{}," handles connection recycling instead.",[15,1326,1327,1328,1330,1331,1333,1334,1338,1339,1341],{},"The last one matters more than it looks. By default, ",[31,1329,160],{}," rotates the whole handler every two minutes for DNS-refresh reasons. If you also set ",[31,1332,177],{}," on the handler, you are paying twice: the factory throws the whole pool away every two minutes, AND each connection has its own two-minute clock that never gets to run because the handler dies first. I wrote about this in more detail in ",[19,1335,1337],{"href":1336},"\u002Fblog\u002Fhttpclient-connection-lifetime-observed","HttpClient connection lifetime, observed","; the short version is \"pin the factory lifetime to infinite, let ",[31,1340,1148],{}," do the rotation\".",[15,1343,1344,1345,1350,1351,1354],{},"Each setting has a longer explanation in ",[19,1346,1349],{"href":1347,"rel":1348},"https:\u002F\u002Fgithub.com\u002Fhaiilong\u002FCloudHttp\u002Fblob\u002Fmain\u002Fdocs\u002Fcloud-defaults.md",[51],"docs\u002Fcloud-defaults.md"," inside the repo. Every default is overridable through the ",[31,1352,1353],{},"customize"," callback if your case is different.",[10,1356,1358],{"id":1357},"composition-with-microsoftextensionshttpresilience","Composition with Microsoft.Extensions.Http.Resilience",[15,1360,1361,1362,1369],{},"CloudHttp does pool selection. It deliberately does not do retries, backoff, jitter, or circuit breaking. Microsoft's ",[19,1363,1366],{"href":1364,"rel":1365},"https:\u002F\u002Flearn.microsoft.com\u002Fdotnet\u002Fcore\u002Fresilience\u002Fhttp-resilience",[51],[31,1367,1368],{},"Microsoft.Extensions.Http.Resilience"," package does all of that on top of Polly v8, and it does it well.",[15,1371,1372,1373,1376],{},"The two libraries compose cleanly. The ",[31,1374,1375],{},"configureBuilder"," callback runs per underlying named client, so the resilience handler attaches inside each pool:",[53,1378,1380],{"className":77,"code":1379,"language":79,"meta":58,"style":58},"services.AddDistributedHttpClient(\"payments\",\n    configureOptions: opts => opts.ClientCount = 4,\n    configureClient: c => c.BaseAddress = new Uri(\"https:\u002F\u002Fpayments.svc\"),\n    configureBuilder: cb => cb.AddStandardResilienceHandler(o =>\n    {\n        o.Retry.MaxRetryAttempts = 3;\n        o.Retry.UseJitter = true;\n        o.CircuitBreaker.FailureRatio = 0.2;\n        o.CircuitBreaker.BreakDuration = TimeSpan.FromSeconds(30);\n\n        o.AttemptTimeout.Timeout = TimeSpan.FromSeconds(5);\n        o.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(30);\n    }));\n",[31,1381,1382,1396,1418,1447,1471,1475,1497,1517,1538,1565,1569,1598,1625],{"__ignoreMap":58},[62,1383,1384,1386,1388,1390,1392,1394],{"class":64,"line":65},[62,1385,87],{"class":86},[62,1387,91],{"class":90},[62,1389,215],{"class":94},[62,1391,141],{"class":90},[62,1393,758],{"class":144},[62,1395,232],{"class":90},[62,1397,1398,1400,1402,1404,1406,1408,1410,1412,1414,1416],{"class":64,"line":114},[62,1399,237],{"class":86},[62,1401,226],{"class":90},[62,1403,242],{"class":86},[62,1405,783],{"class":90},[62,1407,242],{"class":86},[62,1409,91],{"class":90},[62,1411,283],{"class":86},[62,1413,132],{"class":131},[62,1415,289],{"class":288},[62,1417,232],{"class":90},[62,1419,1420,1422,1424,1426,1428,1430,1432,1434,1436,1438,1440,1442,1445],{"class":64,"line":120},[62,1421,303],{"class":86},[62,1423,226],{"class":90},[62,1425,108],{"class":86},[62,1427,783],{"class":90},[62,1429,108],{"class":86},[62,1431,91],{"class":90},[62,1433,128],{"class":86},[62,1435,132],{"class":131},[62,1437,135],{"class":90},[62,1439,138],{"class":101},[62,1441,141],{"class":90},[62,1443,1444],{"class":144},"\"https:\u002F\u002Fpayments.svc\"",[62,1446,990],{"class":90},[62,1448,1449,1451,1453,1456,1458,1460,1462,1464,1466,1469],{"class":64,"line":151},[62,1450,376],{"class":86},[62,1452,226],{"class":90},[62,1454,1455],{"class":86},"cb",[62,1457,783],{"class":90},[62,1459,1455],{"class":86},[62,1461,91],{"class":90},[62,1463,399],{"class":94},[62,1465,141],{"class":90},[62,1467,1468],{"class":86},"o",[62,1470,111],{"class":90},[62,1472,1473],{"class":64,"line":252},[62,1474,249],{"class":90},[62,1476,1477,1480,1482,1485,1487,1490,1492,1495],{"class":64,"line":276},[62,1478,1479],{"class":86},"        o",[62,1481,91],{"class":90},[62,1483,1484],{"class":86},"Retry",[62,1486,91],{"class":90},[62,1488,1489],{"class":86},"MaxRetryAttempts",[62,1491,132],{"class":131},[62,1493,1494],{"class":288}," 3",[62,1496,273],{"class":90},[62,1498,1499,1501,1503,1505,1507,1510,1512,1515],{"class":64,"line":294},[62,1500,1479],{"class":86},[62,1502,91],{"class":90},[62,1504,1484],{"class":86},[62,1506,91],{"class":90},[62,1508,1509],{"class":86},"UseJitter",[62,1511,132],{"class":131},[62,1513,1514],{"class":288}," true",[62,1516,273],{"class":90},[62,1518,1519,1521,1523,1526,1528,1531,1533,1536],{"class":64,"line":300},[62,1520,1479],{"class":86},[62,1522,91],{"class":90},[62,1524,1525],{"class":86},"CircuitBreaker",[62,1527,91],{"class":90},[62,1529,1530],{"class":86},"FailureRatio",[62,1532,132],{"class":131},[62,1534,1535],{"class":288}," 0.2",[62,1537,273],{"class":90},[62,1539,1540,1542,1544,1546,1548,1551,1553,1555,1557,1559,1561,1563],{"class":64,"line":312},[62,1541,1479],{"class":86},[62,1543,91],{"class":90},[62,1545,1525],{"class":86},[62,1547,91],{"class":90},[62,1549,1550],{"class":86},"BreakDuration",[62,1552,132],{"class":131},[62,1554,681],{"class":86},[62,1556,91],{"class":90},[62,1558,686],{"class":94},[62,1560,141],{"class":90},[62,1562,987],{"class":288},[62,1564,148],{"class":90},[62,1566,1567],{"class":64,"line":317},[62,1568,587],{"emptyLinePlaceholder":586},[62,1570,1571,1573,1575,1578,1580,1583,1585,1587,1589,1591,1593,1596],{"class":64,"line":339},[62,1572,1479],{"class":86},[62,1574,91],{"class":90},[62,1576,1577],{"class":86},"AttemptTimeout",[62,1579,91],{"class":90},[62,1581,1582],{"class":86},"Timeout",[62,1584,132],{"class":131},[62,1586,681],{"class":86},[62,1588,91],{"class":90},[62,1590,686],{"class":94},[62,1592,141],{"class":90},[62,1594,1595],{"class":288},"5",[62,1597,148],{"class":90},[62,1599,1600,1602,1604,1607,1609,1611,1613,1615,1617,1619,1621,1623],{"class":64,"line":368},[62,1601,1479],{"class":86},[62,1603,91],{"class":90},[62,1605,1606],{"class":86},"TotalRequestTimeout",[62,1608,91],{"class":90},[62,1610,1582],{"class":86},[62,1612,132],{"class":131},[62,1614,681],{"class":86},[62,1616,91],{"class":90},[62,1618,686],{"class":94},[62,1620,141],{"class":90},[62,1622,987],{"class":288},[62,1624,148],{"class":90},[62,1626,1627],{"class":64,"line":373},[62,1628,1629],{"class":90},"    }));\n",[15,1631,1632,1633,1636],{},"The execution order for a ",[31,1634,1635],{},"GetAsync\u003CT>"," call:",[169,1638,1639,1645,1651,1654,1657],{},[172,1640,1641,1642,91],{},"The distributor picks client ",[31,1643,1644],{},"#0",[172,1646,1647,1648,1650],{},"The resilience pipeline on ",[31,1649,1644],{}," runs, including jittered retries. Each retry stays on the same pool.",[172,1652,1653],{},"If the response or exception is still transient after that pipeline, CloudHttp rotates once to a different client.",[172,1655,1656],{},"The resilience pipeline on the rotated client runs.",[172,1658,1659],{},"Whatever the second client returns is what the caller gets. No further rotation.",[15,1661,1662],{},"Each named client has its own circuit breaker, so a stuck pool does not drag the others down with it.",[15,1664,1665,1666,1669,1670,1673,1674,428],{},"The part of the math people miss: with ",[31,1667,1668],{},"MaxRetryAttempts = 3",", the caller can see up to eight attempts total (four on the first pool, four on the rotated pool). With ",[31,1671,1672],{},"c.Timeout = 30s",", the worst-case wall clock for two attempts is 60 seconds. If you want to bound the total wall clock, set it at the caller with ",[31,1675,1676],{},"CancellationTokenSource.CancelAfter",[53,1678,1680],{"className":77,"code":1679,"language":79,"meta":58,"style":58},"using var cts = CancellationTokenSource.CreateLinkedTokenSource(callerCt);\ncts.CancelAfter(TimeSpan.FromSeconds(30));\nawait http.GetAsync\u003CFoo>(\"\u002Fx\", cts.Token);\n",[31,1681,1682,1710,1734],{"__ignoreMap":58},[62,1683,1684,1687,1690,1693,1695,1698,1700,1703,1705,1708],{"class":64,"line":65},[62,1685,1686],{"class":438},"using",[62,1688,1689],{"class":438}," var",[62,1691,1692],{"class":527}," cts",[62,1694,132],{"class":131},[62,1696,1697],{"class":86}," CancellationTokenSource",[62,1699,91],{"class":90},[62,1701,1702],{"class":94},"CreateLinkedTokenSource",[62,1704,141],{"class":90},[62,1706,1707],{"class":527},"callerCt",[62,1709,148],{"class":90},[62,1711,1712,1715,1717,1720,1722,1724,1726,1728,1730,1732],{"class":64,"line":114},[62,1713,1714],{"class":86},"cts",[62,1716,91],{"class":90},[62,1718,1719],{"class":94},"CancelAfter",[62,1721,141],{"class":90},[62,1723,978],{"class":86},[62,1725,91],{"class":90},[62,1727,686],{"class":94},[62,1729,141],{"class":90},[62,1731,987],{"class":288},[62,1733,365],{"class":90},[62,1735,1736,1739,1742,1744,1746,1748,1751,1753,1756,1758,1760,1762,1765],{"class":64,"line":120},[62,1737,1738],{"class":90},"await ",[62,1740,1741],{"class":86},"http",[62,1743,91],{"class":90},[62,1745,599],{"class":94},[62,1747,98],{"class":90},[62,1749,1750],{"class":101},"Foo",[62,1752,105],{"class":90},[62,1754,1755],{"class":144},"\"\u002Fx\"",[62,1757,507],{"class":90},[62,1759,1714],{"class":86},[62,1761,91],{"class":90},[62,1763,1764],{"class":86},"Token",[62,1766,148],{"class":90},[15,1768,1769,1770,1772,1773,1776],{},"Caller cancellation always wins. CloudHttp does not rotate when the caller's ",[31,1771,510],{}," fires; it rethrows ",[31,1774,1775],{},"OperationCanceledException"," immediately.",[10,1778,1780],{"id":1779},"reads-can-rotate-writes-cannot","Reads can rotate, writes cannot",[15,1782,1783,1784,1786],{},"Two operations on ",[31,1785,194],{}," can rotate after a transient failure:",[1788,1789,1790,1795],"ul",{},[172,1791,1792],{},[31,1793,1794],{},"GetAsync\u003CT>(path, ct)",[172,1796,1797,1800],{},[31,1798,1799],{},"SendAsync(factory, ct)"," (the explicit \"build your own request\" version)",[15,1802,1803,1804,507,1807,507,1810,507,1813,1816,1817,1820],{},"The mutating JSON helpers (",[31,1805,1806],{},"PostAsync",[31,1808,1809],{},"PutAsync",[31,1811,1812],{},"PatchAsync",[31,1814,1815],{},"DeleteAsync",") do not auto-rotate. That is deliberate. A ",[31,1818,1819],{},"POST"," that times out may have already created the row, charged the card, or sent the email. Replaying it can duplicate side effects in ways CloudHttp has no way to detect from the outside.",[15,1822,1823,1824,1827],{},"If a write is genuinely safe to replay, make that explicit at the call site with an idempotency key and use ",[31,1825,1826],{},"SendAsync"," directly:",[53,1829,1831],{"className":77,"code":1830,"language":79,"meta":58,"style":58},"public Task\u003CHttpResponseMessage> ChargeAsync(\n    ChargeRequest body,\n    string idempotencyKey,\n    CancellationToken ct)\n{\n    return http.SendAsync((client, token) =>\n    {\n        using var request = new HttpRequestMessage(HttpMethod.Post, \"\u002Fcharges\")\n        {\n            Content = JsonContent.Create(body)\n        };\n        request.Headers.Add(\"Idempotency-Key\", idempotencyKey);\n        return client.SendAsync(request, token);\n    }, ct);\n}\n",[31,1832,1833,1852,1862,1872,1881,1885,1910,1914,1948,1953,1975,1980,2006,2028,2037],{"__ignoreMap":58},[62,1834,1835,1837,1839,1841,1844,1847,1850],{"class":64,"line":65},[62,1836,439],{"class":438},[62,1838,485],{"class":101},[62,1840,98],{"class":90},[62,1842,1843],{"class":101},"HttpResponseMessage",[62,1845,1846],{"class":90},"> ",[62,1848,1849],{"class":94},"ChargeAsync",[62,1851,218],{"class":90},[62,1853,1854,1857,1860],{"class":64,"line":114},[62,1855,1856],{"class":101},"    ChargeRequest",[62,1858,1859],{"class":86}," body",[62,1861,232],{"class":90},[62,1863,1864,1867,1870],{"class":64,"line":120},[62,1865,1866],{"class":438},"    string",[62,1868,1869],{"class":86}," idempotencyKey",[62,1871,232],{"class":90},[62,1873,1874,1877,1879],{"class":64,"line":151},[62,1875,1876],{"class":101},"    CancellationToken",[62,1878,513],{"class":86},[62,1880,473],{"class":90},[62,1882,1883],{"class":64,"line":252},[62,1884,117],{"class":90},[62,1886,1887,1890,1892,1894,1896,1899,1902,1904,1907],{"class":64,"line":276},[62,1888,1889],{"class":438},"    return",[62,1891,470],{"class":86},[62,1893,91],{"class":90},[62,1895,1826],{"class":94},[62,1897,1898],{"class":90},"((",[62,1900,1901],{"class":86},"client",[62,1903,507],{"class":90},[62,1905,1906],{"class":86},"token",[62,1908,1909],{"class":90},") =>\n",[62,1911,1912],{"class":64,"line":294},[62,1913,249],{"class":90},[62,1915,1916,1919,1921,1924,1926,1928,1931,1933,1936,1938,1941,1943,1946],{"class":64,"line":300},[62,1917,1918],{"class":438},"        using",[62,1920,1689],{"class":438},[62,1922,1923],{"class":527}," request",[62,1925,132],{"class":131},[62,1927,135],{"class":90},[62,1929,1930],{"class":101},"HttpRequestMessage",[62,1932,141],{"class":90},[62,1934,1935],{"class":86},"HttpMethod",[62,1937,91],{"class":90},[62,1939,1940],{"class":86},"Post",[62,1942,507],{"class":90},[62,1944,1945],{"class":144},"\"\u002Fcharges\"",[62,1947,473],{"class":90},[62,1949,1950],{"class":64,"line":312},[62,1951,1952],{"class":90},"        {\n",[62,1954,1955,1958,1960,1963,1965,1968,1970,1973],{"class":64,"line":317},[62,1956,1957],{"class":527},"            Content",[62,1959,132],{"class":131},[62,1961,1962],{"class":86}," JsonContent",[62,1964,91],{"class":90},[62,1966,1967],{"class":94},"Create",[62,1969,141],{"class":90},[62,1971,1972],{"class":527},"body",[62,1974,473],{"class":90},[62,1976,1977],{"class":64,"line":339},[62,1978,1979],{"class":90},"        };\n",[62,1981,1982,1985,1987,1990,1992,1994,1996,1999,2001,2004],{"class":64,"line":368},[62,1983,1984],{"class":86},"        request",[62,1986,91],{"class":90},[62,1988,1989],{"class":86},"Headers",[62,1991,91],{"class":90},[62,1993,356],{"class":94},[62,1995,141],{"class":90},[62,1997,1998],{"class":144},"\"Idempotency-Key\"",[62,2000,507],{"class":90},[62,2002,2003],{"class":527},"idempotencyKey",[62,2005,148],{"class":90},[62,2007,2008,2010,2013,2015,2017,2019,2022,2024,2026],{"class":64,"line":373},[62,2009,592],{"class":438},[62,2011,2012],{"class":86}," client",[62,2014,91],{"class":90},[62,2016,1826],{"class":94},[62,2018,141],{"class":90},[62,2020,2021],{"class":527},"request",[62,2023,507],{"class":90},[62,2025,1906],{"class":527},[62,2027,148],{"class":90},[62,2029,2030,2033,2035],{"class":64,"line":386},[62,2031,2032],{"class":90},"    }, ",[62,2034,613],{"class":527},[62,2036,148],{"class":90},[62,2038,2039],{"class":64,"line":391},[62,2040,625],{"class":90},[15,2042,2043],{},"The idempotency key belongs to the API contract between your caller and the upstream. CloudHttp cannot invent it.",[10,2045,2047],{"id":2046},"a-few-smaller-helpers","A few smaller helpers",[15,2049,2050],{},"Three small utilities that ended up in the library because they kept coming up in real service code:",[15,2052,2053,428],{},[2054,2055,2056],"strong",{},"Route templates",[53,2058,2060],{"className":77,"code":2059,"language":79,"meta":58,"style":58},"var path = HttpRouteBuilder.BuildPath(\n    \"\u002Fapi\u002Fv{ver}\u002Fusers\u002F{id}\",\n    new Dictionary\u003Cstring, object?> { [\"ver\"] = 2, [\"id\"] = userId });\n",[31,2061,2062,2079,2086],{"__ignoreMap":58},[62,2063,2064,2067,2069,2071,2073,2075,2077],{"class":64,"line":65},[62,2065,2066],{"class":438},"var",[62,2068,528],{"class":527},[62,2070,132],{"class":131},[62,2072,533],{"class":86},[62,2074,91],{"class":90},[62,2076,538],{"class":94},[62,2078,218],{"class":90},[62,2080,2081,2084],{"class":64,"line":114},[62,2082,2083],{"class":144},"    \"\u002Fapi\u002Fv{ver}\u002Fusers\u002F{id}\"",[62,2085,232],{"class":90},[62,2087,2088,2091,2093,2095,2097,2099,2101,2103,2106,2108,2110,2113,2115,2118,2120,2122,2125],{"class":64,"line":120},[62,2089,2090],{"class":90},"    new ",[62,2092,555],{"class":101},[62,2094,98],{"class":90},[62,2096,501],{"class":438},[62,2098,507],{"class":90},[62,2100,564],{"class":438},[62,2102,567],{"class":90},[62,2104,2105],{"class":144},"\"ver\"",[62,2107,573],{"class":90},[62,2109,576],{"class":131},[62,2111,2112],{"class":288}," 2",[62,2114,876],{"class":90},[62,2116,2117],{"class":144},"\"id\"",[62,2119,573],{"class":90},[62,2121,576],{"class":131},[62,2123,2124],{"class":527}," userId",[62,2126,581],{"class":90},[15,2128,2129],{},"It URL-encodes each value, uses no template engine, and allocates nothing per call beyond the final string.",[15,2131,2132,428],{},[2054,2133,2134],{},"Query merging",[53,2136,2138],{"className":77,"code":2137,"language":79,"meta":58,"style":58},"var uri = new Uri(\"https:\u002F\u002Fapi.example.com\u002Fsearch\")\n    .AddQuery(new[]\n    {\n        new KeyValuePair\u003Cstring, string?>(\"q\", searchTerm),\n        new KeyValuePair\u003Cstring, string?>(\"page\", page.ToString()),\n    });\n",[31,2139,2140,2160,2170,2174,2203,2235],{"__ignoreMap":58},[62,2141,2142,2144,2147,2149,2151,2153,2155,2158],{"class":64,"line":65},[62,2143,2066],{"class":438},[62,2145,2146],{"class":527}," uri",[62,2148,132],{"class":131},[62,2150,135],{"class":90},[62,2152,138],{"class":101},[62,2154,141],{"class":90},[62,2156,2157],{"class":144},"\"https:\u002F\u002Fapi.example.com\u002Fsearch\"",[62,2159,473],{"class":90},[62,2161,2162,2164,2167],{"class":64,"line":114},[62,2163,1110],{"class":90},[62,2165,2166],{"class":94},"AddQuery",[62,2168,2169],{"class":90},"(new[]\n",[62,2171,2172],{"class":64,"line":120},[62,2173,249],{"class":90},[62,2175,2176,2179,2182,2184,2186,2188,2190,2193,2196,2198,2201],{"class":64,"line":151},[62,2177,2178],{"class":90},"        new ",[62,2180,2181],{"class":101},"KeyValuePair",[62,2183,98],{"class":90},[62,2185,501],{"class":438},[62,2187,507],{"class":90},[62,2189,501],{"class":438},[62,2191,2192],{"class":90},"?>(",[62,2194,2195],{"class":144},"\"q\"",[62,2197,507],{"class":90},[62,2199,2200],{"class":527},"searchTerm",[62,2202,990],{"class":90},[62,2204,2205,2207,2209,2211,2213,2215,2217,2219,2222,2224,2227,2229,2232],{"class":64,"line":252},[62,2206,2178],{"class":90},[62,2208,2181],{"class":101},[62,2210,98],{"class":90},[62,2212,501],{"class":438},[62,2214,507],{"class":90},[62,2216,501],{"class":438},[62,2218,2192],{"class":90},[62,2220,2221],{"class":144},"\"page\"",[62,2223,507],{"class":90},[62,2225,2226],{"class":86},"page",[62,2228,91],{"class":90},[62,2230,2231],{"class":94},"ToString",[62,2233,2234],{"class":90},"()),\n",[62,2236,2237],{"class":64,"line":276},[62,2238,408],{"class":90},[15,2240,2241,428],{},[2054,2242,2243],{},"Best-effort error fallback",[53,2245,2247],{"className":77,"code":2246,"language":79,"meta":58,"style":58},"public async Task\u003CFeatureFlags> GetFlagsAsync(HttpClient client, ILogger logger, CancellationToken ct)\n{\n    return await client.GetWithErrorHandlingAsync(\n        \"\u002Fflags\",\n        defaultResponse: FeatureFlags.Empty,\n        logger: logger,\n        errorLogLevel: LogLevel.Warning,\n        ct: ct);\n}\n",[31,2248,2249,2290,2294,2310,2317,2333,2345,2362,2373],{"__ignoreMap":58},[62,2250,2251,2253,2256,2258,2260,2263,2265,2268,2270,2272,2274,2276,2279,2282,2284,2286,2288],{"class":64,"line":65},[62,2252,439],{"class":438},[62,2254,2255],{"class":438}," async",[62,2257,485],{"class":101},[62,2259,98],{"class":90},[62,2261,2262],{"class":101},"FeatureFlags",[62,2264,1846],{"class":90},[62,2266,2267],{"class":94},"GetFlagsAsync",[62,2269,141],{"class":90},[62,2271,33],{"class":101},[62,2273,2012],{"class":86},[62,2275,507],{"class":90},[62,2277,2278],{"class":101},"ILogger",[62,2280,2281],{"class":86}," logger",[62,2283,507],{"class":90},[62,2285,510],{"class":101},[62,2287,513],{"class":86},[62,2289,473],{"class":90},[62,2291,2292],{"class":64,"line":114},[62,2293,117],{"class":90},[62,2295,2296,2298,2301,2303,2305,2308],{"class":64,"line":120},[62,2297,1889],{"class":438},[62,2299,2300],{"class":90}," await ",[62,2302,1901],{"class":86},[62,2304,91],{"class":90},[62,2306,2307],{"class":94},"GetWithErrorHandlingAsync",[62,2309,218],{"class":90},[62,2311,2312,2315],{"class":64,"line":151},[62,2313,2314],{"class":144},"        \"\u002Fflags\"",[62,2316,232],{"class":90},[62,2318,2319,2322,2324,2326,2328,2331],{"class":64,"line":252},[62,2320,2321],{"class":86},"        defaultResponse",[62,2323,226],{"class":90},[62,2325,2262],{"class":86},[62,2327,91],{"class":90},[62,2329,2330],{"class":86},"Empty",[62,2332,232],{"class":90},[62,2334,2335,2338,2340,2343],{"class":64,"line":276},[62,2336,2337],{"class":86},"        logger",[62,2339,226],{"class":90},[62,2341,2342],{"class":527},"logger",[62,2344,232],{"class":90},[62,2346,2347,2350,2352,2355,2357,2360],{"class":64,"line":294},[62,2348,2349],{"class":86},"        errorLogLevel",[62,2351,226],{"class":90},[62,2353,2354],{"class":86},"LogLevel",[62,2356,91],{"class":90},[62,2358,2359],{"class":86},"Warning",[62,2361,232],{"class":90},[62,2363,2364,2367,2369,2371],{"class":64,"line":300},[62,2365,2366],{"class":86},"        ct",[62,2368,226],{"class":90},[62,2370,613],{"class":527},[62,2372,148],{"class":90},[62,2374,2375],{"class":64,"line":312},[62,2376,625],{"class":90},[15,2378,2379],{},"This is not a general error strategy. It is for non-critical calls where a fallback value is genuinely acceptable, like feature flags or optional metadata. Caller cancellation is always propagated; only HTTP failures, JSON errors, and I\u002FO errors fall through to the default value, and they always get logged.",[10,2381,2383],{"id":2382},"out-of-scope-on-purpose","Out of scope, on purpose",[15,2385,2386],{},"It's worth being explicit about what the library deliberately stays away from.",[1788,2388,2389,2395,2402,2405,2408],{},[172,2390,2391,2392,2394],{},"No retries, exponential backoff, jitter, or circuit breaking. Use ",[31,2393,1368],{}," and let CloudHttp compose with it.",[172,2396,2397,2398,2401],{},"No distributed health state across caller pods. Each replica of your caller service tracks its own health-aware degradations. Pool ",[31,2399,2400],{},"#2"," being marked degraded on pod A does not propagate to pod B.",[172,2403,2404],{},"No service discovery beyond cluster DNS. No Consul, no Kubernetes API integration, no Eureka.",[172,2406,2407],{},"No guarantee of even distribution. The N-pool trick is best-effort. If you need actual L7 load balancing, a service mesh is the right tool.",[172,2409,2410,2411,507,2413,507,2416,507,2419,2422],{},"No automatic replay of mutating operations. ",[31,2412,1819],{},[31,2414,2415],{},"PUT",[31,2417,2418],{},"PATCH",[31,2420,2421],{},"DELETE"," will not retry across pools. That is by design.",[15,2424,2425],{},"These are not bugs. They are scope decisions. Doing fewer things means each one is easier to reason about, and Microsoft already ships a better retry \u002F circuit-breaker library that this one composes with.",[10,2427,2429],{"id":2428},"when-this-is-the-right-fit","When this is the right fit",[15,2431,2432],{},"You probably want CloudHttp if:",[1788,2434,2435,2438,2441,2444],{},[172,2436,2437],{},"Your .NET service calls another service through a cluster DNS name.",[172,2439,2440],{},"You see one TCP connection from each caller pod sticking to the same upstream pod across long lifetimes.",[172,2442,2443],{},"You do not have a service mesh handling L7 load balancing for you.",[172,2445,2446,2447,2449],{},"You want a ",[31,2448,37],{}," profile sane for cluster traffic without writing it from scratch every time.",[15,2451,2452],{},"Skip it when:",[1788,2454,2455,2458,2461,2464],{},[172,2456,2457],{},"You only call public internet APIs. The cluster-traffic defaults are not the right shape for slow, distant, less reliable upstreams.",[172,2459,2460],{},"A single connection pool is enough for your throughput.",[172,2462,2463],{},"You already run a service mesh sidecar that handles L7 balancing.",[172,2465,2466],{},"Your upstream already does client-side balancing (e.g. gRPC name resolvers, AWS SDK-style retries with discovery).",[10,2468,2470],{"id":2469},"try-it","Try it",[15,2472,2473],{},"The repo includes a Docker Compose sample that runs several upstream containers behind one DNS name and prints which backend handled each request:",[53,2475,2479],{"className":2476,"code":2477,"language":2478,"meta":58,"style":58},"language-powershell shiki shiki-themes one-light one-dark-pro","$env:REQUESTS = \"48\"\n$env:CLIENT_COUNT = \"8\"\n$env:DISTRIBUTION_MODE = \"RoundRobin\"\n\ndocker compose `\n  --file .\\samples\\CloudHttp.Sample\\compose.yaml `\n  up --build --abort-on-container-exit --exit-code-from client `\n  --scale upstream=4 client\n","powershell",[31,2480,2481,2486,2491,2496,2500,2505,2510,2515],{"__ignoreMap":58},[62,2482,2483],{"class":64,"line":65},[62,2484,2485],{},"$env:REQUESTS = \"48\"\n",[62,2487,2488],{"class":64,"line":114},[62,2489,2490],{},"$env:CLIENT_COUNT = \"8\"\n",[62,2492,2493],{"class":64,"line":120},[62,2494,2495],{},"$env:DISTRIBUTION_MODE = \"RoundRobin\"\n",[62,2497,2498],{"class":64,"line":151},[62,2499,587],{"emptyLinePlaceholder":586},[62,2501,2502],{"class":64,"line":252},[62,2503,2504],{},"docker compose `\n",[62,2506,2507],{"class":64,"line":276},[62,2508,2509],{},"  --file .\\samples\\CloudHttp.Sample\\compose.yaml `\n",[62,2511,2512],{"class":64,"line":294},[62,2513,2514],{},"  up --build --abort-on-container-exit --exit-code-from client `\n",[62,2516,2517],{"class":64,"line":300},[62,2518,2519],{},"  --scale upstream=4 client\n",[15,2521,2522],{},"After the run, the client prints a summary like this:",[53,2524,2529],{"className":2525,"code":2527,"language":2528,"meta":58},[2526],"language-text","Summary\n-------\ncloudhttp-sample-upstream-1: 11 responses\ncloudhttp-sample-upstream-2: 14 responses\ncloudhttp-sample-upstream-3: 12 responses\ncloudhttp-sample-upstream-4: 11 responses\nFailures observed by client: 0\n","text",[31,2530,2527],{"__ignoreMap":58},[15,2532,2533,2534,2536],{},"The distribution will not be perfectly even, since Docker DNS, connection pooling, and timing all interfere, but you can see a single logical upstream getting reached through several independent client pools. There is also an unstable variant that makes upstreams return 503 every Nth request, which is the cleanest way to watch ",[31,2535,599],{}," rotation in action.",[15,2538,2539,2540,91],{},"The full walkthrough, including environment variables, the health-aware and weighted variants, and a Docker-less local script, is in ",[19,2541,2544],{"href":2542,"rel":2543},"https:\u002F\u002Fgithub.com\u002Fhaiilong\u002FCloudHttp\u002Fblob\u002Fmain\u002Fsamples\u002FCloudHttp.Sample\u002FREADME.md",[51],"samples\u002FCloudHttp.Sample\u002FREADME.md",[10,2546,2548],{"id":2547},"wrapping-the-http-series","Wrapping the HTTP series",[15,2550,2551],{},"This is the third post on this blog about HTTP in .NET, and probably the last for a while. The 2024 piece described the underlying load-balancing problem in Kubernetes. The HttpClient connection lifetime post was about understanding the machinery. CloudHttp turns the workaround into a library you can install.",[15,2553,2554],{},"The package itself is small: a handful of extension methods, one selector type, three distribution strategies, a curated set of cloud defaults, a few JSON helpers. The README and the three docs files in the repo go further than this post for anyone who wants more.",[15,2556,2557,2558],{},"Repo: ",[19,2559,49],{"href":49,"rel":2560},[51],[2562,2563,2564],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s7GmK, html code.shiki .s7GmK{--shiki-default:#383A42;--shiki-dark:#E5C07B}html pre.shiki code .s5ixo, html code.shiki .s5ixo{--shiki-default:#383A42;--shiki-dark:#ABB2BF}html pre.shiki code .sAdtL, html code.shiki .sAdtL{--shiki-default:#4078F2;--shiki-dark:#61AFEF}html pre.shiki code .sC09Y, html code.shiki .sC09Y{--shiki-default:#C18401;--shiki-dark:#E5C07B}html pre.shiki code .sknuh, html code.shiki .sknuh{--shiki-default:#383A42;--shiki-dark:#56B6C2}html pre.shiki code .sDhpE, html code.shiki .sDhpE{--shiki-default:#50A14F;--shiki-dark:#98C379}html pre.shiki code .sAGMh, html code.shiki .sAGMh{--shiki-default:#986801;--shiki-dark:#D19A66}html pre.shiki code .sLKXg, html code.shiki .sLKXg{--shiki-default:#A626A4;--shiki-dark:#C678DD}html pre.shiki code .sz0mV, html code.shiki .sz0mV{--shiki-default:#383A42;--shiki-dark:#E06C75}",{"title":58,"searchDepth":114,"depth":114,"links":2566},[2567,2568,2569,2570,2575,2576,2577,2578,2579,2580,2581,2582,2583],{"id":12,"depth":114,"text":13},{"id":70,"depth":114,"text":71},{"id":187,"depth":114,"text":188},{"id":716,"depth":114,"text":717,"children":2571},[2572,2573,2574],{"id":721,"depth":120,"text":722},{"id":805,"depth":120,"text":806},{"id":924,"depth":120,"text":925},{"id":1038,"depth":114,"text":1039},{"id":1051,"depth":114,"text":1052},{"id":1357,"depth":114,"text":1358},{"id":1779,"depth":114,"text":1780},{"id":2046,"depth":114,"text":2047},{"id":2382,"depth":114,"text":2383},{"id":2428,"depth":114,"text":2429},{"id":2469,"depth":114,"text":2470},{"id":2547,"depth":114,"text":2548},null,"2026-05-15","Cloud-friendly HttpClient extensions for .NET, independent connection pools, cloud-tuned defaults, and composition with Microsoft's resilience pipeline.","md",{},"\u002Fblog\u002Fwhy-i-built-cloudhttp",{"title":5,"description":2586},"blog\u002Fwhy-i-built-cloudhttp",[2593],"tech","G1SBLq5q3ZTrlDR70iEIfvtIMBS5Ap2M5aZ1wPtxdcU",1778998257277]