[{"data":1,"prerenderedAt":1964},["ShallowReactive",2],{"post-\u002Fblog\u002Fbuilding-ml-inference-part-1":3},{"id":4,"title":5,"body":6,"book":1953,"date":1954,"description":1955,"extension":1956,"meta":1957,"navigation":140,"path":1958,"seo":1959,"stem":1960,"tags":1961,"__hash__":1963},"blog\u002Fblog\u002Fbuilding-ml-inference-part-1.md","Building an ML Inference API, Part I",{"type":7,"value":8,"toc":1942},"minimark",[9,14,18,34,37,40,43,47,50,53,66,71,78,427,431,438,491,498,650,653,989,996,1000,1003,1006,1009,1012,1015,1018,1046,1049,1055,1058,1061,1352,1355,1358,1903,1912,1915,1918,1931,1935,1938],[10,11,13],"h2",{"id":12},"background","Background",[15,16,17],"p",{},"Back in early 2020, I was working on a Software Engineering (SWE) team whose stack was .NET (.NET Core 2.2, to be precise) and SQL Server, with all projects deployed to Windows Server VMs. I worked closely with a Data Science team that operated strictly in Python. Even though the two teams had independent workstreams, my responsibility often involved integrating Machine Learning (ML) models into production. The workflow typically looked like this:",[19,20,21,25,28,31],"ol",{},[22,23,24],"li",{},"A .NET project reads from a database or data source for input.",[22,26,27],{},"It formats the input and passes it to the Python-trained ML model.",[22,29,30],{},"The model runs a prediction on the input.",[22,32,33],{},"The result is passed back to the .NET project to be used downstream.",[15,35,36],{},"One of the main challenges was making this process seamless, fast, and scalable (up to a few hundred calls per second), all while ensuring Python-trained models worked effectively within a .NET environment.",[15,38,39],{},"This problem stuck with me. It remained relevant even in early 2023, when most projects in the company were fully Dockerized and managed by Kubernetes (k8s), no longer running on Windows Server VMs.",[15,41,42],{},"I want to document the problems and the solutions I devised, given the unique constraints I had to work with.",[10,44,46],{"id":45},"i-when-net-was-running-in-windows-vms","I. When .NET was running in Windows VMs",[15,48,49],{},"At the time, for \"security\" and \"we-don't-have-enough-non-C#-experts\" reasons, building Python projects on these VMs was not permitted.",[15,51,52],{},"The task was to run an XGBoost model based on specific conditions. The call frequency was low (a few calls every 15 minutes), so performance wasn't the top priority; getting it to run at all was. The simplest solution we came up with was to drop a Python script onto the same VM and invoke it as a subprocess from our .NET application.",[15,54,55,56,60,61,65],{},"While building Python ",[57,58,59],"em",{},"projects"," was strictly forbidden, apparently no one thought to ban installing the Python runtime and ",[62,63,64],"code",{},"pip"," packages. We took the win.",[67,68,70],"h3",{"id":69},"the-python-script","The Python script",[15,72,73,74,77],{},"Save this as ",[62,75,76],{},"predict.py",". It loads the model, accepts a JSON string as an argument, and prints the result to standard output.",[79,80,85],"pre",{"className":81,"code":82,"language":83,"meta":84,"style":84},"language-python shiki shiki-themes one-light one-dark-pro","import sys\nimport json\nimport xgboost as xgb\nimport numpy as np\n\n# 1. Load the model (done once when the script starts)\nmodel = xgb.XGBClassifier()\nmodel.load_model(\"xgboost_model.json\")\n\ndef run_inference(input_features):\n    # Convert input list to numpy array and reshape for single prediction\n    data = np.array(input_features).reshape(1, -1)\n    prediction = model.predict(data)\n    return float(prediction[0])\n\nif __name__ == \"__main__\":\n    try:\n        # 2. Read the argument passed from C#\n        input_json = sys.argv[1]\n        features = json.loads(input_json)\n\n        # 3. Run prediction and print result\n        result = run_inference(features)\n        print(result)\n    except Exception as e:\n        sys.stderr.write(str(e))\n","python","",[62,86,87,100,108,122,135,142,149,169,188,193,212,218,254,271,290,295,314,322,328,344,361,366,372,385,394,410],{"__ignoreMap":84},[88,89,92,96],"span",{"class":90,"line":91},"line",1,[88,93,95],{"class":94},"sLKXg","import",[88,97,99],{"class":98},"s5ixo"," sys\n",[88,101,103,105],{"class":90,"line":102},2,[88,104,95],{"class":94},[88,106,107],{"class":98}," json\n",[88,109,111,113,116,119],{"class":90,"line":110},3,[88,112,95],{"class":94},[88,114,115],{"class":98}," xgboost ",[88,117,118],{"class":94},"as",[88,120,121],{"class":98}," xgb\n",[88,123,125,127,130,132],{"class":90,"line":124},4,[88,126,95],{"class":94},[88,128,129],{"class":98}," numpy ",[88,131,118],{"class":94},[88,133,134],{"class":98}," np\n",[88,136,138],{"class":90,"line":137},5,[88,139,141],{"emptyLinePlaceholder":140},true,"\n",[88,143,145],{"class":90,"line":144},6,[88,146,148],{"class":147},"sW2Sy","# 1. Load the model (done once when the script starts)\n",[88,150,152,155,159,162,166],{"class":90,"line":151},7,[88,153,154],{"class":98},"model ",[88,156,158],{"class":157},"sknuh","=",[88,160,161],{"class":98}," xgb.",[88,163,165],{"class":164},"slOjB","XGBClassifier",[88,167,168],{"class":98},"()\n",[88,170,172,175,178,181,185],{"class":90,"line":171},8,[88,173,174],{"class":98},"model.",[88,176,177],{"class":164},"load_model",[88,179,180],{"class":98},"(",[88,182,184],{"class":183},"sDhpE","\"xgboost_model.json\"",[88,186,187],{"class":98},")\n",[88,189,191],{"class":90,"line":190},9,[88,192,141],{"emptyLinePlaceholder":140},[88,194,196,199,203,205,209],{"class":90,"line":195},10,[88,197,198],{"class":94},"def",[88,200,202],{"class":201},"sAdtL"," run_inference",[88,204,180],{"class":98},[88,206,208],{"class":207},"so_Uh","input_features",[88,210,211],{"class":98},"):\n",[88,213,215],{"class":90,"line":214},11,[88,216,217],{"class":147},"    # Convert input list to numpy array and reshape for single prediction\n",[88,219,221,224,226,229,232,235,238,240,244,247,250,252],{"class":90,"line":220},12,[88,222,223],{"class":98},"    data ",[88,225,158],{"class":157},[88,227,228],{"class":98}," np.",[88,230,231],{"class":164},"array",[88,233,234],{"class":98},"(input_features).",[88,236,237],{"class":164},"reshape",[88,239,180],{"class":98},[88,241,243],{"class":242},"sAGMh","1",[88,245,246],{"class":98},", ",[88,248,249],{"class":157},"-",[88,251,243],{"class":242},[88,253,187],{"class":98},[88,255,257,260,262,265,268],{"class":90,"line":256},13,[88,258,259],{"class":98},"    prediction ",[88,261,158],{"class":157},[88,263,264],{"class":98}," model.",[88,266,267],{"class":164},"predict",[88,269,270],{"class":98},"(data)\n",[88,272,274,277,281,284,287],{"class":90,"line":273},14,[88,275,276],{"class":94},"    return",[88,278,280],{"class":279},"s_Sar"," float",[88,282,283],{"class":98},"(prediction[",[88,285,286],{"class":242},"0",[88,288,289],{"class":98},"])\n",[88,291,293],{"class":90,"line":292},15,[88,294,141],{"emptyLinePlaceholder":140},[88,296,298,301,305,308,311],{"class":90,"line":297},16,[88,299,300],{"class":94},"if",[88,302,304],{"class":303},"sJa8x"," __name__",[88,306,307],{"class":157}," ==",[88,309,310],{"class":183}," \"__main__\"",[88,312,313],{"class":98},":\n",[88,315,317,320],{"class":90,"line":316},17,[88,318,319],{"class":94},"    try",[88,321,313],{"class":98},[88,323,325],{"class":90,"line":324},18,[88,326,327],{"class":147},"        # 2. Read the argument passed from C#\n",[88,329,331,334,336,339,341],{"class":90,"line":330},19,[88,332,333],{"class":98},"        input_json ",[88,335,158],{"class":157},[88,337,338],{"class":98}," sys.argv[",[88,340,243],{"class":242},[88,342,343],{"class":98},"]\n",[88,345,347,350,352,355,358],{"class":90,"line":346},20,[88,348,349],{"class":98},"        features ",[88,351,158],{"class":157},[88,353,354],{"class":98}," json.",[88,356,357],{"class":164},"loads",[88,359,360],{"class":98},"(input_json)\n",[88,362,364],{"class":90,"line":363},21,[88,365,141],{"emptyLinePlaceholder":140},[88,367,369],{"class":90,"line":368},22,[88,370,371],{"class":147},"        # 3. Run prediction and print result\n",[88,373,375,378,380,382],{"class":90,"line":374},23,[88,376,377],{"class":98},"        result ",[88,379,158],{"class":157},[88,381,202],{"class":164},[88,383,384],{"class":98},"(features)\n",[88,386,388,391],{"class":90,"line":387},24,[88,389,390],{"class":279},"        print",[88,392,393],{"class":98},"(result)\n",[88,395,397,400,404,407],{"class":90,"line":396},25,[88,398,399],{"class":94},"    except",[88,401,403],{"class":402},"st7oF"," Exception",[88,405,406],{"class":94}," as",[88,408,409],{"class":98}," e:\n",[88,411,413,416,419,421,424],{"class":90,"line":412},26,[88,414,415],{"class":98},"        sys.stderr.",[88,417,418],{"class":164},"write",[88,420,180],{"class":98},[88,422,423],{"class":279},"str",[88,425,426],{"class":98},"(e))\n",[67,428,430],{"id":429},"the-c-implementation","The C# implementation",[15,432,433,434,437],{},"The ",[62,435,436],{},"jsonString"," here is the feature array. Pull it from a DTO or wherever the input lives, then form the string.",[79,439,443],{"className":440,"code":441,"language":442,"meta":84,"style":84},"language-csharp shiki shiki-themes one-light one-dark-pro","var jsonString = \"[0.52,0.1,0.8,1.5,2.2,1.4]\";\nvar pythonPath = @\"python\";\nvar scriptPath = @\"predict.py\";\n","csharp",[62,444,445,463,477],{"__ignoreMap":84},[88,446,447,450,454,457,460],{"class":90,"line":91},[88,448,449],{"class":94},"var",[88,451,453],{"class":452},"sz0mV"," jsonString",[88,455,456],{"class":157}," =",[88,458,459],{"class":183}," \"[0.52,0.1,0.8,1.5,2.2,1.4]\"",[88,461,462],{"class":98},";\n",[88,464,465,467,470,472,475],{"class":90,"line":102},[88,466,449],{"class":94},[88,468,469],{"class":452}," pythonPath",[88,471,456],{"class":157},[88,473,474],{"class":183}," @\"python\"",[88,476,462],{"class":98},[88,478,479,481,484,486,489],{"class":90,"line":110},[88,480,449],{"class":94},[88,482,483],{"class":452}," scriptPath",[88,485,456],{"class":157},[88,487,488],{"class":183}," @\"predict.py\"",[88,490,462],{"class":98},[15,492,493,494,497],{},"Next, we configure ",[62,495,496],{},"ProcessStartInfo",". This is the critical part: we tell Windows to suppress the black command prompt window and redirect the output so we can read it in C#.",[79,499,501],{"className":440,"code":500,"language":442,"meta":84,"style":84},"var processStartInfo = new ProcessStartInfo\n{\n    FileName = pythonPath,\n    \u002F\u002F Wrap the JSON in quotes to handle spaces safely\n    Arguments = $\"{scriptPath} \\\"{jsonString}\\\"\",\n\n    \u002F\u002F Crucial settings:\n    RedirectStandardOutput = true, \u002F\u002F Capture the print() result\n    RedirectStandardError = true,  \u002F\u002F Capture errors\n    UseShellExecute = false,       \u002F\u002F Required to redirect streams\n    CreateNoWindow = true          \u002F\u002F Don't pop up a black CMD window\n};\n",[62,502,503,519,524,536,541,578,582,587,602,617,633,645],{"__ignoreMap":84},[88,504,505,507,510,512,515],{"class":90,"line":91},[88,506,449],{"class":94},[88,508,509],{"class":452}," processStartInfo",[88,511,456],{"class":157},[88,513,514],{"class":98}," new ",[88,516,518],{"class":517},"sC09Y","ProcessStartInfo\n",[88,520,521],{"class":90,"line":102},[88,522,523],{"class":98},"{\n",[88,525,526,529,531,533],{"class":90,"line":110},[88,527,528],{"class":452},"    FileName",[88,530,456],{"class":157},[88,532,469],{"class":452},[88,534,535],{"class":98},",\n",[88,537,538],{"class":90,"line":124},[88,539,540],{"class":147},"    \u002F\u002F Wrap the JSON in quotes to handle spaces safely\n",[88,542,543,546,548,551,555,558,561,564,566,568,570,573,576],{"class":90,"line":137},[88,544,545],{"class":452},"    Arguments",[88,547,456],{"class":157},[88,549,550],{"class":183}," $\"",[88,552,554],{"class":553},"sMj0N","{",[88,556,557],{"class":452},"scriptPath",[88,559,560],{"class":553},"}",[88,562,563],{"class":279}," \\\"",[88,565,554],{"class":553},[88,567,436],{"class":452},[88,569,560],{"class":553},[88,571,572],{"class":279},"\\\"",[88,574,575],{"class":183},"\"",[88,577,535],{"class":98},[88,579,580],{"class":90,"line":144},[88,581,141],{"emptyLinePlaceholder":140},[88,583,584],{"class":90,"line":151},[88,585,586],{"class":147},"    \u002F\u002F Crucial settings:\n",[88,588,589,592,594,597,599],{"class":90,"line":171},[88,590,591],{"class":452},"    RedirectStandardOutput",[88,593,456],{"class":157},[88,595,596],{"class":242}," true",[88,598,246],{"class":98},[88,600,601],{"class":147},"\u002F\u002F Capture the print() result\n",[88,603,604,607,609,611,614],{"class":90,"line":190},[88,605,606],{"class":452},"    RedirectStandardError",[88,608,456],{"class":157},[88,610,596],{"class":242},[88,612,613],{"class":98},",  ",[88,615,616],{"class":147},"\u002F\u002F Capture errors\n",[88,618,619,622,624,627,630],{"class":90,"line":195},[88,620,621],{"class":452},"    UseShellExecute",[88,623,456],{"class":157},[88,625,626],{"class":242}," false",[88,628,629],{"class":98},",       ",[88,631,632],{"class":147},"\u002F\u002F Required to redirect streams\n",[88,634,635,638,640,642],{"class":90,"line":214},[88,636,637],{"class":452},"    CreateNoWindow",[88,639,456],{"class":157},[88,641,596],{"class":242},[88,643,644],{"class":147},"          \u002F\u002F Don't pop up a black CMD window\n",[88,646,647],{"class":90,"line":220},[88,648,649],{"class":98},"};\n",[15,651,652],{},"Finally, we execute the process. This is a synchronous operation, meaning the C# app will wait for Python to finish before continuing.",[79,654,656],{"className":440,"code":655,"language":442,"meta":84,"style":84},"Process process = null;\n\ntry\n{\n    process = new Process { StartInfo = processStartInfo };\n    process.Start();\n\n    \u002F\u002F Read the output synchronously (wait for Python to finish)\n    var stringResult = process.StandardOutput.ReadToEnd();\n    var errors = process.StandardError.ReadToEnd();\n\n    process.WaitForExit();\n\n    if (process.ExitCode == 0)\n    {\n        var prediction = double.Parse(stringResult);\n        \u002F\u002F Use prediction...\n    }\n    else\n    {\n        _logger.LogError($\"Python script failed: {errors}\");\n    }\n}\ncatch (Exception ex)\n{\n    _logger.LogError($\"C# execution failed: {ex.Message}\");\n}\nfinally\n{\n    process?.Dispose();\n}\n",[62,657,658,673,677,682,686,710,724,728,733,757,779,783,794,798,821,826,852,857,862,867,871,897,901,906,921,925,955,960,966,971,984],{"__ignoreMap":84},[88,659,660,663,666,668,671],{"class":90,"line":91},[88,661,662],{"class":517},"Process",[88,664,665],{"class":452}," process",[88,667,456],{"class":157},[88,669,670],{"class":242}," null",[88,672,462],{"class":98},[88,674,675],{"class":90,"line":102},[88,676,141],{"emptyLinePlaceholder":140},[88,678,679],{"class":90,"line":110},[88,680,681],{"class":94},"try\n",[88,683,684],{"class":90,"line":124},[88,685,523],{"class":98},[88,687,688,691,693,695,697,700,703,705,707],{"class":90,"line":137},[88,689,690],{"class":452},"    process",[88,692,456],{"class":157},[88,694,514],{"class":98},[88,696,662],{"class":517},[88,698,699],{"class":98}," { ",[88,701,702],{"class":452},"StartInfo",[88,704,456],{"class":157},[88,706,509],{"class":452},[88,708,709],{"class":98}," };\n",[88,711,712,715,718,721],{"class":90,"line":144},[88,713,690],{"class":714},"s7GmK",[88,716,717],{"class":98},".",[88,719,720],{"class":201},"Start",[88,722,723],{"class":98},"();\n",[88,725,726],{"class":90,"line":151},[88,727,141],{"emptyLinePlaceholder":140},[88,729,730],{"class":90,"line":171},[88,731,732],{"class":147},"    \u002F\u002F Read the output synchronously (wait for Python to finish)\n",[88,734,735,738,741,743,745,747,750,752,755],{"class":90,"line":190},[88,736,737],{"class":94},"    var",[88,739,740],{"class":452}," stringResult",[88,742,456],{"class":157},[88,744,665],{"class":714},[88,746,717],{"class":98},[88,748,749],{"class":714},"StandardOutput",[88,751,717],{"class":98},[88,753,754],{"class":201},"ReadToEnd",[88,756,723],{"class":98},[88,758,759,761,764,766,768,770,773,775,777],{"class":90,"line":195},[88,760,737],{"class":94},[88,762,763],{"class":452}," errors",[88,765,456],{"class":157},[88,767,665],{"class":714},[88,769,717],{"class":98},[88,771,772],{"class":714},"StandardError",[88,774,717],{"class":98},[88,776,754],{"class":201},[88,778,723],{"class":98},[88,780,781],{"class":90,"line":214},[88,782,141],{"emptyLinePlaceholder":140},[88,784,785,787,789,792],{"class":90,"line":220},[88,786,690],{"class":714},[88,788,717],{"class":98},[88,790,791],{"class":201},"WaitForExit",[88,793,723],{"class":98},[88,795,796],{"class":90,"line":256},[88,797,141],{"emptyLinePlaceholder":140},[88,799,800,803,806,809,811,814,816,819],{"class":90,"line":273},[88,801,802],{"class":94},"    if",[88,804,805],{"class":98}," (",[88,807,808],{"class":714},"process",[88,810,717],{"class":98},[88,812,813],{"class":714},"ExitCode",[88,815,307],{"class":157},[88,817,818],{"class":242}," 0",[88,820,187],{"class":98},[88,822,823],{"class":90,"line":292},[88,824,825],{"class":98},"    {\n",[88,827,828,831,834,836,839,841,844,846,849],{"class":90,"line":297},[88,829,830],{"class":94},"        var",[88,832,833],{"class":452}," prediction",[88,835,456],{"class":157},[88,837,838],{"class":94}," double",[88,840,717],{"class":98},[88,842,843],{"class":201},"Parse",[88,845,180],{"class":98},[88,847,848],{"class":452},"stringResult",[88,850,851],{"class":98},");\n",[88,853,854],{"class":90,"line":316},[88,855,856],{"class":147},"        \u002F\u002F Use prediction...\n",[88,858,859],{"class":90,"line":324},[88,860,861],{"class":98},"    }\n",[88,863,864],{"class":90,"line":330},[88,865,866],{"class":94},"    else\n",[88,868,869],{"class":90,"line":346},[88,870,825],{"class":98},[88,872,873,876,878,881,883,886,888,891,893,895],{"class":90,"line":363},[88,874,875],{"class":714},"        _logger",[88,877,717],{"class":98},[88,879,880],{"class":201},"LogError",[88,882,180],{"class":98},[88,884,885],{"class":183},"$\"Python script failed: ",[88,887,554],{"class":553},[88,889,890],{"class":452},"errors",[88,892,560],{"class":553},[88,894,575],{"class":183},[88,896,851],{"class":98},[88,898,899],{"class":90,"line":368},[88,900,861],{"class":98},[88,902,903],{"class":90,"line":374},[88,904,905],{"class":98},"}\n",[88,907,908,911,913,916,919],{"class":90,"line":387},[88,909,910],{"class":94},"catch",[88,912,805],{"class":98},[88,914,915],{"class":517},"Exception",[88,917,918],{"class":452}," ex",[88,920,187],{"class":98},[88,922,923],{"class":90,"line":396},[88,924,523],{"class":98},[88,926,927,930,932,934,936,939,941,944,946,949,951,953],{"class":90,"line":412},[88,928,929],{"class":714},"    _logger",[88,931,717],{"class":98},[88,933,880],{"class":201},[88,935,180],{"class":98},[88,937,938],{"class":183},"$\"C# execution failed: ",[88,940,554],{"class":553},[88,942,943],{"class":714},"ex",[88,945,717],{"class":553},[88,947,948],{"class":714},"Message",[88,950,560],{"class":553},[88,952,575],{"class":183},[88,954,851],{"class":98},[88,956,958],{"class":90,"line":957},27,[88,959,905],{"class":98},[88,961,963],{"class":90,"line":962},28,[88,964,965],{"class":94},"finally\n",[88,967,969],{"class":90,"line":968},29,[88,970,523],{"class":98},[88,972,974,976,979,982],{"class":90,"line":973},30,[88,975,690],{"class":714},[88,977,978],{"class":98},"?.",[88,980,981],{"class":201},"Dispose",[88,983,723],{"class":98},[88,985,987],{"class":90,"line":986},31,[88,988,905],{"class":98},[15,990,991,992,995],{},"For our use case, this hacked-together solution worked surprisingly well. A nice bonus: updating the model was as simple as deploying a new ",[62,993,994],{},"xgboost_model.json"," to all the servers (i.e., remoting in and swapping the file).",[10,997,999],{"id":998},"ii-when-net-was-running-in-windows-vms-but-now-we-had-to-care-about-scale","II. When .NET was running in Windows VMs (but now we had to care about scale)",[15,1001,1002],{},"Within half a year, a similar task came up, but this time the model would be called very frequently and asynchronously. We're talking an average of 400,000 times per day, with distinct peak and off-peak periods.",[15,1004,1005],{},"The task was to integrate this model into a .NET Web API deployed across 10 VMs. The model would be invoked in one of the API's endpoints.",[15,1007,1008],{},"The previous approach doesn't scale at all. We were spinning up a new OS process for every single prediction. For 400k requests a day, that means the server has to start the Python runtime, load the libraries, load the model into memory, run the prediction, and tear everything down, 400,000 times. The overhead and CPU jitter alone would be crippling.",[15,1010,1011],{},"The ideal solution would have been to deploy a Python Web API to those VMs and let it serve as a dedicated inference service, keeping the model loaded in memory and handling requests over HTTP. But given the earlier restrictions, that wasn't on the table.",[15,1013,1014],{},"So the question became: can we run XGBoost directly in C#? And, naturally, has anyone actually done it?",[15,1016,1017],{},"I found two options:",[19,1019,1020,1038],{},[22,1021,1022,1029,1030,1033,1034,1037],{},[1023,1024,1028],"a",{"href":1025,"rel":1026},"https:\u002F\u002Fgithub.com\u002FPicNet\u002FXGBoost.Net",[1027],"nofollow","PicNet\u002FXGBoost.Net",", a community library built on top of the native ",[62,1031,1032],{},"xgboost.dll"," (Windows) or ",[62,1035,1036],{},"libxgboost.so"," (Linux).",[22,1039,1040,1045],{},[1023,1041,1044],{"href":1042,"rel":1043},"https:\u002F\u002Fgithub.com\u002Fdotnet\u002Fmachinelearning",[1027],"ML.NET",", Microsoft's own ML framework, where the approach is to convert the XGBoost model to ONNX (Open Neural Network Exchange) and run inference through ML.NET.",[15,1047,1048],{},"Option 2 felt like overkill for a pure inference problem. ONNX interoperability is a powerful concept, but it's bringing a cannon to a knife fight.",[15,1050,1051,1052,1054],{},"Option 1, on the other hand, was direct: a thin wrapper around the native C++ XGBoost library. It just worked. I did run into issues ensuring the native binaries (",[62,1053,1032],{},") and model files were copied to the right output directories and referenced correctly, but that was more a setup fumble on my part than a problem with the library itself.",[67,1056,430],{"id":1057},"the-c-implementation-1",[15,1059,1060],{},"Using the PicNet library, the code looks surprisingly close to the Python version.",[79,1062,1064],{"className":440,"code":1063,"language":442,"meta":84,"style":84},"using XGBoost.Lib;\n\n\u002F\u002F 1. Load the model\n\u002F\u002F You'll likely need to resolve the path via the executing or calling assembly. Fun times.\nvar modelPath = @\"xgboost_model.json\";\nusing var booster = XGBoost.Booster.BoosterLoad(modelPath);\n\n\u002F\u002F 2. Prepare the data\n\u002F\u002F Pass in an array of floats, which gets converted to a DMatrix internally\nvar features = new float[] { 0.52f, 0.1f, 0.8f, 1.5f, 2.2f, 1.4f };\nvar numRows = 1;\nvar numCols = features.Length;\nusing var matrix = XGBoost.DMatrix.FromMat(features, numRows, numCols, 0.0f);\n\n\u002F\u002F 3. Run inference\n\u002F\u002F Predict returns a 2D array [rows, output_classes]\nvar prediction = booster.Predict(matrix);\n\n\u002F\u002F 4. Parse the result\n\u002F\u002F For simple classification, grab the first value of the first row\nvar result = prediction[0][0];\n",[62,1065,1066,1081,1085,1090,1095,1109,1140,1144,1149,1154,1201,1215,1233,1278,1282,1287,1292,1314,1318,1323,1328],{"__ignoreMap":84},[88,1067,1068,1071,1074,1076,1079],{"class":90,"line":91},[88,1069,1070],{"class":94},"using",[88,1072,1073],{"class":517}," XGBoost",[88,1075,717],{"class":98},[88,1077,1078],{"class":517},"Lib",[88,1080,462],{"class":98},[88,1082,1083],{"class":90,"line":102},[88,1084,141],{"emptyLinePlaceholder":140},[88,1086,1087],{"class":90,"line":110},[88,1088,1089],{"class":147},"\u002F\u002F 1. Load the model\n",[88,1091,1092],{"class":90,"line":124},[88,1093,1094],{"class":147},"\u002F\u002F You'll likely need to resolve the path via the executing or calling assembly. Fun times.\n",[88,1096,1097,1099,1102,1104,1107],{"class":90,"line":137},[88,1098,449],{"class":94},[88,1100,1101],{"class":452}," modelPath",[88,1103,456],{"class":157},[88,1105,1106],{"class":183}," @\"xgboost_model.json\"",[88,1108,462],{"class":98},[88,1110,1111,1113,1116,1119,1121,1123,1125,1128,1130,1133,1135,1138],{"class":90,"line":144},[88,1112,1070],{"class":94},[88,1114,1115],{"class":94}," var",[88,1117,1118],{"class":452}," booster",[88,1120,456],{"class":157},[88,1122,1073],{"class":714},[88,1124,717],{"class":98},[88,1126,1127],{"class":714},"Booster",[88,1129,717],{"class":98},[88,1131,1132],{"class":201},"BoosterLoad",[88,1134,180],{"class":98},[88,1136,1137],{"class":452},"modelPath",[88,1139,851],{"class":98},[88,1141,1142],{"class":90,"line":151},[88,1143,141],{"emptyLinePlaceholder":140},[88,1145,1146],{"class":90,"line":171},[88,1147,1148],{"class":147},"\u002F\u002F 2. Prepare the data\n",[88,1150,1151],{"class":90,"line":190},[88,1152,1153],{"class":147},"\u002F\u002F Pass in an array of floats, which gets converted to a DMatrix internally\n",[88,1155,1156,1158,1161,1163,1165,1168,1171,1174,1176,1179,1181,1184,1186,1189,1191,1194,1196,1199],{"class":90,"line":195},[88,1157,449],{"class":94},[88,1159,1160],{"class":452}," features",[88,1162,456],{"class":157},[88,1164,514],{"class":98},[88,1166,1167],{"class":94},"float",[88,1169,1170],{"class":98},"[] { ",[88,1172,1173],{"class":242},"0.52f",[88,1175,246],{"class":98},[88,1177,1178],{"class":242},"0.1f",[88,1180,246],{"class":98},[88,1182,1183],{"class":242},"0.8f",[88,1185,246],{"class":98},[88,1187,1188],{"class":242},"1.5f",[88,1190,246],{"class":98},[88,1192,1193],{"class":242},"2.2f",[88,1195,246],{"class":98},[88,1197,1198],{"class":242},"1.4f",[88,1200,709],{"class":98},[88,1202,1203,1205,1208,1210,1213],{"class":90,"line":214},[88,1204,449],{"class":94},[88,1206,1207],{"class":452}," numRows",[88,1209,456],{"class":157},[88,1211,1212],{"class":242}," 1",[88,1214,462],{"class":98},[88,1216,1217,1219,1222,1224,1226,1228,1231],{"class":90,"line":220},[88,1218,449],{"class":94},[88,1220,1221],{"class":452}," numCols",[88,1223,456],{"class":157},[88,1225,1160],{"class":714},[88,1227,717],{"class":98},[88,1229,1230],{"class":714},"Length",[88,1232,462],{"class":98},[88,1234,1235,1237,1239,1242,1244,1246,1248,1251,1253,1256,1258,1261,1263,1266,1268,1271,1273,1276],{"class":90,"line":256},[88,1236,1070],{"class":94},[88,1238,1115],{"class":94},[88,1240,1241],{"class":452}," matrix",[88,1243,456],{"class":157},[88,1245,1073],{"class":714},[88,1247,717],{"class":98},[88,1249,1250],{"class":714},"DMatrix",[88,1252,717],{"class":98},[88,1254,1255],{"class":201},"FromMat",[88,1257,180],{"class":98},[88,1259,1260],{"class":452},"features",[88,1262,246],{"class":98},[88,1264,1265],{"class":452},"numRows",[88,1267,246],{"class":98},[88,1269,1270],{"class":452},"numCols",[88,1272,246],{"class":98},[88,1274,1275],{"class":242},"0.0f",[88,1277,851],{"class":98},[88,1279,1280],{"class":90,"line":273},[88,1281,141],{"emptyLinePlaceholder":140},[88,1283,1284],{"class":90,"line":292},[88,1285,1286],{"class":147},"\u002F\u002F 3. Run inference\n",[88,1288,1289],{"class":90,"line":297},[88,1290,1291],{"class":147},"\u002F\u002F Predict returns a 2D array [rows, output_classes]\n",[88,1293,1294,1296,1298,1300,1302,1304,1307,1309,1312],{"class":90,"line":316},[88,1295,449],{"class":94},[88,1297,833],{"class":452},[88,1299,456],{"class":157},[88,1301,1118],{"class":714},[88,1303,717],{"class":98},[88,1305,1306],{"class":201},"Predict",[88,1308,180],{"class":98},[88,1310,1311],{"class":452},"matrix",[88,1313,851],{"class":98},[88,1315,1316],{"class":90,"line":324},[88,1317,141],{"emptyLinePlaceholder":140},[88,1319,1320],{"class":90,"line":330},[88,1321,1322],{"class":147},"\u002F\u002F 4. Parse the result\n",[88,1324,1325],{"class":90,"line":346},[88,1326,1327],{"class":147},"\u002F\u002F For simple classification, grab the first value of the first row\n",[88,1329,1330,1332,1335,1337,1339,1342,1344,1347,1349],{"class":90,"line":363},[88,1331,449],{"class":94},[88,1333,1334],{"class":452}," result",[88,1336,456],{"class":157},[88,1338,833],{"class":714},[88,1340,1341],{"class":98},"[",[88,1343,286],{"class":242},[88,1345,1346],{"class":98},"][",[88,1348,286],{"class":242},[88,1350,1351],{"class":98},"];\n",[15,1353,1354],{},"This eliminated all the process-spawning overhead. The model is loaded once at startup, and predictions happen entirely in memory. It handled 400k requests per day across 10 VMs without breaking a sweat, and arguably faster than a Python Web API would have been, since there's no network hop and no need to think about how to scale the Python side.",[15,1356,1357],{},"There was one significant caveat, though. Because the model is loaded once at startup, updating it requires restarting the application. Since we wanted to avoid forcing a restart every time the data science team retrained the model, we added two admin endpoints: one to upload a new model file, and one to trigger an in-memory reload.",[79,1359,1361],{"className":440,"code":1360,"language":442,"meta":84,"style":84},"[ApiController]\n[Route(\"api\u002F[controller]\")]\npublic class ModelAdminController : ControllerBase\n{\n    private static XGBoost.Booster _currentBooster;\n    private static readonly object _lock = new object();\n\n    \u002F\u002F Endpoint 1: Upload the new model file to the server\n    [HttpPost(\"upload\")]\n    public IActionResult UploadModel(IFormFile file)\n    {\n        if (file == null || file.Length == 0)\n            return BadRequest(\"File is empty\");\n\n        var filePath = Path.Combine(Directory.GetCurrentDirectory(), \"xgboost_model.json\");\n\n        using (var stream = new FileStream(filePath, FileMode.Create))\n        {\n            file.CopyTo(stream);\n        }\n\n        return Ok(\"Model file uploaded successfully. Call \u002Freload to apply.\");\n    }\n\n    \u002F\u002F Endpoint 2: Reload the model into memory\n    [HttpPost(\"reload\")]\n    public IActionResult ReloadModel()\n    {\n        try\n        {\n            lock (_lock)\n            {\n                _currentBooster?.Dispose();\n\n                var modelPath = Path.Combine(Directory.GetCurrentDirectory(), \"xgboost_model.json\");\n                _currentBooster = XGBoost.Booster.BoosterLoad(modelPath);\n            }\n\n            return Ok(\"Model reloaded successfully.\");\n        }\n        catch (Exception ex)\n        {\n            return StatusCode(500, $\"Failed to reload model: {ex.Message}\");\n        }\n    }\n}\n",[62,1362,1363,1372,1387,1404,1408,1427,1451,1455,1460,1475,1496,1500,1529,1544,1548,1582,1586,1623,1628,1645,1650,1654,1669,1673,1677,1682,1695,1706,1710,1715,1719,1731,1737,1749,1754,1784,1807,1813,1818,1832,1837,1851,1856,1888,1893,1898],{"__ignoreMap":84},[88,1364,1365,1367,1370],{"class":90,"line":91},[88,1366,1341],{"class":98},[88,1368,1369],{"class":517},"ApiController",[88,1371,343],{"class":98},[88,1373,1374,1376,1379,1381,1384],{"class":90,"line":102},[88,1375,1341],{"class":98},[88,1377,1378],{"class":517},"Route",[88,1380,180],{"class":98},[88,1382,1383],{"class":183},"\"api\u002F[controller]\"",[88,1385,1386],{"class":98},")]\n",[88,1388,1389,1392,1395,1398,1401],{"class":90,"line":110},[88,1390,1391],{"class":94},"public",[88,1393,1394],{"class":94}," class",[88,1396,1397],{"class":517}," ModelAdminController",[88,1399,1400],{"class":98}," : ",[88,1402,1403],{"class":517},"ControllerBase\n",[88,1405,1406],{"class":90,"line":124},[88,1407,523],{"class":98},[88,1409,1410,1413,1416,1418,1420,1422,1425],{"class":90,"line":137},[88,1411,1412],{"class":94},"    private",[88,1414,1415],{"class":94}," static",[88,1417,1073],{"class":517},[88,1419,717],{"class":98},[88,1421,1127],{"class":517},[88,1423,1424],{"class":303}," _currentBooster",[88,1426,462],{"class":98},[88,1428,1429,1431,1433,1436,1439,1442,1444,1446,1449],{"class":90,"line":144},[88,1430,1412],{"class":94},[88,1432,1415],{"class":94},[88,1434,1435],{"class":94}," readonly",[88,1437,1438],{"class":94}," object",[88,1440,1441],{"class":303}," _lock",[88,1443,456],{"class":157},[88,1445,514],{"class":98},[88,1447,1448],{"class":94},"object",[88,1450,723],{"class":98},[88,1452,1453],{"class":90,"line":151},[88,1454,141],{"emptyLinePlaceholder":140},[88,1456,1457],{"class":90,"line":171},[88,1458,1459],{"class":147},"    \u002F\u002F Endpoint 1: Upload the new model file to the server\n",[88,1461,1462,1465,1468,1470,1473],{"class":90,"line":190},[88,1463,1464],{"class":98},"    [",[88,1466,1467],{"class":517},"HttpPost",[88,1469,180],{"class":98},[88,1471,1472],{"class":183},"\"upload\"",[88,1474,1386],{"class":98},[88,1476,1477,1480,1483,1486,1488,1491,1494],{"class":90,"line":195},[88,1478,1479],{"class":94},"    public",[88,1481,1482],{"class":517}," IActionResult",[88,1484,1485],{"class":201}," UploadModel",[88,1487,180],{"class":98},[88,1489,1490],{"class":517},"IFormFile",[88,1492,1493],{"class":714}," file",[88,1495,187],{"class":98},[88,1497,1498],{"class":90,"line":214},[88,1499,825],{"class":98},[88,1501,1502,1505,1507,1510,1512,1514,1517,1519,1521,1523,1525,1527],{"class":90,"line":220},[88,1503,1504],{"class":94},"        if",[88,1506,805],{"class":98},[88,1508,1509],{"class":452},"file",[88,1511,307],{"class":157},[88,1513,670],{"class":242},[88,1515,1516],{"class":157}," ||",[88,1518,1493],{"class":714},[88,1520,717],{"class":98},[88,1522,1230],{"class":714},[88,1524,307],{"class":157},[88,1526,818],{"class":242},[88,1528,187],{"class":98},[88,1530,1531,1534,1537,1539,1542],{"class":90,"line":256},[88,1532,1533],{"class":94},"            return",[88,1535,1536],{"class":201}," BadRequest",[88,1538,180],{"class":98},[88,1540,1541],{"class":183},"\"File is empty\"",[88,1543,851],{"class":98},[88,1545,1546],{"class":90,"line":273},[88,1547,141],{"emptyLinePlaceholder":140},[88,1549,1550,1552,1555,1557,1560,1562,1565,1567,1570,1572,1575,1578,1580],{"class":90,"line":292},[88,1551,830],{"class":94},[88,1553,1554],{"class":452}," filePath",[88,1556,456],{"class":157},[88,1558,1559],{"class":714}," Path",[88,1561,717],{"class":98},[88,1563,1564],{"class":201},"Combine",[88,1566,180],{"class":98},[88,1568,1569],{"class":714},"Directory",[88,1571,717],{"class":98},[88,1573,1574],{"class":201},"GetCurrentDirectory",[88,1576,1577],{"class":98},"(), ",[88,1579,184],{"class":183},[88,1581,851],{"class":98},[88,1583,1584],{"class":90,"line":297},[88,1585,141],{"emptyLinePlaceholder":140},[88,1587,1588,1591,1593,1595,1598,1600,1602,1605,1607,1610,1612,1615,1617,1620],{"class":90,"line":316},[88,1589,1590],{"class":94},"        using",[88,1592,805],{"class":98},[88,1594,449],{"class":94},[88,1596,1597],{"class":452}," stream",[88,1599,456],{"class":157},[88,1601,514],{"class":98},[88,1603,1604],{"class":517},"FileStream",[88,1606,180],{"class":98},[88,1608,1609],{"class":452},"filePath",[88,1611,246],{"class":98},[88,1613,1614],{"class":714},"FileMode",[88,1616,717],{"class":98},[88,1618,1619],{"class":714},"Create",[88,1621,1622],{"class":98},"))\n",[88,1624,1625],{"class":90,"line":324},[88,1626,1627],{"class":98},"        {\n",[88,1629,1630,1633,1635,1638,1640,1643],{"class":90,"line":330},[88,1631,1632],{"class":714},"            file",[88,1634,717],{"class":98},[88,1636,1637],{"class":201},"CopyTo",[88,1639,180],{"class":98},[88,1641,1642],{"class":452},"stream",[88,1644,851],{"class":98},[88,1646,1647],{"class":90,"line":346},[88,1648,1649],{"class":98},"        }\n",[88,1651,1652],{"class":90,"line":363},[88,1653,141],{"emptyLinePlaceholder":140},[88,1655,1656,1659,1662,1664,1667],{"class":90,"line":368},[88,1657,1658],{"class":94},"        return",[88,1660,1661],{"class":201}," Ok",[88,1663,180],{"class":98},[88,1665,1666],{"class":183},"\"Model file uploaded successfully. Call \u002Freload to apply.\"",[88,1668,851],{"class":98},[88,1670,1671],{"class":90,"line":374},[88,1672,861],{"class":98},[88,1674,1675],{"class":90,"line":387},[88,1676,141],{"emptyLinePlaceholder":140},[88,1678,1679],{"class":90,"line":396},[88,1680,1681],{"class":147},"    \u002F\u002F Endpoint 2: Reload the model into memory\n",[88,1683,1684,1686,1688,1690,1693],{"class":90,"line":412},[88,1685,1464],{"class":98},[88,1687,1467],{"class":517},[88,1689,180],{"class":98},[88,1691,1692],{"class":183},"\"reload\"",[88,1694,1386],{"class":98},[88,1696,1697,1699,1701,1704],{"class":90,"line":957},[88,1698,1479],{"class":94},[88,1700,1482],{"class":517},[88,1702,1703],{"class":201}," ReloadModel",[88,1705,168],{"class":98},[88,1707,1708],{"class":90,"line":962},[88,1709,825],{"class":98},[88,1711,1712],{"class":90,"line":968},[88,1713,1714],{"class":94},"        try\n",[88,1716,1717],{"class":90,"line":973},[88,1718,1627],{"class":98},[88,1720,1721,1724,1726,1729],{"class":90,"line":986},[88,1722,1723],{"class":94},"            lock",[88,1725,805],{"class":98},[88,1727,1728],{"class":452},"_lock",[88,1730,187],{"class":98},[88,1732,1734],{"class":90,"line":1733},32,[88,1735,1736],{"class":98},"            {\n",[88,1738,1740,1743,1745,1747],{"class":90,"line":1739},33,[88,1741,1742],{"class":714},"                _currentBooster",[88,1744,978],{"class":98},[88,1746,981],{"class":201},[88,1748,723],{"class":98},[88,1750,1752],{"class":90,"line":1751},34,[88,1753,141],{"emptyLinePlaceholder":140},[88,1755,1757,1760,1762,1764,1766,1768,1770,1772,1774,1776,1778,1780,1782],{"class":90,"line":1756},35,[88,1758,1759],{"class":94},"                var",[88,1761,1101],{"class":452},[88,1763,456],{"class":157},[88,1765,1559],{"class":714},[88,1767,717],{"class":98},[88,1769,1564],{"class":201},[88,1771,180],{"class":98},[88,1773,1569],{"class":714},[88,1775,717],{"class":98},[88,1777,1574],{"class":201},[88,1779,1577],{"class":98},[88,1781,184],{"class":183},[88,1783,851],{"class":98},[88,1785,1787,1789,1791,1793,1795,1797,1799,1801,1803,1805],{"class":90,"line":1786},36,[88,1788,1742],{"class":452},[88,1790,456],{"class":157},[88,1792,1073],{"class":714},[88,1794,717],{"class":98},[88,1796,1127],{"class":714},[88,1798,717],{"class":98},[88,1800,1132],{"class":201},[88,1802,180],{"class":98},[88,1804,1137],{"class":452},[88,1806,851],{"class":98},[88,1808,1810],{"class":90,"line":1809},37,[88,1811,1812],{"class":98},"            }\n",[88,1814,1816],{"class":90,"line":1815},38,[88,1817,141],{"emptyLinePlaceholder":140},[88,1819,1821,1823,1825,1827,1830],{"class":90,"line":1820},39,[88,1822,1533],{"class":94},[88,1824,1661],{"class":201},[88,1826,180],{"class":98},[88,1828,1829],{"class":183},"\"Model reloaded successfully.\"",[88,1831,851],{"class":98},[88,1833,1835],{"class":90,"line":1834},40,[88,1836,1649],{"class":98},[88,1838,1840,1843,1845,1847,1849],{"class":90,"line":1839},41,[88,1841,1842],{"class":94},"        catch",[88,1844,805],{"class":98},[88,1846,915],{"class":517},[88,1848,918],{"class":452},[88,1850,187],{"class":98},[88,1852,1854],{"class":90,"line":1853},42,[88,1855,1627],{"class":98},[88,1857,1859,1861,1864,1866,1869,1871,1874,1876,1878,1880,1882,1884,1886],{"class":90,"line":1858},43,[88,1860,1533],{"class":94},[88,1862,1863],{"class":201}," StatusCode",[88,1865,180],{"class":98},[88,1867,1868],{"class":242},"500",[88,1870,246],{"class":98},[88,1872,1873],{"class":183},"$\"Failed to reload model: ",[88,1875,554],{"class":553},[88,1877,943],{"class":714},[88,1879,717],{"class":553},[88,1881,948],{"class":714},[88,1883,560],{"class":553},[88,1885,575],{"class":183},[88,1887,851],{"class":98},[88,1889,1891],{"class":90,"line":1890},44,[88,1892,1649],{"class":98},[88,1894,1896],{"class":90,"line":1895},45,[88,1897,861],{"class":98},[88,1899,1901],{"class":90,"line":1900},46,[88,1902,905],{"class":98},[15,1904,1905,1906,1911],{},"At the time of writing, there is a newer and better-maintained XGBoost library for C# available at ",[1023,1907,1910],{"href":1908,"rel":1909},"https:\u002F\u002Fgithub.com\u002Fmdabros\u002FXGBoostSharp",[1027],"mdabros\u002FXGBoostSharp",", built on the same foundations. If you need a native implementation today, use that one. For inference-only use cases, you can ignore the training capabilities entirely.",[15,1913,1914],{},"Then, ...",[15,1916,1917],{},"One day, the team switched from XGBoost to LightGBM. Both are gradient boosting frameworks, but while XGBoost is known for its execution speed and model performance, LightGBM is often praised for being faster still and using less memory, which is a meaningful advantage when working with large datasets.",[15,1919,1920,1921,1926,1927,1930],{},"There is a LightGBM wrapper for .NET called ",[1023,1922,1925],{"href":1923,"rel":1924},"https:\u002F\u002Fgithub.com\u002Frca22\u002FLightGBM.Net",[1027],"LightGBM.Net",", which saved us here (Thank God!). HOWEVER, I couldn't get it to locate ",[62,1928,1929],{},"lib_lightgbm.dll"," correctly no matter where I placed the file on the server. In the end, I forked the repository and extended it to accept an absolute path for the DLL.",[10,1932,1934],{"id":1933},"closing-thoughts","Closing thoughts",[15,1936,1937],{},"Path issues and dependency management in Windows VM environments became a recurring theme. Running native libraries inside .NET solved the performance problem, but it introduced a new set of maintenance headaches. What if the data science team wanted to integrate a CatBoost model next? We needed a better way to abstract this complexity away. I'll cover that in Part II, which is, if anything, more relevant in 2025.",[1939,1940,1941],"style",{},"html pre.shiki code .sLKXg, html code.shiki .sLKXg{--shiki-default:#A626A4;--shiki-dark:#C678DD}html pre.shiki code .s5ixo, html code.shiki .s5ixo{--shiki-default:#383A42;--shiki-dark:#ABB2BF}html pre.shiki code .sW2Sy, html code.shiki .sW2Sy{--shiki-default:#A0A1A7;--shiki-default-font-style:italic;--shiki-dark:#7F848E;--shiki-dark-font-style:italic}html pre.shiki code .sknuh, html code.shiki .sknuh{--shiki-default:#383A42;--shiki-dark:#56B6C2}html pre.shiki code .slOjB, html code.shiki .slOjB{--shiki-default:#383A42;--shiki-dark:#61AFEF}html pre.shiki code .sDhpE, html code.shiki .sDhpE{--shiki-default:#50A14F;--shiki-dark:#98C379}html pre.shiki code .sAdtL, html code.shiki .sAdtL{--shiki-default:#4078F2;--shiki-dark:#61AFEF}html pre.shiki code .so_Uh, html code.shiki .so_Uh{--shiki-default:#986801;--shiki-default-font-style:inherit;--shiki-dark:#D19A66;--shiki-dark-font-style:italic}html pre.shiki code .sAGMh, html code.shiki .sAGMh{--shiki-default:#986801;--shiki-dark:#D19A66}html pre.shiki code .s_Sar, html code.shiki .s_Sar{--shiki-default:#0184BC;--shiki-dark:#56B6C2}html pre.shiki code .sJa8x, html code.shiki .sJa8x{--shiki-default:#E45649;--shiki-dark:#E06C75}html pre.shiki code .st7oF, html code.shiki .st7oF{--shiki-default:#0184BC;--shiki-dark:#ABB2BF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sz0mV, html code.shiki .sz0mV{--shiki-default:#383A42;--shiki-dark:#E06C75}html pre.shiki code .sC09Y, html code.shiki .sC09Y{--shiki-default:#C18401;--shiki-dark:#E5C07B}html pre.shiki code .sMj0N, html code.shiki .sMj0N{--shiki-default:#50A14F;--shiki-dark:#ABB2BF}html pre.shiki code .s7GmK, html code.shiki .s7GmK{--shiki-default:#383A42;--shiki-dark:#E5C07B}",{"title":84,"searchDepth":102,"depth":102,"links":1943},[1944,1945,1949,1952],{"id":12,"depth":102,"text":13},{"id":45,"depth":102,"text":46,"children":1946},[1947,1948],{"id":69,"depth":110,"text":70},{"id":429,"depth":110,"text":430},{"id":998,"depth":102,"text":999,"children":1950},[1951],{"id":1057,"depth":110,"text":430},{"id":1933,"depth":102,"text":1934},null,"2025-09-02","Some history. The naive approach given the architecture back then.","md",{},"\u002Fblog\u002Fbuilding-ml-inference-part-1",{"title":5,"description":1955},"blog\u002Fbuilding-ml-inference-part-1",[1962],"tech","0j3-BAE7xZF4ZVsFSdsfpUjGmI6PWFnjXQr4o1Vg9MA",1778998257280]