就我个人而言，我认为尝试发出 500 个并发请求总是容易出错.您提到您是异步执行的，但实际上，当您启动 500 个热"程序时，您的代码中并没有很多异步.任务然后等待它们全部完成.

Personally, I think attempting to issue 500 concurrent requests is always going to be error prone. You mention that you're doing it asynchronously, but in reality there's not a whole lot of asynchrony in your code as you fire-up 500 "hot" tasks then wait for them all to finish.

我会使用信号量来控制一次可以发出多少请求.您可能需要玩弄数字才能找到***位置.

I would use a semaphore to control how many requests can be made at once. You may have to play with the numbers to find the sweet spot.

以下代码在 LINQPad 中运行良好(尽管 bing 很快注意到奇数个请求并开始向页面添加验证码):

The following code works well in LINQPad (although bing quickly notices the odd number of requests and starts adding a CAPTCHA to the page):

// using System.Threading;
async Task Main()
{
    var httpClient = new HttpClient();
    var urls = Enumerable.Range(1, 500).Select(e => "https://www.bing.com/").ToList();
    
    // 10 concurrent requests - tweak this number
    var semaphore = new SemaphoreSlim(10, 10);
    
    var tasks = urls.Select(u => MakeRequest(u, semaphore, httpClient));
        
    var allResponses = await Task.WhenAll(tasks);
    
    // Do something with allResponses
}

private async Task<string> MakeRequest(string url, SemaphoreSlim semaphore, HttpClient httpClient)
{
    try
    {
        await semaphore.WaitAsync();
        var request = new HttpRequestMessage(HttpMethod.Get, new Uri(url));
        var response = await httpClient.SendAsync(request);
        
        // Add an optional delay for further throttling:
        //await Task.Delay(TimeSpan.FromMilliseconds(100));
        
        return await response.Content.ReadAsStringAsync();
    }
    finally
    {
        semaphore.Release();
    }
}