使用 C# 读取 azure 函数正文中的 4MB json 文件,进程在正文读取部分挂起

Read a 4MB json file in the body of azure function by C#, and the Process hangs in body reading part

提问人:Ronnie Yuan 提问时间:7/6/2021 最后编辑:Ronnie Yuan 更新时间:7/30/2021 访问量:772

问:

dotnet 版本:5.0.203 ide:JetBrain Rider

线程池日志

当我调试到函数中时,我只能看到线程池中的线程开始和退出,而不是进入断点。

所有的读取和写入都是由 Task Async Await 完成的。

之后,我尝试了MemoryStream,将读取器的缓冲区大小添加到40000+,但它仍然挂起。

控制台日志:控制台日志

再现样本:

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.WebUtilities;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace XXXXXXXX.XXXXXX.Functions
{
    
    public static class TestFunction
    {
        [Function("TestFunction")]
        public static async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function,  "post")]
            HttpRequestData req,
            FunctionContext executionContext)
        {
            var logger = executionContext.GetLogger("TestFunction");
            logger.LogInformation("C# HTTP trigger function processed a request.");
            

            /* test01 */
            // var clDataJsonStr = await new HttpRequestStreamReader(req.Body, Encoding.UTF8, 4096).ReadToEndAsync();
            // var clDataJsonStr = await new HttpRequestStreamReader(req.Body, Encoding.UTF8, 4096).ReadToEndAsync();
            
            // req.Body.Seek(0, SeekOrigin.Begin);
            /* test02 */
            // var serializer = new JsonSerializer(); 
            // ClData clData;
            
            // using (var stream = new StreamReader(req.Body, Encoding.UTF8, false, 4096))
            // {
            //     using (JsonReader reader = new JsonTextReader(stream))
            //     {
            //         while (await reader.ReadAsync())
            //         {
            //             if (reader.TokenType != JsonToken.StartObject) continue;
            //             clData = serializer.Deserialize<ClData>(reader);
            //             logger.LogInformation(clData.ToString());
            //         }
            //     }
            // }
            
            /* test03 */
            // var stringReader = new StringReader(req.Body.ToString());
            // var str = await stringReader.ReadToEndAsync();
            
            // Console.WriteLine(str);

            /*test04*/
            var ms= new MemoryStream();
            await req.Body.CopyToAsync(ms);
            var jsonBytes = ms.ToArray();
            logger.LogInformation(jsonBytes.Length.ToString());
            
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

            await response.WriteStringAsync("Welcome to Azure Functions!");

            return response;
            
        }
    }
}

测试 Json 文件:bycc.json

C# .NET HTTP IO AZURE-FUNCTIONS

评论

1赞 aepot 7/8/2021
如果答案有帮助,您可以将其标记为已接受。
0赞 Harshita Singh 8/10/2021
下面的答案有帮助吗?

答:

0赞 Harshita Singh 7/6/2021 #1

您可以改用来读取 JSON 正文:StreamReader

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
Class1[] data = JsonConvert.DeserializeObject<Class1[]>(requestBody);

此代码适用于您的有效负载:

enter image description here

enter image description here

而且,这是在上面的屏幕截图中:Class1

public class Class1
    {
        public string ReportDay { get; set; }
        public string Type { get; set; }
        public string ReportTypeDescription { get; set; }
        public string DoorName { get; set; }
        public string DoorCode { get; set; }
        public string Cc { get; set; }
        public float TotalSaleAmountIncludingVisitors { get; set; }
        public float TotalSaleAmountIncludingVisitorsLy { get; set; }
        public float TotalSaleAmountIncludingVisitorsEvol { get; set; }
        public float NewCustomerSaleAmount { get; set; }
        public float NewCustomerSaleAmountLy { get; set; }
        public float NewCustomerSaleAmountEvol { get; set; }
        public float OldCustomerSaleAmount { get; set; }
        public float OldCustomerSaleAmountLy { get; set; }
        public float OldCustomerSaleAmountEvol { get; set; }
        public float VisitorSaleAmount { get; set; }
        public float VisitorSaleAmountLy { get; set; }
        public float VisitorSaleAmountEvol { get; set; }
        public float CustomerSaleRatio { get; set; }
        public float TotalSaleQtyIncludingVisitor { get; set; }
        public float TotalSaleQtyIncludingVisitorLy { get; set; }
        public float TotalSaleQtyIncludingVisitorEvol { get; set; }
        public int NewCustomerSaleQty { get; set; }
        public int NewCustomerSaleQtyLy { get; set; }
        public float NewCustomerSaleQtyEvol { get; set; }
        public int OldCustomerSaleQty { get; set; }
        public int OldCustomerSaleQtyLy { get; set; }
        public float OldCustomerSaleQtyEvol { get; set; }
        public int VisitorSaleQty { get; set; }
        public int VisitorSaleQtyLy { get; set; }
        public float VisitorSaleQtyEvol { get; set; }
        public int TotalActiveCustomerQty { get; set; }
        public int TotalActiveCustomerQtyLy { get; set; }
        public float TotalActiveCustomerQtyEvol { get; set; }
        public int NewCustomerQty { get; set; }
        public int NewCustomerQtyLy { get; set; }
        public float NewCustomerQtyEvol { get; set; }
        public int OldCustomerQty { get; set; }
        public int OldCustomerQtyLy { get; set; }
        public float OldCustomerQtyEvol { get; set; }
        public float NewCustomerAchievingRate { get; set; }
        public float AusNotIncludingVisitor { get; set; }
        public float AusNotIncludingVisitorLy { get; set; }
        public float AusEvol { get; set; }
        public float NewCustomerAus { get; set; }
        public float NewCustomerAusLy { get; set; }
        public float NewCustomerAusEvol { get; set; }
        public float OldCustomerAus { get; set; }
        public float OldCustomerAusLy { get; set; }
        public float OldCustomerAusEvol { get; set; }
        public float IptNotIncludingVisitor { get; set; }
        public float IptNotIncludingVisitorLy { get; set; }
        public float IptNotIncludingVisitorEvol { get; set; }
        public float NewCustomerIpt { get; set; }
        public float NewCustomerIptLy { get; set; }
        public float NewCustomerIptEvol { get; set; }
        public float OldCustomerIpt { get; set; }
        public float OldCustomerIptLy { get; set; }
        public float OldCustomerIptEvol { get; set; }
        public int IptEqOneCount { get; set; }
        public int IptEqOneCountLy { get; set; }
        public float IptEqOneCountEvol { get; set; }
        public float PurchaseFreq { get; set; }
        public float PurchaseFreqLy { get; set; }
        public float PurchaseFreqEvol { get; set; }
        public float RecruitmentRate { get; set; }
        public float WechatBindingRate { get; set; }
        public float WechatBindingRateEvol { get; set; }
        public float CompanyWechatBindingRate { get; set; }
        public float CompanyWechatBindingRateEvol { get; set; }
        public float NewCustomerRepurchaseRatePerMonth { get; set; }
        public int NewCustomerRepurchaseQtyPerMonth { get; set; }
        public float NewCustomerRepurchaseAmountPerMonth { get; set; }
        public float NewCustomerRepurchaseAusPerMonth { get; set; }
        public float NewCustomerRepurchaseIptPerMonth { get; set; }
        public float NewCustomerRepurchaseRatePerThreeMonth { get; set; }
        public int NewCustomerRepurchaseQtyPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseAmountPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseAusPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseIptPerThreeMonth { get; set; }
        public float NewCustomerRepurchaseRatePerSixMonth { get; set; }
        public int NewCustomerRepurchaseQtyPerSixMonth { get; set; }
        public float NewCustomerRepurchaseAmountPerSixMonth { get; set; }
        public float NewCustomerRepurchaseAusPerSixMonth { get; set; }
        public float NewCustomerRepurchaseIptPerSixMonth { get; set; }
        public int SkuServiceTimeCount { get; set; }
        public int SkuServicePeopleTimeCount { get; set; }
        public int SkuServicePeopleCount { get; set; }
        public int PurchasePeopleCountAfterService { get; set; }
        public float PurchaseRateAfterService { get; set; }
        public float PurchaseAmountOfPeopleCountAfterService { get; set; }
        public int PotentialCustomer { get; set; }
        public float PotentialCustomerInversionRate { get; set; }
        public float RepurchaseRate { get; set; }
        public float RetentionRate { get; set; }
        public float AnnualSpending { get; set; }
    }

评论

0赞 aepot 7/7/2021
可以忽略吗?IDisposableStreamReader
1赞 Harshita Singh 7/7/2021
是的, 你可以的。在这里查看 Joe 的评论: stackoverflow.com/questions/692263/...
0赞 Harshita Singh 7/8/2021
这有帮助吗?
0赞 Ronnie Yuan 7/14/2021
我的 azure 函数是 isolated-worker(.NET 5+),req 实际上是旧 api 的 HttpRequest 的 HttpRequestData intead。这会受到 api 更改的影响吗?
0赞 Harshita Singh 7/30/2021
是的,这适用于 HttpRequestData: learn.microsoft.com/en-us/dotnet/api/... also .更新了我的代码。请检查