NodeJS Promise 消耗太多内存？-解网

问：

我正在尝试分析 NodeJS 在处理异步函数方面的有效性。

我有下面的 NodeJS 脚本来启动 10 数百万个 Promise，这些 Promise 将休眠 2 秒以模拟密集的后端 API 调用。脚本运行了一段时间（~30 秒），消耗了多达 4096 MB 的内存并引发了错误。JavaScript heap out of memory

承诺真的会消耗那么多内存吗？
为什么 NodeJS 在使用太多内存时应该适合 I/O 密集型操作？
Golang 只用 10MB 的内存来处理 100 数百万个 Go Routine，Golang 在处理 I/O 密集型操作方面是否比 NodeJS 更好？

const sleep = async (ms) => new Promise((resolve) => setTimeout(resolve, ms));

const fakeAPICall = async (i) => {
  await sleep(2000);
  return i;
};

const NUM_OF_EXECUTIONS = 1e7;
console.time(`${NUM_OF_EXECUTIONS} executions:`);

[...Array(NUM_OF_EXECUTIONS).keys()].forEach((i) => {
  fakeAPICall(i).then((r) => {
    if (r === NUM_OF_EXECUTIONS - 1) {
      console.timeEnd(`${NUM_OF_EXECUTIONS} executions:`);
    }
  });
});

错误

<--- Last few GCs --->

[41215:0x10281b000]    36071 ms: Mark-sweep (reduce) 4095.5 (4100.9) -> 4095.3 (4105.7) MB, 5864.0 / 0.0 ms  (+ 1.3 ms in 2767 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 7190 ms) (average mu = 0.296, current mu = 0.[41215:0x10281b000]    44534 ms: Mark-sweep (reduce) 4096.3 (4104.7) -> 4096.3 (4105.7) MB, 8461.4 / 0.0 ms  (average mu = 0.140, current mu = 0.000) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
 1: 0x100098870 node::Abort() [/usr/local/opt/node@14/bin/node]
 2: 0x1000989eb node::OnFatalError(char const*, char const*) [/usr/local/opt/node@14/bin/node]
 3: 0x1001a6d55 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/opt/node@14/bin/node]
 4: 0x1001a6cff v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/opt/node@14/bin/node]
 5: 0x1002dea5b v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/opt/node@14/bin/node]
 6: 0x100316819 v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject, int) [/usr/local/opt/node@14/bin/node]

节点 .js 性能内存泄漏比较

const sleep = async (ms) => { // redundant async - Promise#1
  return new Promise((resolve) => setTimeout(resolve, ms)); // Promise#2
}

const fakeAPICall = async (i) => { // async - Promise#3
  await sleep(2000); // await - Promise#4
  return i;
};

const NUM_OF_EXECUTIONS = 1e7;

console.time(`${NUM_OF_EXECUTIONS} executions:`);

for (let i = 0; i < NUM_OF_EXECUTIONS; i++) {
  fakeAPICall(i).then((r) => { // then - Promise#5
    if (r === NUM_OF_EXECUTIONS - 1) {
      console.timeEnd(`${NUM_OF_EXECUTIONS} executions:`);
    }
  });
}

在每次迭代中，您实际上至少创建了 5 个 promise 和一个生成器，因此您在内存中有 5000 万个 promise 和大量其他对象。这是很多的，因为它们是用JS编写的纯JS对象，当然，它们比低级预编译语言消耗更多的内存。节点与低内存消耗无关，但内存成为您案例中的瓶颈。承诺易于使用，如果您需要内存优化 - 纯回调可以更便宜。

在这里，我们创建了 10M 的承诺：

const NUM_OF_EXECUTIONS = 5_000_000;

console.log(`Start `, NUM_OF_EXECUTIONS);

const sleep = (ms, i) => new Promise((resolve) => setTimeout(resolve, ms, I)); // Promise#1

console.time(`${NUM_OF_EXECUTIONS} executions`);

for (let i = 0; i < NUM_OF_EXECUTIONS; i++) {
  sleep(2000, i).then((r) => { // then - Promise#2
    if (r === NUM_OF_EXECUTIONS - 1) {
      console.timeEnd(`${NUM_OF_EXECUTIONS} executions`);
    }
  });
}

内存（2.6 GB）：

Start  5000000
{
  rss: '2.72 GB',
  heapTotal: '2.68 GB',
  heapUsed: '2.6 GB',
  external: '308 kB',
  arrayBuffers: '10.4 kB'
}
5000000 executions: 24.776s

Process finished with exit code 0

NodeJS Promise 消耗太多内存？

NodeJS Promises consume too much memory?

评论

评论