发送请求直到获得响应 [重复]-解网

问：

5个月前关闭。

我怎样才能在网页没有得到响应的情况下继续获取网页？

for (node of NodeList) {
  const url = node.getAttribute('href')
  const res = await fetch(url) //Code below won't execute if no response
  const html = await response.text()
  const scraper = new DOMParser()
  const doc = scraper.parseFromString(text, 'text/html')
  alert('successfully parsed')
}

因此，由于它以循环方式工作并一次发送非常多的请求，即使是网络的小而短的问题也会破坏整个过程，我该怎么办？

javascript fetch httprequest httpresponse for-of-loop

const urls = [...NodeList].map(node => node.getAttribute('href'));
const docs = [];
const max = urls.length;
let cnt = 0;
const getHtml = () => {
  if (cnt >= max) {
    processAllDocs(); // here we have them all
    return;
  }
  try {
    const res = fetch(url) //Code below won't execute if no response
      .then(response => {
        if (!response.ok) {
          throw new Error(`HTTP error! Status: ${response.status}`);
        }
        return response.text()
      })
      .then(html => {
        const scraper = new DOMParser()
        const doc = scraper.parseFromString(text, 'text/html')
        docs.push(doc)
        cnt++
        getHtml(); // get next - use setTimeout if you want to throttle
  } catch (err) {
    console.log(err.message);
    if (err.message.includes("something really bad, please stop")) return; // stop all processing (server down or ten 500 errors in a row)
    getHtml(); // get same - use setTimeout if you want to throttle
  }
}
getHtml();

发送请求直到获得响应 [重复]

Send Request Until Get Response [duplicate]

评论

评论