PHP 生成器对象未从已处理的外部 JSON API 接收数据

PHP Generator object not receiving data from processed external JSON API

提问人：Mike Hermary 提问时间：11/16/2023 最后编辑：Brian Tompsett - 汤莱恩Mike Hermary 更新时间：11/17/2023 访问量：57

问：

我一直在使用此资源作为参考将基于 Joomla 3 的 CLI 脚本转换为基于 Joomla 4/5 的 API 脚本。该脚本从外部 API 检索新闻帖子，并将它们作为单独的文章添加到 Joomla。

我有这个功能在工作。它成功与外部 API 连接，并在使用函数时在浏览器中输出数据数组。我在下面包含了全部功能。$processprint_r$process

$process = function (string $givenHttpVerb, string $endPoint, string $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

该函数是使用$process$dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl, $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);

我有一个接受匿名函数和 的函数，它用于基于 Joomla 的站点。这是 upgrade.domain.com/api/index.php/v1。完整功能如下。$generator$dataSourceResponse$apiUrl$apiUrl$generator

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => $article->title,
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => $article->topic,
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
    }
    $dataString = json_encode($data);
  } finally {
    echo 'Done processing data' . PHP_EOL;
  }
};

该函数是使用$generator$postData = $generator($dataSourceResponse, $apiUrl);

当我使用该变量时，浏览器中会显示以下内容：.print_r$postDataGenerator Object() Done processing data

在我看来，从外部 API 检索到的数据没有与通过 API 将新闻帖子项目插入 Joomla 文章的函数成功共享。$generator

我在下面提供了完整的PHP脚本以供参考。

declare(strict_types=1);
ini_set('error_reporting', E_ALL & ~E_DEPRECATED);

$dataSourceUrl = 'https://api.dtn.com/publishing/news/articles?categoryId=1%2C2%2C3%2C4%2C5%2C6%2C16%2C17&limit=10&maxAge=15&apikey=redacted';
$apiUrl = 'https://upgrade.domain.com/api/index.php/v1';
$token = redacted;
$timeOut = 60;

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => $article->title,
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => $article->topic,
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
    }
    $dataString = json_encode($data);
  } finally {
    echo 'Done processing data' . PHP_EOL;
  }
};

$process = function (string $givenHttpVerb, string $endPoint, string $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

$dataSourceHttpVerb = 'GET';
$dataSourceDataString = '';
$dataSourceHeaders = [
  'Accept: application/json',
  'Accept-Encoding: deflate, gzip, br',
  'Content-Type: application/json',
  'Connection: keep-alive',
];
$dataSourceTimeout = 60;
$dataSourceTransport = curl_init();

try {
  $dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl,   $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);
  $postData = $generator($dataSourceResponse, $apiUrl);

  foreach ($postData as $dataString) {
    if (!is_string($dataString)) {
      continue;
    }
    $curl = curl_init();
    try {
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
  
      $output = $process('POST', $apiUrl, $dataString, $headers, $timeOut, $curl);
  
    } catch (Throwable $e) {
      echo $e->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($curl);
    }
  }
} catch (Throwable $e) {
  echo $e->getMessage() . PHP_EOL;
} finally {
  curl_close($dataSourceTransport);
}

这是从基于 CLI 的脚本中运行 PHP，我尝试对其进行调整以供参考。foreach loop

foreach ($articles as $article) {
  $articleData = [
    'id' => 0,
    'catid' => 13,
    'title' => $article->title,
    'introtext' => $article->storySummary,
    'fulltext' => $article->content,
    'note' => $article->topic,
    'state' => 1,
    'access' => 1,
    'created_by' => 386,
    'created_by_alias' => 'DTN News',
    'language' => '*',
  ];

  if (!$articleModel->save($articleData)) {
    throw new Exception($articleModel->getError());
  }
}

更新

在数据数组上使用的输出：var_dump(json_encode($data)); die();

{
  "id": 0,
  "catid": 13,
  "title": "DTN Midday Livestock Comments",
  "articletext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "introtext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "fulltext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "note": "DTN\\/Ag\\/Markets",
  "state": 1,
  "access": 1,
  "created_by": 386,
  "created_by_alias": "DTN News",
  "language": "*"
}
php json curl joomla

评论

0赞 Nick 11/16/2023
当它成功时，你实际上并没有从你的生成器函数中获得任何东西......
0赞 Mike Hermary 11/16/2023
@Nick我已更改为 ，但生成器函数仍然没有输出。我分享的资源示例以相同的方式定义产量，只是组合数组。$dataString = json_encode($data);yield json_encode($data);
0赞 Nick 11/16/2023
嗯......你有没有检查过它实际上返回了任何文章？$process
0赞 Mike Hermary 11/16/2023
@Nick 是，在 PHP 脚本上显示来自外部 JSON API 的预期数据。你可以在这里看到它的实际效果var_dump($dataSourceResponse);
0赞 Mike Hermary 11/16/2023
@Nick 当我从 foreach 数组中获取数组时，浏览器中会显示一个结果。今天的新闻提要最多应显示三个。我已将此输出添加到我的原始问题中。var_dump(json_encode($data)); die();$data

答：

1赞 Mike Hermary 11/17/2023 #1

我通过重构我的代码并从我用作此脚本基础的资源中实现一些缺失的功能来解决了我的问题。我在下面突出显示了重构的代码和新实现的代码，以及完整的脚本以供参考。

通过移动循环内部进行了重构。我还添加了字符串操作函数来清理一些导入的文本内容。$data foreach loopyield (json_encode($data));

// Strip CSS classes from tags without removing the tags themselves
$stripClasses = '/\bclass=["\'][^"\']*["\']/';

foreach ($resource as $article) {
  $data = [
    'id' => 0,
    'catid' => 13,
    'title' => ucwords(strtolower($article->title)),
    'alias' => strtolower(str_replace(' ', '-', urlencode($article->title))),
    'articletext' => $article->content,
    'introtext' => $article->storySummary,
    'fulltext' => $article->content,
    'note' => 'DTN supplied topics: ' . str_replace('/', ', ', $article->topic),
    'state' => 1,
    'access' => 1,
    'created_by' => 386,
    'created_by_alias' => 'DTN News',
    'language' => '*',
  ];
  
  if (isset($data['articletext']) && !empty($data['articletext'])) {
    $data['articletext'] = strip_tags($data['articletext'], $allowedTags);
    $data['articletext'] = preg_replace($stripClasses, '', $data['articletext']);
  } else {
    $data['articletext'] = '';
  }
  if (isset($data['introtext']) && !empty($data['introtext'])) {
    $data['introtext'] = strip_tags($data['introtext'], $allowedTags);
    $data['introtext'] = preg_replace($stripClasses, '', $data['introtext']);
  } else {
    $data['introtext'] = '';
  }
  if (isset($data['fulltext']) && !empty($data['fullotext'])) {
    $data['fulltext'] = strip_tags($data['fulltext'], $allowedTags);
    $data['fulltext'] = preg_replace($stripClasses, '', $data['fulltext']);
  } else {
    $data['fulltext'] = '';
  }
  
  if ($data === false) {
  } else {
    yield (json_encode(
      $data
    ));
  }
}

我从资源中包含了这个匿名函数来处理外部和 Joomla 端点。这对于解决我在重构代码时遇到的错误至关重要。404 Resource not found

$endPoint = function (string $givenBaseUrl, string $givenBasePath, int $givenResourceId = 0): string {
  return $givenResourceId ? sprintf('%s/%s/%s/%d', $givenBaseUrl, $givenBasePath, 'content/articles', $givenResourceId)
: sprintf('%s/%s/%s', $givenBaseUrl, $givenBasePath, 'content/articles');
};

我从匿名函数中删除了该变量，因为它没有必要。$apiUrl$postData

$postData = $generator($dataSourceResponse);

最后一个重构中的第一个重构，以检查 API 或 API 并处理新包含的匿名函数。foreach looptry > catch > finallyPATCHPOST$endPoint

$storage = [];
foreach ($postData as $dataString) {
  if (!is_string($dataString)) {
    continue;
  }
  $curl = curl_init();
  try {
    $decodedDataString = json_decode($dataString);
    if ($decodedDataString === false) {
      continue;
    }
  
    $headers = [
      'Accept: application/vnd.api+json',
      'Content-Type: application/json',
      'Content-length: ' . mb_strlen($dataString),
      sprintf('X-Joomla-Token: %s', trim($token)),
    ];
    $pk = (int) $decodedDataString->id;
    $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $curl);
  
    $decodedJsonOutput = json_decode($output);
  
    if (isset($decodedJsonOutput->errors)) {
      $storage[] = ['mightExists' => $decodedJsonOutput->errors[0]->code === 400, 'decodedDataString' => $decodedDataString];
      continue;
    }
  
  } catch (Throwable $e) {
    echo $e->getMessage() . PHP_EOL;
    continue;
  } finally {
    curl_close($curl);
  }
}

第二个是添加的，用于处理错误并检查 Joomla 文章的重复别名。foreach loop

foreach ($storage as $item) {
  $storageCurl = curl_init();
  try {
    if ($item['mightExists']) {
      $pk = (int) $item['decodedDataString']->id;
      $item['decodedDataString']->alias = sprintf('%s-%s', $item['decodedDataString']->alias, bin2hex(random_bytes(4)));
      // No need to do another json_encode anymore
      $dataString = json_encode($item['decodedDataString']);
      // HTTP request headers
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-Length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
      $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $storageCurl);
    }
  } catch (Throwable $storageThrowable) {
    echo $storageThrowable->getMessage() . PHP_EOL;
    continue;
  } finally {
    curl_close($storageCurl);
  }
}

完整脚本

declare(strict_types=1);
// ini_set('error_reporting', E_ALL & ~E_DEPRECATED);

$dataSourceUrl = Redacted;

$baseUrl= Redacted;
$basePath = 'api/index.php/v1';
$token = Redacted;

$timeOut = 120;

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  $allowedTags = ['p', 'img', 'table'];
  $stripClasses = '/\bclass=["\'][^"\']*["\']/';

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => ucwords(strtolower($article->title)),
        'alias' => strtolower(str_replace(' ', '-', urlencode($article->title))),
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => 'DTN supplied topics: ' . str_replace('/', ', ', $article->topic),
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
  
      if (isset($data['articletext']) && !empty($data['articletext'])) {
        $data['articletext'] = strip_tags($data['articletext'], $allowedTags);
        $data['articletext'] = preg_replace($stripClasses, '', $data['articletext']);
      } else {
        $data['articletext'] = '';
      }
      if (isset($data['introtext']) && !empty($data['introtext'])) {
        $data['introtext'] = strip_tags($data['introtext'], $allowedTags);
        $data['introtext'] = preg_replace($stripClasses, '', $data['introtext']);
      } else {
        $data['introtext'] = '';
      }
      if (isset($data['fulltext']) && !empty($data['fullotext'])) {
        $data['fulltext'] = strip_tags($data['fulltext'], $allowedTags);
        $data['fulltext'] = preg_replace($stripClasses, '', $data['fulltext']);
      } else {
        $data['fulltext'] = '';
      }
  
      if ($data === false) {
      } else {
        yield (json_encode(
          $data
        ));
      }
    }
  } finally {
    // echo 'Done processing data' . PHP_EOL;
  }
};

$endPoint = function (string $givenBaseUrl, string $givenBasePath, int $givenResourceId = 0): string {
  return $givenResourceId ? sprintf('%s/%s/%s/%d', $givenBaseUrl, $givenBasePath, 'content/articles', $givenResourceId)
: sprintf('%s/%s/%s', $givenBaseUrl, $givenBasePath, 'content/articles');
};

$process = function (string $givenHttpVerb, string $endPoint, string   $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

$dataSourceHttpVerb = 'GET';
$dataSourceDataString = '';
$dataSourceHeaders = [
  'Accept: application/json',
  'Accept-Encoding: deflate, gzip, br',
  'Content-Type: application/json',
  'Connection: keep-alive',
];
$dataSourceTimeout = 120;
$dataSourceTransport = curl_init();

try {
  $dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl, $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);
  $postData = $generator($dataSourceResponse);

  $storage = [];
  foreach ($postData as $dataString) {
    if (!is_string($dataString)) {
      continue;
    }
    $curl = curl_init();
    try {
      $decodedDataString = json_decode($dataString);
      if ($decodedDataString === false) {
        continue;
      }
  
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
      $pk = (int) $decodedDataString->id;
      $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $curl);
  
      $decodedJsonOutput = json_decode($output);
  
      if (isset($decodedJsonOutput->errors)) {
        $storage[] = ['mightExists' => $decodedJsonOutput->errors[0]->code === 400, 'decodedDataString' => $decodedDataString];
        continue;
      }
  
    } catch (Throwable $e) {
      echo $e->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($curl);
    }
  }

  foreach ($storage as $item) {
    $storageCurl = curl_init();
    try {
      if ($item['mightExists']) {
        $pk = (int) $item['decodedDataString']->id;
        $item['decodedDataString']->alias = sprintf('%s-%s', $item['decodedDataString']->alias, bin2hex(random_bytes(4)));
        // No need to do another json_encode anymore
        $dataString = json_encode($item['decodedDataString']);
        // HTTP request headers
        $headers = [
          'Accept: application/vnd.api+json',
          'Content-Type: application/json',
          'Content-Length: ' . mb_strlen($dataString),
          sprintf('X-Joomla-Token: %s', trim($token)),
        ];
        $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $storageCurl);
      }
    } catch (Throwable $storageThrowable) {
      echo $storageThrowable->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($storageCurl);
    }
  }
} catch (Throwable $e) {
  echo $e->getMessage() . PHP_EOL;
} finally {
  curl_close($dataSourceTransport);
}

上一个：如何测试 testcontainers 使用的 podman 套接字 HTTP

下一个：在 WordPress 中发出 cURL 请求时出现 504 网关超时错误