使用 Ruby / Nokogiri 解析随机类名

Using Ruby / Nokogiri to parse randomized class names

提问人:kraftydevil 提问时间:11/7/2020 更新时间:11/7/2020 访问量:38

问:

当涉及到各州美国总统选举选票的剩余百分比时,我一直在手工计算。有这么多的更新和状态 - 这越来越累了。那么,为什么不将流程自动化呢?

这是我正在看的:

enter image description here

问题是类名是随机的。例如,这是我感兴趣的:
<td class="jsx-3768461732 votes votes-row">2,450,186</td>

在 irb 中,我尝试在“votes votes-row”上使用通配符,因为这仅在我在文档中需要它时才会出现:

require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open("https://www.politico.com/2020-election/results/georgia/"))
votes = doc.css("[td*='votes-row']")

...不产生任何结果 (=> [])

我做错了什么以及如何解决?我对 xpath 没问题——我只想确保在文档其他地方所做的更改不会影响查找这些元素。

Ruby XPath CSS 选择器 HTML 解析 nokogiri

评论

0赞 Ben Stephens 11/7/2020
如果您查看 politico.com/2020-election/results/georgia 的来源并在那里找到,则似乎没有投票行结果。也许这些类是用 JS 添加的?

答:

2赞 Ben Stephens 11/7/2020 #1

可能有更好的方法,但是......

require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open("https://www.politico.com/2020-election/results/georgia/"))
votes = doc.css('tr[class*="candidate-row"]').map { |row| row.css('td').map { |cell| cell.content } }
biden_row = votes.find_index { |row| row[0] =~ /biden/i }
trump_row = votes.find_index { |row| row[0] =~ /trump/i }
biden_votes = votes[biden_row][1].split('%')[1]
trump_votes = votes[trump_row][1].split('%')[1]

编辑:从HTML源代码来看,相关表格如下所示:

<table class="jsx-1526769828 candidate-table">
  <thead class="jsx-3554868417 table-head">
    <tr class="jsx-3554868417">
      <th class="table-header jsx-3554868417 candidate-name">
        <h5 class="jsx-3554868417">Candidate</h5>
      </th>
      <th class="table-header jsx-3554868417 percent">
        <h5 class="jsx-3554868417">Pct.</h5>
      </th>
      <th class="table-header jsx-3554868417 vote-bar"></th>
    </tr>
  </thead>
  <tbody class="jsx-2085888330 table-head">
    <tr class="jsx-2677388595 candidate-row">
      <td class="jsx-3948343365 candidate-name name-row">
        <div class="jsx-1912693590 name-only candidate-short-name">Biden</div>
        <div class="jsx-3948343365 candidate-party-tag">
          <div class="jsx-1420258095 party-label dem">dem</div>
        </div>
        <div class="jsx-3948343365 candidate-winner-check"></div>
      </td>
      <td class="jsx-3830922081 percent percent-row">
        <div class="candidate-percent-only jsx-3830922081">49.4%</div>
        <div class="candidate-votes-next-to-percent jsx-3830922081">2,450,193</div>
      </td>
      <td class="jsx-3458171655 vote-bar vote-bar-row">
        <div style="width:49.4%" class="jsx-3458171655 bar dem"></div>
      </td>
    </tr>
    <tr class="jsx-2677388595 candidate-row">
      <td class="jsx-3948343365 candidate-name name-row">
        <div class="jsx-1912693590 name-only candidate-short-name">Trump*</div>
        <div class="jsx-3948343365 candidate-party-tag">
          <div class="jsx-1420258095 party-label gop">gop</div>
        </div>
        <div class="jsx-3948343365 candidate-winner-check"></div>
      </td>
      <td class="jsx-3830922081 percent percent-row">
        <div class="candidate-percent-only jsx-3830922081">49.4%</div>
        <div class="candidate-votes-next-to-percent jsx-3830922081">2,448,635</div>
      </td>
      <td class="jsx-3458171655 vote-bar vote-bar-row">
        <div style="width:49.4%" class="jsx-3458171655 bar gop"></div>
      </td>
    </tr>
  </tbody>
</table>

因此,您可以使用 candidate-votes-next-to-percent 来获取此值。例如:

require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open("https://www.politico.com/2020-election/results/georgia/"))
votes = doc.css('tr[class*="candidate-row"]').map do |row|
  [
    row.css('div[class*="candidate-short-name"]').first.content,
    row.css('div[class*="candidate-votes-next-to-percent"]').first.content
  ]
end
# => [["Biden", "2,450,193"], ["Trump*", "2,448,635"]]

评论

0赞 kraftydevil 11/7/2020
那么 javascript 毕竟不是一个因素吗?
1赞 Ben Stephens 11/7/2020
doc.css('td[class*=“votes”]').size 为 0,并且 votes-row 没有出现在源代码中,所以我想这些类一定是在 JS 中添加的。candidate-row 确实出现在源代码中,因此您可以将其与 Nokogiri 一起使用