如何从 TXT 文件中删除仅 IP

how to remove only ip from a txt file

提问人：Saeed 提问时间：10/29/2023 最后编辑：RavinderSingh13Saeed 更新时间：10/29/2023 访问量：138

问：

我有两个文件，一个是：ips.yaml

-
  dedicatedip: 1.1.1.11
-
  dedicatedip: 2.2.2.2
-
  dedicatedip: ''
-
  dedicatedip: 3.3.3.3
-
  dedicatedip: 3.3.3.33

这是我的：result.txt

+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some-hostname         | some-value       | Active | External_Networks=1.1.1.11  | not important |
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | the-other.com         | IHaveSomething   | Active | External_Networks=2.2.2.2   | not important |
| 11111111 | not.least.com         |                  | Active | External_Networks=2.2.2.25  | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

我想从中删除每个 ips，以便预期的输出为：ips.yamlresult.txt

+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | not.least.com         |                  | Active | External_Networks=2.2.2.25  | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

这是我目前的 bash：

while read -r yaml_ips
do
    while read -r result_ips
    do
        if [ "$yaml_ips" == "$result_ips" ]
        then
            sed "/$yaml_ips/d" result.txt
        fi
    done < <(grep -Eo 'External_Networks=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' result.txt | awk -F '=' '{print $2}')
done < <(awk '/dedicatedip: [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/ {print $2}' ips.yaml)

这是我目前的输出：

+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some-hostname         | some-value       | Active | External_Networks=1.1.1.11  | not important |
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
正则表达式 bash awk grep

评论

3赞 Ed Morton 10/29/2023
关于您标记为“不重要”的数据 - 是的，是的。很容易想出一个解决方案，但该解决方案对于该字段中的某些值失败，或者根据该字段可能包含的内容提出更好的答案。我无法写出我最初想要的答案，因为我不知道那些“不重要”的值可能是什么。在提出问题时，永远不要假设任何事情都不重要，因为您不知道答案可能是什么 - 提供所有数据值的真正代表性示例。
2赞 Ed Morton 10/29/2023
我敢肯定你之前已经提到过这个，但你的脚本中有嵌套循环，所以，请阅读为什么使用-shell-loop-to-process-text-considered-bad-practice。
0赞 Saeed 10/30/2023
@EdMorton非常感谢您的评论。关于字段，我只想说不是最后一个字段。此外，所有计划名称都包含两个部分，这就是我添加空格的原因。是的，你是对的。我本可以为所有这些写一个计划名称，只是为了表明最后一个字段不是.not-importantnetworks$NFA Bnetworks
1赞 Ed Morton 10/30/2023
很明显，这不是问题所在。我的观点是，根据字段的内容，不同的解决方案是可能的。您现在告诉我们它包含 2 个空格分隔的字符串 - 我们不可能从每个字段中猜到这一点，而如果您在问题中陈述了这一点并为它们提供了代表性值，那将很有用，会导致不同的解决方案被建议，所以很重要。networks$NFplannot important
0赞 Saeed 10/30/2023
@EdMorton是的，先生，这是我错误的假设。

答：

2赞 anubhava 10/29/2023 #1

这 2 次传递可以在单个命令中执行此操作：awk

awk -F ': ' '
FNR == NR {
   if (NF > 1 && $2 ~ /^[0-9]/)
      ips[$2]
   next
}
FNR > 3 {
   nw = $6
   sub(/^[^=]*=/, "", nw)
   if (nw in ips)
      next
}
1' ips.yaml FS='[[:blank:]]*[|][[:blank:]]*' result.txt

i+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | not.least.com         |                  | Active | External_Networks=2.2.2.25  | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

这里：

  • -F ': '将字段分隔符设置为第一个文件，即": "ips.yaml
  • FS='[[:blank:]]*[|][[:blank:]]*'将字段分隔符设置为第二个文件的两侧用空格包围，即|result.txt
  • NF > 1 && $2 ~ /^[0-9]/：过滤第一个文件中的实际地址行，并将每个 IP 地址存储在关联数组中ipips[$2]ips
  • FNR > 3从第 4 行开始处理第 2 个文件，留下前 3 个标题行
  • sub(/^[^=]*=/, "", nw)删除列前和列中的文本，即 在第二个文件中==networks$6
2赞 RavinderSingh13 10/29/2023 #2

对于您显示的示例，请尝试遵循 GNU 代码。awk

awk '
FNR==NR && /^ +/{
  ips[$2]
  next
}
(match($0,/External_Networks=(\S+)/,arr) && (arr[1] in ips)){
  next
}
(/^+-/) || (/^\|/){
  print
  next
} ' FS=": " ips.yaml result.txt

解释：为上述代码添加详细说明。

awk '                              ##starting awk program from here.
FNR==NR && /^ +/{                  ##Checking condition FNR==NR which will be TRUE when ips.yaml file is being read and checking condition if a line starts with 1 or more spaces then do following.
  ips[$2]                          ##Creating array ips with index of $2 here.
  next                             ##next will skip all further statements from here.
}
(match($0,/External_Networks=(\S+)/,arr) && (arr[1] in ips)){ ##Using match function to match External_Networks= followed by all 1 or more non-spaces and storing IPs values into array arr.
  next                             ##next will skip all further statements from here.
}
(/^+-/) || (/^\|/){                ##Checking condition if line starts from +- OR | then do following.
  print                            ##Print that line.
  next                             ##next will skip all further statements from here.
} ' FS=": " ips.yaml result.txt   ##Setting FS to colon space for file ips.yaml and then mentioning result.txt files.
2赞 Ed Morton 10/29/2023 #3

使用任何 awk：

$ awk '
    NR==FNR { if ($1 == "dedicatedip:") ips[$2]; next }
    { o=$0; f=sub(/.*External_Networks=/,""); ip=$1; $0=o }
    !( f && (ip in ips) )
' ips.yaml result.txt
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | not.least.com         |                  | Active | External_Networks=2.2.2.25  | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

评论

1赞 anubhava 10/29/2023
非常好，避免了 2 个 FS 模式

上一个：带正则表达式的 grep - 过滤包含连续相同数字并忽略空格的行

下一个：Bash 正则表达式验证