提问人:Kafka 提问时间:12/24/2020 更新时间:12/24/2020 访问量:468
ifstream 和 boost::iostream 在 Linux 和 Windows 上的性能比较
Performances comparison between ifstream and boost::iostream on linux and windows
问:
我需要良好的性能才能在我的实际程序中读取二进制文件。所以我尝试使用内存映射来提高读取速度。在第一次尝试中,我尝试使用 boost::iostream,所以我写了一个小程序来测试性能:
#include <string>
#include <vector>
#include <iostream>
#include <filesystem>
#include <fstream>
#include <cassert>
#include <chrono>
#include <boost/iostreams/device/mapped_file.hpp>
#include <boost/iostreams/stream.hpp>
namespace fs = std::filesystem;
template<typename TYPE>
inline void read_binary_file_ifstream(const fs::path filename, std::vector<TYPE>& result)
{
std::ifstream file(filename, std::ios::in | std::ios::binary);
size_t filesize = fs::file_size(filename);
assert(filesize%sizeof(TYPE) == 0);
result.resize(filesize/sizeof(TYPE));
file.read(reinterpret_cast<char *>(result.data()), filesize);
file.close();
}
template<typename TYPE>
inline void read_binary_file_boost(const fs::path filename, std::vector<TYPE>& result)
{
using boost::iostreams::mapped_file_source;
using boost::iostreams::stream;
size_t filesize = fs::file_size(filename);
assert(filesize%sizeof(TYPE) == 0);
result.resize(filesize/sizeof(TYPE));
mapped_file_source mmap(filename.string().c_str());
stream<mapped_file_source> file(mmap, std::ios::binary);
file.read(reinterpret_cast<char *>(result.data()), filesize);
}
int main()
{
fs::path path = "idx-position-Deces_Agit_FrHex_aPartir1979_Dom_aPartir2000_enCours-liens_age_tranche_age#age_quinquenal_0_100.dat";
std::cout << "file size : " << fs::file_size(path) << std::endl;
std::chrono::time_point<std::chrono::system_clock> start, end;
start = std::chrono::system_clock::now();
for(int i = 0; i<10; ++i)
{
std::vector<uint32_t> result;
read_binary_file_ifstream<uint32_t>(path, result);
}
end = std::chrono::system_clock::now();
std::chrono::duration<double> elapsed_seconds = end-start;
std::cout << "elapsed time ifstream : " << elapsed_seconds.count() << "s\n";
start = std::chrono::system_clock::now();
for(int i = 0; i<10; ++i)
{
std::vector<uint32_t> result;
read_binary_file_boost<uint32_t>(path, result);
}
end = std::chrono::system_clock::now();
elapsed_seconds = end-start;
std::cout << "elapsed time boost iostream : " << elapsed_seconds.count() << "s\n";
return 0;
}
我使用 cmake 进行编译:
cmake_minimum_required(VERSION 3.1)
project (boost_io)
set(Boost_USE_STATIC_LIBS OFF)
set(Boost_USE_MULTITHREADED ON)
set(Boost_USE_STATIC_RUNTIME OFF)
find_package(Boost COMPONENTS iostreams REQUIRED)
add_executable(boost_io main.cpp)
target_compile_features(boost_io PRIVATE cxx_std_17)
target_include_directories(boost_io
PUBLIC
$<$<PLATFORM_ID:Windows>: ${PARENT_DIR_INSTALL_PREFIX}/Boost/include >
)
target_link_libraries(boost_io
PUBLIC
Boost::iostreams
-lstdc++fs)
我尝试了两个系统:Linux 和 Windows(visual studio 2019 生成器)。在linux上,我的结果相当不错:
file size : 3202777528
elapsed time ifstream : 10.1622s
elapsed time boost iostream : 8.10151s
但是在Windows上,我得到了令人惊讶的结果:
file size : 3202777528
elapsed time ifstream : 30.6484s
elapsed time boost iostream : 77.9328s
我的电脑处于双启动状态,因此它是相同的 ssd 磁盘。读取的文件完全相同。我的 boost 版本在 Windows 上是 1.75(我认为在 linux 上更旧,我已经安装了带有 apt 的 boost)。那么为什么我有这些差异,以及为什么 boost 的内存映射在带有 Visual Studio 19 生成器的 Windows 上如此缓慢。它们是快速读取二进制文件的更好方法吗?
答: 暂无答案
评论