提问人:Calu River 提问时间:11/7/2023 最后编辑:Calu River 更新时间:11/9/2023 访问量:46
如何从 Oracle 数据库中读取块中的大型 blob 以避免 Java 堆空间 OutOfMemoryError?
How to Read a Large Blob from an Oracle Database in Chunks to Avoid Java Heap Space OutOfMemoryError?
问:
我正在开发一个 Java Spring 服务,该服务从 Oracle 数据库中读取 BLOB 类型的字段。此 BLOB 字段包含一个 zip 文件,我的服务应该在响应正文中将此文件返回给客户端。
我实现了以下代码:
public void datasetDownload(String id, HttpServletResponse response) throws SQLException, IOException {
try (Connection connection = dataSource.getConnection()) {
String sql = "SELECT CONTENT FROM MY_TABLE_NAME WHERE ID = ?";
try (PreparedStatement preparedStatement = connection.prepareStatement(sql)) {
preparedStatement.setString(1, id);
try (ResultSet resultSet = preparedStatement.executeQuery()) {
if (resultSet.next()) {
Blob blob = resultSet.getBlob("CONTENT");
try (InputStream inputStream = blob.getBinaryStream();
OutputStream outputStream = response.getOutputStream()) {
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
}
deleteRecord(id, connection);
}
}
}
}
}
但是,当我调用此服务时,我遇到以下异常:
java.lang.OutOfMemoryError:Java 堆空间
我怀疑该问题是由于将整个 blob 从 Oracle 数据库读取到内存中引起的。为了解决这个问题,我想我需要以较小的块读取 blob 以避免耗尽 Java 堆空间,但我不知道该怎么做。
谢谢你的帮助!
编辑:我尝试添加一些日志:
public void datasetDownload(String id, HttpServletResponse response) throws SQLException, IOException {
try (Connection connection = dataSource.getConnection()) {
String sql = "SELECT CONTENT FROM MY_TABLE_NAME WHERE ID = ?";
try (PreparedStatement preparedStatement = connection.prepareStatement(sql)) {
preparedStatement.setString(1, id);
try (ResultSet resultSet = preparedStatement.executeQuery()) {
if (resultSet.next()) {
log.info("Start downloading the BLOB");
log.info("Total memory available: " + (int) (Runtime.getRuntime().totalMemory() / (1024 * 1024)) + " MB");
log.info("Max memory available: " + (int) (Runtime.getRuntime().maxMemory() / (1024 * 1024)) + " MB");
try (InputStream inputStream = resultSet.getBinaryStream("CONTENT");
OutputStream outputStream = response.getOutputStream()) {
byte[] buffer = new byte[1024 * 1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
log.info("Memory available: " + (int) (Runtime.getRuntime().freeMemory() / (1024 * 1024)) + " MB");
}
}
deleteRecord(id, connection);
log.info("BLOB download completed");
}
}
}
}
}
发生的情况是我可以看到最后一个日志“BLOB 下载完成” 然后我得到OutOfMemoryError。
此外,最后一个可用内存日志为: 可用内存: 570 MB
编辑:实际上我找到了一个解决方案,我用 ZipOutputStream 压缩了 blob,机器不会出现 OutOfMemory。我仍然无法弄清楚问题出在哪里
答:
评论
Blob blob = resultSet.getBlob("CONTENT");
OutOfMemoryError
OutOfMemoryError
log.info(...
log.info(...