English
English
When handling large files, PHP’s hash_update function is commonly used to compute hashes of file contents, such as MD5, SHA-1, or the more secure SHA-256. However, directly using hash_update on large files may encounter performance bottlenecks, mainly due to excessive memory consumption or slow processing speed. This article explores several effective methods to improve the performance of hash_update, accompanied by example code.
Reading the entire file into memory at once before hashing can lead to memory overflow or low efficiency. The best practice is to read the file in chunks, gradually passing data to hash_update.
<?php
$filename = '/path/to/large/file.zip';
$context = hash_init('sha256');
<p>$handle = fopen($filename, 'rb');<br>
if ($handle === false) {<br>
die('Failed to open file');<br>
}</p>
<p>while (!feof($handle)) {<br>
$buffer = fread($handle, 8192); // Read 8KB at a time<br>
hash_update($context, $buffer);<br>
}</p>
<p>fclose($handle);</p>
<p>$hash = hash_final($context);<br>
echo "File hash: $hash\n";<br>
?><br>
Here, an 8KB buffer size is used, which can be adjusted based on system memory and I/O performance.
The buffer size directly affects read/write performance. A buffer that is too small causes many I/O operations, while one that is too large consumes excessive memory. Generally, 8KB to 64KB is a good range. You can test the best performance by adjusting the second parameter of fread.
PHP’s built-in hash_file function is often more efficient at the lower level than reading files in chunks with PHP scripts. If you only need to calculate a hash, consider using it directly:
<?php
$hash = hash_file('sha256', '/path/to/large/file.zip');
echo "File hash: $hash\n";
?>
This method requires no manual file pointer management and typically performs better.
If the environment permits, you can split the file into multiple parts and calculate hashes on each part using multi-processing or multi-threading, then combine the results (e.g., via a custom method or partial hash merges). PHP natively does not support multi-threading, but you can implement this using pcntl_fork or external extensions.
However, this approach is complex and requires special handling of hash algorithm implementations. It is usually suitable only for extremely large files and specialized scenarios.
If the file does not change frequently, consider caching its hash value to reduce repeated computations.
For example, save the file’s last modification time and its hash:
<?php
$filename = '/path/to/large/file.zip';
$cacheFile = '/tmp/file_hash_cache.json';
<p>$cache = json_decode(file_get_contents($cacheFile) ?: '{}', true);<br>
$filemtime = filemtime($filename);</p>
<p>if (isset($cache[$filename]) && $cache[$filename]['mtime'] === $filemtime) {<br>
$hash = $cache[$filename]['hash'];<br>
} else {<br>
$hash = hash_file('sha256', $filename);<br>
$cache[$filename] = ['mtime' => $filemtime, 'hash' => $hash];<br>
file_put_contents($cacheFile, json_encode($cache));<br>
}</p>
<p>echo "File hash: $hash\n";<br>
?><br>
This avoids recalculating the hash every time when the file is unchanged.
The performance of different hashing algorithms varies significantly. MD5 and SHA-1 generally run faster than SHA-256 but offer weaker security. Choose an algorithm according to your scenario, balancing speed and security requirements.
Chunked Reading: Avoid loading the entire file into memory; read in chunks and call hash_update incrementally.
Adjust Buffer Size: Choose a reasonable buffer size to improve I/O performance.
Utilize hash_file Function: PHP’s built-in hash_file function offers better performance.
Parallel Processing: For extremely large files, consider multi-process chunked hashing.
Cache Hash Results: Avoid repeated hash calculations for unchanged files.
Select Appropriate Algorithm: Balance speed and security based on needs.
Mastering these techniques can significantly improve PHP program performance when hashing large files.
<?php
// Example code: chunked file reading and SHA-256 calculation with hash_update
$filename = 'https://gitbox.net/path/to/large/file.zip';
$context = hash_init('sha256');
<p>$handle = fopen($filename, 'rb');<br>
if ($handle === false) {<br>
die('Failed to open file');<br>
}</p>
<p>while (!feof($handle)) {<br>
$buffer = fread($handle, 65536); // 64KB<br>
hash_update($context, $buffer);<br>
}</p>
<p>fclose($handle);</p>
<p data-is-last-node="" data-is-only-node="">$hash = hash_final($context);<br>
echo "File hash: $hash\n";<br>
?><br>
