I can confirm with my test case that NTFS compression DOES OFFER SPEED BENEFITS.
Test setup:
- 15360 x 1080
- DXT1 DDS image stream
- ~10,000 frame duration
- Samsung 850 Evo drive
- Dual Xeon E5-2630 v3 @ 2.4GHz
- Quadro M4000 (seem to get exact same results on GeForce 1080 also)
- CrystalDiskMark read speed ~ 540 MB/s
Results
Without NTFS compression
With NTFS compression
-
<1000 frames - 60fps OK
-
<3000 frames - 30fps OK
-
All frames (10,600) - 29fps OK
-
23.8GB on disk
-
Reported read speed ~ 740MB/s
-
Effective read speed at 29fps ~ 261 MB/s
Notes
The “System and compressed memory” process only uses 4% CPU generally (approximately 3 threads on my setup). With HAP’s “chunked” option, you could theoretically use more threads for decompression, but perhaps we’re limited more by disk read speed than by CPU (suggesting more aggressive CPU compression might not be helpful).
I’m not sure exactly what causes the buffer underruns to appear in the latter parts of playback. They tend to appear all of a sudden, and sometimes the playback recovers (up to 100% buffer) within 500 frames. I suggest playing the whole sequence back many times to guarantee your performance speed.
The symptom of failing at larger frame counts is that it can’t read from the disk fast enough. My theories for why this happens:
- Windows is running out of memory at this point, and starts unloading old frames whilst loading new ones (it’s been keeping a cache of all the uncompressed frames in memory, which is unnecessary)
- It caches the frames to uncompressed disk space (making readback slower)
Most likely the cause is (1).
Final notes
When evaluating your setup, the essential number to look at is the disk usage speed in Task Manager. By using disk compression we’re effectively increasing our disk read speed, and that speed is still the limiting factor of playback. (You might notice that the number only rises after you’ve started playing back for a little while, this is because Windows automatically creates a RAM cache of files).
By ‘effective disk speed’ I mean the equivalent speed at which we can read the DDS’s from disk if they were uncompressed. In the case of NTFS compression, this number can be read from task manager.
This thread is about trying to mimic the personality of HAP codec by using NTFS compression. Although I understand that RAID, M.2, etc are great, there’s plenty of other threads about those options for increasing effective disk speed. Please note that using a faster SSD in combination with NTFS compression will further increase your effective disk speed.
When performing the compression, the disk speed is ~ 200MB/s and CPU is 0.9% (process name is COM Surrogate). This speed is a combination of read and write together.
The HAP compressed version of the same image sequence is 16GB, suggesting that Snappy (HAP’s CPU codec) is more aggressive than the NTFS compression codec, and will allow for better effective disk read speeds (higher fps playback than NTFS compression alone). I don’t know what codec NTFS compression uses.
