DiskZIP Review - Finished!

Discussion in 'Windows OS and Software' started by msintle, Apr 11, 2018.

  1. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    Thank you for your support.

    Some battery benchmarks would be interesting, indeed!

    Your actual compression savings will depend on the data you have installed on your PC. To give you an idea, the core OS files create at least 5 GB free disk space on the MaxSpace setting, less with MaxSpeed. The games in the benchmarks above also must have compressed appreciably, resulting in the performance boosts (otherwise, there couldn't have been any acceleration effect). I would say, bet on a 1.5:1 compression ratio (so if you you have 256 GB free disk space, after compression, bet on having 128 GB more free space, for a total of 384 GB free disk space).

    You make an excellent point regarding CPU utilization - I hadn't thought of it this way, but in a sense, DiskZIP consumes your unused CPU cycles to accelerate your disk performance, and hence your PC overall. That's a very nice way of putting it - thank you!
     
    Last edited: Apr 13, 2018
    Maleko48 likes this.
  2. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    All of this is still coming in the review, but to quickly address your questions:

    You can reverse if you have an external drive to use as temporary storage while decompressing (or, an internal partition with enough free space while decompressing). This completely decompresses your existing disk. As I will also cover in the review, you can also apply any existing compressed disk image in either compressed or uncompressed mode, to any target PC - which is extremely convenient for cloning/imaging purposes.

    I have also seen that you can uninstall the DiskZIP software itself from a compressed disk, the software uninstaller doesn't stop you from doing so, and Windows will work just fine after the uninstallation. Of course, once you've uninstalled, you will not be able to maintain or recompress your disk at all - all coming in the main review.

    No repartitioning is required at all. The software works on a per-partition basis, processing whatever partition you choose.

    You may even size partitions after processing (I have done this many times to create a new partition with the free space for use with virtual machines exclusively, which are ideally compressed with DiskZIP Max at dream ratios of 3.1:1, which I decline to cover in this review due to DiskZIP's installer limitations as I discussed above).

    SIDEBAR: If you are going to shrink an existing partition, and Windows offers you extremely little shrink room (say a couple of gigs only, when you actually have tens or hundreds of gigs free space on the partition to shrink), run the following commands in a console:

    defrag c: /d /v /u /h -> performs traditional defrag, even on an SSD.

    defrag c: /x /v /u /h -> performs free disk space consolidation, even on an SSD.

    Why would you ever want to run these commands, even on an SSD? This is because to shrink your partition using the built-in Windows tools, free disk space needs to be contiguous at the end of the partition, or you won't be able to shrink your partition at all. Otherwise, you should of course *not* run a traditional defrag on your SSD at all. If you are using third-party tools to shrink an existing partition, the limitation/workaround above may or may not apply to you.
     
    Last edited: Apr 13, 2018
    alexhawker and Maleko48 like this.
  3. Maleko48

    Maleko48 Notebook Evangelist

    Reputations:
    148
    Messages:
    608
    Likes Received:
    457
    Trophy Points:
    76
    Excellent work. Thank you again.

    And as to your final statement above, I think it's similar to how 10-20 years ago GPUs were strictly for games and were otherwise a waste of hardware that was going unutilized unless you were actually gaming.

    To change that, first they incorporated GPU acceleration at the O/S level to take some strain off the CPU and make better use of that 'wasted' hardware.

    Then they integrated an adequately capable GPU onto the same CPU package and we got integrated graphics so everything just 'worked' automatically for the average Joe.

    Whereas now, we are finally taking advantage of our GPUs for more than just graphics, but now also specialized data processing, VR, statistics, neural networks, and hopefully more cool **** in the future.

    My point being, anything that improves a users experience and makes better average use of their hardware with no or minimal downsides, really only increases the value a user gets out of their machine.
     
    msintle likes this.
  4. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    You're very welcome!

    I realize the review has been taking a while, but I want to do a good job and not rush through anything at all.

    That's a perfect analogy you make with respect to GPUs.

    And I completely agree with this point you made:

    My point being, anything that improves a users experience and makes better average use of their hardware with no or minimal downsides, really only increases the value a user gets out of their machine.
     
    Maleko48 likes this.
  5. Maleko48

    Maleko48 Notebook Evangelist

    Reputations:
    148
    Messages:
    608
    Likes Received:
    457
    Trophy Points:
    76
    All solid info here. Now you really caught my interest with regards to saving space with VMs. I actually have a triple boot system myself right now. Win7/Win10/Ubuntu along with a Mac VM living in my Win10 disk.
     
    Last edited: Apr 13, 2018
    msintle likes this.
  6. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    VM's compress extremely well with DiskZIP Max, but it is not something DiskZIP can take credit for, in my opinion. Their DiskZIP Max software is just a GUI around Microsoft data deduplication drivers, as betrayed by their convoluted installation routine. I certainly use DiskZIP Max because I like the GUI and the information it displays, instead of messing around with PowerShell commandlets. However, there's no real value-add there on behalf of DiskZIP.

    You can just compress your VM partition after installing data deduplication drivers on your Windows 10 workstation OS as I described earlier on this thread, with or without DiskZIP. Data deduplication will not improve performance (I believe, to the contrary, it may hurt performance noticeably). However, it will compress data incredibly well for virtual machines, and it lets you use a virtual machine in a compressed state, which is nothing short of excellent.

    DiskZIP Offline actually exceeds the compression ratios possible with DiskZIP Max (bad naming there DiskZIP!), but if you were to compress your virtual machines with DiskZIP Offline, they would be decompressed the first time you make a write inside a virtual machine. This naturally introduces a huge first-use delay for absolutely no benefit, space or time wise.

    Another thing you could try would be to compress your virtual machines with NTFS compression (with or without DiskZIP Online), which retains the VMs in a compressed state like data deduplication. However, NTFS compression is extremely poor from both a time and a space perspective, so there's no justification to using it for compressing VMs if data deduplication is available. If data deduplication is unavailable, then NTFS may be a viable option to compress VMs and keep using them in a compressed state, of course.
     
    Maleko48 likes this.
  7. Maleko48

    Maleko48 Notebook Evangelist

    Reputations:
    148
    Messages:
    608
    Likes Received:
    457
    Trophy Points:
    76
    At this point the thing I am most unclear on are the deduplication drivers. Maybe this will make more sense in the morning when I read it again after some sleep...

    Right now I'm under the impression these deduplication drivers are essentially a driver-level compression algorithm that handles everything at a lower level than say, a service would?

    I think it's their naming convention compared to how I am interpreting and understanding your explanation of them that is confusing me.

    That and I also haven't really looked into what exactly they are on my own yet because I'm using my cell phone right now and it's annoying trying to do a bunch of multitasking and typing, lol.
     
    msintle likes this.
  8. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    Sorry for the confusion! Any faults with the explanations are fully mine. Let me try again:

    The data deduplication drivers are provided by Microsoft on Windows Server operating systems only. The links I provided are hacks by third parties (be it the DiskZIP partner Orontes, or the unaffiliated second link). They essentially lift the drivers out of a Windows Server installation, and inject them into a Windows 10 (Workstation) installation. Microsoft does not normally allow this, otherwise we'd already be having data deduplication support in Windows 10 out-of-the-box.

    That is why I will not be covering them (or the associated DiskZIP Max software) in this review. I am not sure about the legality of using such components lifted from a Server SKU on a Workstation SKU. I am sure DiskZIP aren't sure about it either, and this is why they aren't installing these drivers with their own setup. Otherwise, I'm sure their setup, which is sophisticated enough to do many things automatically for us already, would be installing dedup drivers for us as well.

    Please stay tuned for the main part of my review where I will be covering the actual software built by DiskZIP - DiskZIP Online and DiskZIP Offline. That is where they add real value, both in terms of acceleration and space.
     
    Maleko48 likes this.
  9. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    3. In Use - DiskZIP Online (Continued)

    Conversely, it can handle a whole lot more threads at lesser compression grades, such as LZNT1 - which appears to be just another name for good 'ole NTFS compression. In fact, DiskZIP Online suggests to spawn double the number of threads as the actual available CPU cores when doing NTFS compression. This makes sense, given that NTFS compression was invented in the early 90s when CPU's were thousands of times less powerful and single-core only:

    maxspeed.png

    Did I mention NTFS compression remains unchanged to this day? This is also the only compression grade you will be able to enjoy if you're running any operating system earlier than Windows 10. Fortunately, Microsoft finally added some new algorithms with Windows 10, as exposed by DiskZIP Online.

    You may have also noticed that, in addition to the CPU thread count slider, the check-boxes below change as well, based on the type of the compression algorithm you choose.

    Don't compress root: This one can be tricky to make sense of. When SSDs first came out, before TRIM was available at the operating system level, some manufacturer SSD applications wrote specific byte-patterns to disk to tell the disk controller that it was supposed to TRIM those areas.

    SIDEBAR: TRIM on an SSD does a kind of garbage collection, sort of like a defragmentation operation specific for SSDs only.

    Now, NTFS compression automatically compresses data on a per-file basis as it is being written to disk. Therefore, enabling NTFS compression on special TRIM byte patterns can totally change the original byte pattern and disrupt the TRIM process. This "don't compress root" option must be a heuristic attempt to prevent that, in case the special TRIM pattern is being recorded on a file at the root of the disk being TRIMmed. Presumably, DiskZIP have already talked to SSD manufacturers, and know TRIM byte pattern files are not stored anywhere else other than the root of a disk?

    DiskZIP Online enables this option only when you select NTFS compression as your desired algorithm. It does not need to enable this option for the other algorithms at all, because they cannot automatically compress new files as they are being written to at all. That is only the exclusive purview of NTFS compression. Otherwise, of course, this would have been potentially problematic for the other algorithms as well.

    This issue does highlight a problem with the new compression algorithms: That they cannot do in-place recompression unlike the decades old NTFS compression. So while Microsoft did finally give us new compression algorithms after more than two decades of waiting, they did not make these algorithms very sustainable, at least not without third party tooling like DiskZIP.

    Don't compress new files in folders: This one is straightforward enough. With NTFS compression, it is possible to mark a folder itself compressed, although NTFS compression happens on a per-file basis. When a folder is marked compressed, any new (or existing) files in that folder will be automatically compressed upon write. While this sounds great, its actually sort of a performance killer, because NTFS compression is very slow and inefficient. So DiskZIP Online leaves this option turned off by default.

    Again, the other algorithms do not enjoy the possibility of automatic compression upon write at all. Any file that is written to gets automatically decompressed upon first write. You also cannot mark a folder to compress automatically using any of the other algorithms - only NTFS compression.

    Clear archive attribute on incompressible files/Compress files only if archive attribute is set: Some files may be pre-compressed, encrypted, or may just be incompressible for any number of reasons. It is also theoretically possible (although extremely rare) that when you compress a file, it grows larger instead, due to a quirk of the compression algorithm. This option helps you tackle such files.

    When you compress your disk for the first time with DiskZIP Online, if this option is enabled, it will use the archive attribute to mark incompressible files. It will do this by clearing the archive attribute. Why is this helpful? This is because the operating system will automatically set the archive attribute on the file when it is modified, alerting DiskZIP Online to the fact that the file may now be compressible.

    So the next time you compress your disk with DiskZIP Online, it will save you a lot of time over the first pass. It will know to skip each file with the archive attribute cleared, as these are files known to be impregnable to the compression algorithm you have chosen.

    Of course, if you change your compression algorithm, you may want to disable this option - as files which were impregnable to a previous compression algorithm may be susceptible to a new one. There's only one way to find out!

    If you're confused about which options to use, just click the Reset button.

    If you're unsure about which algorithm to pick, here's my suggestions:

    1) Stay away from NTFS (LZNT1) if its not the only one available.
    2) If you want to accelerate your disk, use the algorithm indicated by your disk type.
    3) If you want to maximize your free space, select LZX.
    4) Note that sometimes maximum space is the option to choose when accelerating your disk, as is in the case for mechanical/spinning platter based traditional disk drives (but rarely, if ever, SSDs).

    Click OK when done, and then click Compress to begin compressing your disk:


    smallmao.png

    What we get here is another blast from the past. These DiskZIP folks have a sense of humor. They actually replicated the Microsoft ScanDisk disk map display, from way back in MS-DOS 6.x in the early 90s, pixel per pixel, in this modern Windows 10 app! For example, you can maximize the window, and the disk map grows proportionally:

    largemao.png

    As processing continues, the processed areas of the disk are painted yellow, and more free space should appear (the gray shaded areas are empty, with the dots designating files stored on disk).

    A little more information is also available during processing. Click the Status dog-ear at the bottom left of the map to switch to a throughput display:

    status.png

    And clicking Threads found in the same location will display exactly which files DiskZIP Online is currently compressing:

    threads.png

    Do you want to quit? Just click Exit! You may want to do this if you need your PC to become more responsive, as DiskZIP Online can be quite processor and hard disk intensive, especially the first time it is compressing your disk. You may also need to be patient after clicking Exit - DiskZIP Online cannot leave files on disk in an inconsistent state during a compression pass, so it will register your request to exit, but keep processing until the current batch of files it was working on have finished processing.

    If you want to uncompress, just quit, start DiskZIP Online again, and click the Uncompress button.

    That's all there is to it with DiskZIP Online!
     
    Last edited: Apr 14, 2018
    Maleko48 likes this.
  10. msintle

    msintle Notebook Consultant

    Reputations:
    10
    Messages:
    127
    Likes Received:
    50
    Trophy Points:
    41
    3. In Use - DiskZIP Online, Approximating with Built-In Windows Tools

    You may be wondering how to approximate what DiskZIP Online does without using the tool itself. This is certainly possible, although the results will be less than satisfactory, due to a variety of reasons.

    Approximating NTFS Compression: Simply use Windows File Explorer to bring up the properties window of a hard disk, and check the highlighted section:

    approxntfs.PNG

    You will notice that with my disk, this is unchecked, even though I have already compressed my disk using DiskZIP.

    The problems with this approach are:

    1) All folders will be marked for NTFS compression, which will automatically compress all new/updated files in them, but wreck your performance at the same time.

    2) A majority of your disk remains uncompressed, because Windows does not process non-user files with this tool. So all program files, system files - the best of what's highly compressible on your disk - will be skipped.

    3) Only a single thread will be spawned, meaning it can take you up to tens of times longer than is actually necessary to compress your disk, especially on an SSD (but not so much on an HDD).

    4) You can only use this option to enable NTFS compression.


    Approximating Windows 10 Compression: Fire up a command line window as administrator, change to the root of your drive, and enter in one of the following commands, based on what algorithm you want to use:

    compact /c /s /a /i /f /EXE:LZX
    or
    compact /c /s /a /i /f /EXE:XPRESS16K
    or
    compact /c /s /a /i /f /EXE:XPRESS8K
    or
    compact /c /s /a /i /f /EXE:XPRESS4K

    There are also several problems with this approach:

    1) The most serious one is that, on occasion, compact.exe may render your system unbootable, as it processes your entire disk, and apparently this renders some vital boot files compressed as well.

    2)This approach is also single-threaded only, wasting you precious time when you are compressing an SSD.

    3) Even with the force flag (/f), it doesn't seem to be possible to change the compression algorithm on previously compressed files (if they were compressed with Windows 10 compression). You would need to uncompress first, and then recompress.

    4) The command line may not be ideally suited for some people.

    For these reasons, it makes sense to use DiskZIP Online if you're going to compress your disk while Windows is running, even though Windows does offer some approximation of what you can do with DiskZIP using its own built-in capabilities.

    Do you want even better compression and more acceleration? It's finally time to look at DiskZIP's crown jewel, DiskZIP Offline!
     
    Maleko48 likes this.
Loading...

Share This Page