当前位置:网站首页>Discussion on the method of using zlib compression and decompression in C

Discussion on the method of using zlib compression and decompression in C

2021-04-27 17:51:29 Compasslg

author :Compasslg

Introduce

Recently used c# To develop an archive editing tool for a game, you need to use Zlib The standard Deflate The algorithm decompresses the data . stay StackOverflow Went on a walk , Find out c# The more common way is provided by Microsoft System.IO.Compression, zlib.net, as well as ICSharpCode Of SharpZipLib. I simply tested and packaged it , I would like to share the achievements and my personal views here .

System.IO.Compression

Generally speaking , Use c# If you can use the tools provided by Microsoft, try to use them as much as possible , One is bug There will be less , The maintenance will be more stable . Besides , The solutions provided by the government are often superior to the third-party tools in terms of optimization .

Although in .NET Framework 4.5 Start System.IO.Compression.DeflateStream Also used Zlib Conduct Deflate Format compression and decompression , But after testing, its compression and decompression results are similar to other Zlib The library is different .
If you look closely, you will find , use DeflateStream The compressed data starts with more than Zlib The compressed data is two bytes less , Ending ratio Zlib Four bytes less ; This output format is called Raw Deflate .
After verification ,C# Provided DeflateStream It can only be compressed or decompressed Raw Deflate, Instead of dealing with standard Zlib Deflate Format ( But it's said that it can generate itself ); But, in turn, ,Zlib It can process or generate this kind of data without head and tail Raw Deflate.
Of course , You can also choose to add it manually header and trailer. How to add specific reference materials that can be read at the end of the article , Because it's not particularly important , I'm just lazy .

Here is my code for compressing and decompressing data simply packaged with this method :

//  Use System.IO.Compression Conduct Deflate Compress 
public static byte[] MicrosoftCompress(byte[] data)
{
    MemoryStream uncompressed = new MemoryStream(data); //  Here is an example of data in memory ; If you need to compress the text , Use  FileStream  that will do 
    MemoryStream compressed = new MemoryStream();
    DeflateStream deflateStream = new DeflateStream(compressed, CompressionMode.Compress); //  Be careful : The first parameter here is where the compressed data should be output 
    uncompressed.CopyTo(deflateStream); //  use  CopyTo  Input the data to be compressed at one time ; You can also use Write Do some input 
    deflateStream.Close();  //  stay Close in , It will be carried out successively  Finish  and  Flush  operation .
    byte[] result = compressed.ToArray();
    return result;
}
//  Use System.IO.Compression Conduct Deflate decompression 
public static byte[] MicrosoftDecompress(byte[] data)
{
    MemoryStream compressed = new MemoryStream(data);
    MemoryStream decompressed = new MemoryStream();
    DeflateStream deflateStream = new DeflateStream(compressed, CompressionMode.Decompress); //  Be careful :  The first parameter here is also to fill in the compressed data , But this time it's data as input 
    deflateStream.CopyTo(decompressed); 
    byte[] result = decompressed.ToArray();
    return result;
}

zlib.net

zlib.net It's a very small open source third-party tool . After my limited research and understanding , This library is more like a semi-finished product , Many functions are not perfect , But the advantage is that it's very lightweight , And with the c++ End use boost::iostreams::zlib The effect is the same .

The following is to use zlib.net Provided ZOutputStream Class to compress data

public static byte[] ZLibDotnetCompress(byte[] data)
{
    MemoryStream compressed = new MemoryStream();
    ZOutputStream outputStream = new ZOutputStream(compressed, 2); 
    outputStream.Write(data, 0, data.Length); //  Here we use  Write  To write data that needs to be compressed ; You can also use the same method as above 
    outputStream.Close();
    byte[] result = compressed.ToArray();
    return result;
}

The following is to use zlib.net Provided ZInputStream Class to unzip the data

public static byte[] ZLibDotnetDecompress(byte[] data, int size)
{
    MemoryStream compressed = new MemoryStream(data);
    ZInputStream inputStream = new ZInputStream(compressed);
    byte[] result = new byte[size];   //  because ZInputStream  Inheritance is BinaryReader instead of Stream,  You can only prepare the output in advance  buffer  And then use  read  Get fixed length data .
    inputStream.read(result, 0, result.Length); //  Notice the  read  The first letter is lowercase 
    return result;
}

You need to go through read To get the decompressed data , At the same time, you need to provide external buffer Used to store output data , This buffer It's a matter of size .
If you're going to use this , In addition to storing compressed data , Add data of the size before compression in places that will not be compressed .

But overall , I personally don't recommend using this tool .

https://github.com/zyborg/zlib.net
http://www.componentace.com/zlib_.NET.htm

SharpZipLib

I finally chose to use SharpZipLib. ( edit : There was no speed test , And the file I need to unzip is not too big , Speed doesn't matter , Otherwise, this option is not recommended ...)

ICSharpCode It is worthy of developing ILSpy The team ,SharpZipLib While providing powerful functions , It's also easy to use . Limited to the subject , This is just about Deflate Format to compress the data stream .

Simply speaking , All you need to do is pass DeflaterOutputStream To compress ,InflaterInputStream To decompress , In addition to compression and decompression in two different classes , Other ways of operation and System.IO.Compression.DeflateStream It can be exactly the same .
And its compression and decompression results and direct use Zlib The official library is as like as two peas , Don't worry about the head and tail data when developing tools to assist other programs , It's very easy .

Here's how I use this simple package :

public static byte[] SharpZipLibCompress(byte[] data)
{
    MemoryStream compressed = new MemoryStream();
    DeflaterOutputStream outputStream = new DeflaterOutputStream(compressed);
    outputStream.Write(data, 0, data.Length);
    outputStream.Close();
    return compressed.ToArray();
}
public static byte[] SharpZipLibDecompress(byte[] data)
{
    MemoryStream compressed = new MemoryStream(data);
    MemoryStream decompressed = new MemoryStream();
    InflaterInputStream inputStream = new InflaterInputStream(compressed);
    inputStream.CopyTo(decompressed);
    return decompressed.ToArray();
}

Speed comparison

In order to compare the advantages and disadvantages of several methods in compression and decompression efficiency , I prepared two sets of data and did a simple test .

The first group is short data , It's a simple string "this is just a string for testing, see how this compression thing works."
The second group is long data , It's an English version downloaded from the Internet 《 A song of ice and fire : Game of thrones 》txt Text , The size is about 1.7mb.

I use each method to compress and decompress short data 1000 Time , Long data 100 Time , The final result is as follows :

Length of Short Data: 144
Length of Long Data: 1685502

============================================
Compress and decompress with Microsoft Zlib Compression (1000 times): 54
Compress and decompress with Microsoft Zlib Compression (long data 100 times): 7924

============================================
Compress and decompress with Zlib.net Compression (1000 times): 254
Compress and decompress with Zlib.net Compression (long data 100 times): 9924

============================================
Compress and decompress with SharpZipLib Compression (1000 times): 442
Compress and decompress with SharpZipLib Compression (long data 100 times): 26782

obvious , No matter long data or short data compression and decompression ,System.IO.Compression The methods provided in are superior to the other two methods .

Zlib.net The disadvantage in speed is not obvious , And the same algorithm SharpZipLib It takes two to three times as long .

summary

Final , As expected , It's official from Microsoft System.IO.Compression The method in has obvious advantages in speed ; Although it won't provide Deflate The head and tail information of , But you can find a way to generate it yourself , And this shortcoming can be completely ignored . Zlib.net Although it's good in speed , It also generates Deflate Compressed head and tail information , But because of its sloppy packaging , It's relatively inconvenient to use . and SharpZipLib It is a pity that , Although other aspects are convenient , But the speed flaw is quite fatal , Only when it is necessary Deflate Instead of RawDeflate Or using .Net Framework Before 4.5 When ( And running time consumption is not important ) It's a lazy use .

Reference and extension

About Zlib

https://zlib.net/

About Deflate and Raw Deflate

https://stackoverflow.com/questions/37845440/net-deflatestream-vs-linux-zlib-difference
https://www.ietf.org/rfc/rfc1950.txt
https://www.ietf.org/rfc/rfc1951.txt

About CSharp System.IO.Compression.DeflateStream

https://docs.microsoft.com/en-us/dotnet/api/system.io.compression.deflatestream?view=net-5.0

One of the developers Mark Adler stay StackOverflow The answer on

deflate and compress Difference of function
https://stackoverflow.com/questions/10166122/zlib-differences-between-the-deflate-and-compress-functions/10168441#10168441

How to manually add header and trailer
https://stackoverflow.com/questions/39939869/data-format-for-system-io-compression-deflatestream

版权声明
本文为[Compasslg]所创,转载请带上原文链接,感谢
https://chowdera.com/2021/04/20210427174954930R.html