Example LZ4 compress and decompress

Feb 6, 2014 at 12:50 PM
Edited Feb 6, 2014 at 12:51 PM
Hi,

I just wanted to share a very basic example of encode and decode LZ4.
If somebody has a better solution for the estimated size of the decompress buffer, please share :)

Cheers,
Bagwani
using System;
using LZ4s;

namespace ExampleLz4
{
    public static class Lz4
    {
        public static byte[] CompressBytes(byte[] uncompressed)
        {
            if (uncompressed == null || uncompressed.Length == 0)
            {
                throw new ArgumentException("uncompressed data is null or size is 0");
            }

            return LZ4Codec.Encode32(uncompressed, 0, uncompressed.Length);
        }

        public static byte[] DecompressBytes(byte[] encoded)
        {
            if (encoded == null || encoded.Length == 0)
            {
                throw new ArgumentException("encoded data is null or size is 0");
            }

            byte[] decoded;

            // estimate, 10 times the compressed size
            byte[] buffer = new byte[encoded.Length * 10]; 
            int decodedsize = LZ4Codec.Decode32(encoded, 0, encoded.Length, buffer, 0, buffer.Length, false);
            
            if (decodedsize > 0)
            {
                decoded = new byte[decodedsize];
                Array.Copy(buffer, decoded, decodedsize);
            }
            else
            {
                throw new InvalidOperationException("Failed to decode");
            }

            return decoded;
        }
    }
}
Apr 3, 2014 at 10:11 AM
Hi Krashan,

I am trying to de-compress compressed data using LZ4 using this example code in above post, but it take more time than expected like for my case it takes 6mb/sec on i7 8gb ram. I am looking to improve this speed. Would you please suggest a better way of compression/de-compression?

Secondly I am concerned about the following statement in the decompression function.
// estimate, 10 times the compressed size
byte[] buffer = new byte[encoded.Length * 10];

In my case it gives error (LZ4 block is corrupted, or invalid length has been given.) until I change it to 30-40 times.

Thanks
Coordinator
Apr 3, 2014 at 1:49 PM
I'm not the author of this code, and your concerns are definitely justified. This is generally bad idea to allocate A LOT of memory only because you didn't want to sacrifice 4 bytes to store uncompressed length in a first place.

When I get home I'll write some Documentation about putting compressed data into some neat envelope.
Apr 3, 2014 at 2:12 PM
That would be of a great help. waiting... :)
Coordinator
Apr 16, 2014 at 8:16 AM
I've added some documentation for it. Check this out.