Regression in compression size for level 6 in versions >= 0.5.3

[LaurenzV]

I tried updating flate2 in one of my crates, and noticed that in my tests that the flate streams were getting significantly larger. After some digging, this turned out to be a regression in a zlib-rs release. Using the below code (AI-generated, so maybe not optimally written but it reproduces the regressions):

struct Example {
    name: &'static str,
    source_test: &'static str,
    data: Vec<u8>,
}

fn main() {
    println!("flags=default-features:false features:[std,rust-allocator]");
    println!();

    for example in examples() {
        println!("example={}", example.name);
        println!("source_test={}", example.source_test);
        println!("input_len={}", example.data.len());
        for level in 0..=9 {
            println!("level={level} compressed_len={}", compress(&example.data, level).len());
        }
        println!();
    }
}

fn examples() -> [Example; 2] {
    [
        Example {
            name: "image_deduplicate_luma8_raw",
            source_test: "krilla-tests::image_deduplicate",
            data: decode_png(include_bytes!("../assets/luma8.png")),
        },
        Example {
            name: "image_interpolate_rgba8_raw",
            source_test: "krilla-tests::image_interpolate",
            data: decode_png(include_bytes!("../assets/rgba8.png")),
        },
    ]
}

fn decode_png(bytes: &[u8]) -> Vec<u8> {
    let decoder = png::Decoder::new(std::io::Cursor::new(bytes));
    let mut reader = decoder.read_info().unwrap();
    let mut buffer = vec![0; reader.output_buffer_size()];
    let info = reader.next_frame(&mut buffer).unwrap();
    buffer.truncate(info.buffer_size());
    buffer
}

fn compress(data: &[u8], level: i32) -> Vec<u8> {
    let mut output = vec![0; zlib_rs::deflate::compress_bound(data.len())];
    let config = zlib_rs::deflate::DeflateConfig::new(level);
    let (compressed, rc) = zlib_rs::deflate::compress_slice(&mut output, data, config);
    assert_eq!(rc, zlib_rs::ReturnCode::Ok);
    compressed.to_vec()
}

When pinning to zlib-rs 0.5.2, I get:

example=image_deduplicate_luma8_raw
source_test=krilla-tests::image_deduplicate
input_len=40000
level=0 compressed_len=40011
level=1 compressed_len=745
level=2 compressed_len=436
level=3 compressed_len=450
level=4 compressed_len=399
level=5 compressed_len=388
level=6 compressed_len=342
level=7 compressed_len=341
level=8 compressed_len=341
level=9 compressed_len=339

example=image_interpolate_rgba8_raw
source_test=krilla-tests::image_interpolate
input_len=160000
level=0 compressed_len=160021
level=1 compressed_len=3004
level=2 compressed_len=1718
level=3 compressed_len=1636
level=4 compressed_len=1638
level=5 compressed_len=1629
level=6 compressed_len=1632
level=7 compressed_len=1473
level=8 compressed_len=1425
level=9 compressed_len=1419

Using 0.5.3, I get:

example=image_deduplicate_luma8_raw
source_test=krilla-tests::image_deduplicate
input_len=40000
level=0 compressed_len=40011
level=1 compressed_len=745
level=2 compressed_len=436
level=3 compressed_len=491
level=4 compressed_len=462
level=5 compressed_len=462
level=6 compressed_len=485
level=7 compressed_len=341
level=8 compressed_len=341
level=9 compressed_len=339

example=image_interpolate_rgba8_raw
source_test=krilla-tests::image_interpolate
input_len=160000
level=0 compressed_len=160021
level=1 compressed_len=3004
level=2 compressed_len=1718
level=3 compressed_len=1649
level=4 compressed_len=1683
level=5 compressed_len=1647
level=6 compressed_len=1779
level=7 compressed_len=1473
level=8 compressed_len=1425
level=9 compressed_len=1419

As you can see, for level 6 (which is the level I've been using so far), I go from 342 to 485, for the second it goes from 1632 to 1779. Happy to provide the original files in case desired. Is this regression intentional/expected? Feel free to close if so, and I might just switch to level 7 instead.

[folkertdev]

I'll look into it more later, but since you've got that test set up: what does zlib-ng do here? We try to mirror its output byte-for-byte.

[folkertdev]

From a quick look at the release notes, maybe this is due to #423 which itself mirrors zlib-ng/zlib-ng#1938 in zlib-ng.

[LaurenzV]

Ah yep, can confirm that this is the offending one, the change in medium.rs in particular. Feel free to close the issue if you consider this out-of-scope then.