shithub: jbig2

Info  •  Files  •  Log

Bug 701197: jbig2dec: Fix incorrectly computed halftone skip mask.

Halftone regions using a skip mask and negative horizontal grid origin
offsets caused issues due to multiplying signed and unsigned integers.
The mixed types expression was computed unsigned unsigned arithmetic
before being converted back to a signed integer. This meant that an
expected negative value became positive.

Several of the test files mentioned in the README rendered incorrectly,
e.g. 200-6-45.jb2. With this fix all files render correctly again.

--- a/jbig2_halftone.c

+++ b/jbig2_halftone.c

@@ -458,7 +458,7 @@

     Jbig2Image *HSKIP = NULL;

     Jbig2PatternDict *HPATS;

     uint32_t i;

-    uint32_t mg, ng;

+    int32_t mg, ng;

     int32_t x, y;

     uint16_t gray_val;

     int code = 0;

@@ -481,8 +481,8 @@

         for (mg = 0; mg < params->HGH; ++mg) {

             for (ng = 0; ng < params->HGW; ++ng) {

-                x = (params->HGX + mg * (int32_t) params->HRY + ng * (int32_t) params->HRX) >> 8;

-                y = (params->HGY + mg * (int32_t) params->HRX - ng * (int32_t) params->HRY) >> 8;

+                x = (params->HGX + mg * params->HRY + ng * params->HRX) >> 8;

+                y = (params->HGY + mg * params->HRX - ng * params->HRY) >> 8;

                 if (x + HPATS->HPW <= 0 || x >= (int32_t) image->width || y + HPATS->HPH <= 0 || y >= (int32_t) image->height) {

                     jbig2_image_set_pixel(HSKIP, ng, mg, 1);