shithub: libvpx

Info  •  Files  •  Log  •  Branches

Fix error in get_ul_intra_threshold() for 10/12 bit.

The scaling of the threshold for 10 and 12 bit here appears
to be in the wrong direction. For 10 and 12 bit we expect sse
values to be higher and hence the threshold used should be
scaled up not down.

Change-Id: I2678116652b539aef48100e0f22873edd4f5a786

--- a/vp9/encoder/vp9_firstpass.c

+++ b/vp9/encoder/vp9_firstpass.c

@@ -506,10 +506,10 @@

         ret_val = UL_INTRA_THRESH;

         break;

       case VPX_BITS_10:

-        ret_val = UL_INTRA_THRESH >> 2;

+        ret_val = UL_INTRA_THRESH << 2;

         break;

       case VPX_BITS_12:

-        ret_val = UL_INTRA_THRESH >> 4;

+        ret_val = UL_INTRA_THRESH << 4;

         break;

       default:

         assert(0 && "cm->bit_depth should be VPX_BITS_8, "