shithub: libvpx

Info  •  Files  •  Log  •  Branches

vp9: Clamp tx_size in model_rd_large

For nonrd_pickmode: add clamp/check to make
sure tx_size is not set to lower than 8X8,
for the model_rd_large function (which is only
called for big block sizes).

No change in behavior.

Change-Id: I9c6093068e406ac16cfd6784ba75868906225378

--- a/vp9/encoder/vp9_pickmode.c

+++ b/vp9/encoder/vp9_pickmode.c

@@ -416,6 +416,9 @@

                      tx_mode_to_biggest_tx_size[cpi->common.tx_mode]);

   }

+  // The code below for setting skip flag assumes tranform size of at least 8x8,

+  // so force this lower limit on transform.

+  if (tx_size < TX_8X8) tx_size = TX_8X8;

   xd->mi[0]->tx_size = tx_size;

   if (cpi->oxcf.content == VP9E_CONTENT_SCREEN && x->zero_temp_sad_source &&