shithub: libvpx

Download patch

ref: 4f95a7fe3ef8e786dbb6c9c1dc3240da24538f92
parent: e7e426b2748e93d32be21457902cbcf97de350a9
author: Yaowu Xu <yaowu@google.com>
date: Wed Apr 9 07:37:27 EDT 2014

Fix the setting of mode_skip_mask

In mode selection loop, once mode_index pass mode_skip_start, all
modes with a different reference frame from current best mode are
masked out using mode_skip_mask.

However, the setting of mode_skip_mask may use an invalid mode if
there is no mode tested yet. This commit fixes the issue by making
sure a mode has been tested and selected. Otherwise, no mode will be
masked out because of their reference frame.

Change-Id: Ib0009e8a96836a65cf5347440fff8a2e1a67f29f

--- a/vp9/encoder/vp9_rdopt.c

+++ b/vp9/encoder/vp9_rdopt.c

@@ -3304,7 +3304,7 @@

     // Look at the reference frame of the best mode so far and set the

     // skip mask to look at a subset of the remaining modes.

-    if (mode_index == mode_skip_start) {

+    if (mode_index == mode_skip_start && best_rd < INT64_MAX) {

       switch (vp9_mode_order[best_mode_index].ref_frame[0]) {

         case INTRA_FRAME:

           break;