shithub: aubio

Download patch

ref: b1ed6ce4dd48b7328ecba01c73010a5ea7d12c1c
parent: 982629c87d47921cb85a562fbd26cd9e3696689a
parent: 2244f004403844e28440c956a3a073691845a6d0
author: Paul Brossier <piem@piem.org>
date: Sat Jun 29 10:45:48 EDT 2019 

Merge branch 'master' into feature/quietmode


--- a/.landscape.yml
+++ /dev/null
@@ -1,5 +1,0 @@
-strictness: medium
-test-warnings: true
-python-targets:
-  - 2
-  - 3
--- a/.travis.yml
+++ b/.travis.yml
@@ -38,15 +38,15 @@
     - language: C
       os: osx
       compiler: clang
-      env: WAFOPTS="--enable-fat --disable-avcodec --disable-sndfile"
+      env: WAFOPTS="--enable-fat --disable-avcodec --disable-sndfile --disable-samplerate"
     - language: C
       os: osx
       compiler: clang
-      env: WAFOPTS="--with-target-platform=ios --disable-avcodec --disable-sndfile" AUBIO_NOTESTS=1
+      env: WAFOPTS="--with-target-platform=ios --disable-avcodec --disable-sndfile --disable-samplerate" AUBIO_NOTESTS=1
     - language: C
       os: osx
       compiler: clang
-      env: WAFOPTS="--with-target-platform=iosimulator --disable-avcodec --disable-sndfile" AUBIO_NOTESTS=1
+      env: WAFOPTS="--with-target-platform=iosimulator --disable-avcodec --disable-sndfile --disable-samplerate" AUBIO_NOTESTS=1
 
 # use trusty
 dist: trusty
@@ -73,7 +73,7 @@
     - ffmpeg
     - libsndfile
     - lcov
-    #update: true
+    update: true
 
 before_install:
    - |
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,83 @@
+2018-12-19 Paul Brossier <piem@aubio.org>
+
+	[ Overview ]
+
+	* VERSION: bump to 0.4.9
+	* library: improve stability, fixing potential crashes and memory leaks on
+	invalid arguments; improve library messages and reporting of system errors
+	* tests/: major clean-up, check return codes, increase code coverage
+	* python/tests/: switch to pytest (closes gh-163), check emitted warnings
+	* python/: add pages to manual with brief descriptions of classes
+
+	[ Fixes ]
+
+	* security: improve arguments validation in new_aubio_filterbank (prevent
+	possible null-pointer dereference on invalid n_filters, CVE-2018-19801),
+	new_aubio-tempo (prevent possible buffer overflow, CVE-2018-19800), and
+	new_aubio_onset (prevent null-pointer dereference, CVE-2018-19802). Thanks
+	to Guoxiang Niu (@niugx), from the EaglEye Team for reporting these issues.
+	* tempo: fix delay_ms methods
+	* filterbank: fix aubio_filterbank_get_power (thanks to @romanbsd who
+	also noticed this issue)
+	* dct: creation fail on negative sizes or invalid accelerate radix,
+	fix typo in error and warning messages, prevent possible memory leak
+	* pitch: prevent null pointer dereference in yinfast, comment out unused
+	functions in mcomb and yin, prevent possible leak in specacf
+	* mfcc: always use dct module, strengthen input validation, change
+	get_{scale,power} to return smpl_t
+	* specdesc: improve error message
+	* notes: prevent null pointer dereference
+	* hist: add validation for size argument, prevent possible leak
+	* awhitening: use shortest length available (closes gh-216)
+	* io: add macros to display system errors, add helpers to validate input
+	arguments of source and sink methods, always clean-up after failure
+	* source: validate input sizes to prevent invalid reads
+	* apple_audio: use native format conversions in source and sink, prevent
+	possible apple_audio crash on empty string, get_duration returns 0 on failure
+	* ffmpeg/avcodec: prevent deprecation warnings, read after close, and skipped
+	samples warnings, improve warning messages, only show a warning when
+	swr_convert failed, prevent possible memory leak when closing swr context
+	* wavwrite: copy to all channels if needed, check fseek and fwrite return
+	values, call fflush in open to return failure on full disk-system
+	* source_sndfile: fix reading sizes when resampling, set error message when
+	reading after close
+	* aubio_priv.h: include blas first (see gh-225), add STRERROR macros
+
+	[ Python ]
+
+	* documentation: add pages to manual, add minimal docstrings for fft,
+	digital_filter, and generated objects, improve specdesc documentation
+	* filterbank: add get_norm/power documentation
+	* source: take a copy of the last frame before resizing it, raise an
+	exception when read failed, fix compilation warning
+	* fixes: remove unneeded check convert with PyFloat_FromDouble or
+	PyFloat_FromDouble, check if sink, digital_filter, were created before
+	deleting
+
+	[ Tests ]
+
+	* python/tests/: switch to pytest (slightly slower than nose2 but better at
+	capturing warnings and parametrization), improve coding style and coverage.
+	Tests should now be run with `pytest`.
+	* tests/: Each test program in C must now return 0, otherwise the test will
+	fail. Examples have been modified to run themselves on a test audio file,
+	but can still be run with arguments. Tests for `source` and `sink` have been
+	factorised, and some code cleaning. A python script is used to create a
+	test sound file. Tested on linux, macos, and windows, improvements to
+	test-mfcc (closes gh-219).
+
+	[ Build system ]
+
+	* waf: upgrade to 2.0.14, check the return code of each test program,
+	update rules to build manual and api documentation into build/, check
+	for errno.h
+	* osx: use -Os in scripts/build_apple_frameworks
+	* Makefile: improve coverage reports
+	* appveyor, travis, circleci: switch to pytest, set one travis config to use
+	sndfile only
+	* travis: add py3.6, drop py3.4, use py3.5 to test debug mode
+	* azure: add basic configuration
+
 2018-11-21 Paul Brossier <piem@aubio.org>
 
 	[ Overview ]
--- a/README.md
+++ b/README.md
@@ -3,7 +3,6 @@
 
 [![Travis build status](https://travis-ci.org/aubio/aubio.svg?branch=master)](https://travis-ci.org/aubio/aubio "Travis build status")
 [![Appveyor build status](https://img.shields.io/appveyor/ci/piem/aubio/master.svg)](https://ci.appveyor.com/project/piem/aubio "Appveyor build status")
-[![Landscape code health](https://landscape.io/github/aubio/aubio/master/landscape.svg?style=flat)](https://landscape.io/github/aubio/aubio/master "Landscape code health")
 [![Commits since last release](https://img.shields.io/github/commits-since/aubio/aubio/latest.svg)](https://github.com/aubio/aubio "Commits since last release")
 
 [![Documentation](https://readthedocs.org/projects/aubio/badge/?version=latest)](http://aubio.readthedocs.io/en/latest/?badge=latest "Latest documentation")
--- a/VERSION
+++ b/VERSION
@@ -1,6 +1,6 @@
 AUBIO_MAJOR_VERSION=0
-AUBIO_MINOR_VERSION=4
-AUBIO_PATCH_VERSION=9
+AUBIO_MINOR_VERSION=5
+AUBIO_PATCH_VERSION=0
 AUBIO_VERSION_STATUS='~alpha'
 LIBAUBIO_LT_CUR=5
 LIBAUBIO_LT_REV=4
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -3,7 +3,7 @@
 
 - job: linux
   pool:
-    vmImage: 'Ubuntu 16.04'
+    vmImage: 'ubuntu-16.04'
   steps:
   - script: |
       make
@@ -13,7 +13,7 @@
 
 - job: windows
   pool:
-    vmIMage: 'VS2017-Win2016'
+    vmImage: 'vs2017-win2016'
   steps:
   - script: |
       make
@@ -24,7 +24,7 @@
 
 - job: macos
   pool:
-    vmIMage: macOS-10.13
+    vmImage: 'macos-10.13'
   steps:
   - script: |
       brew update
--- a/doc/binaries.rst
+++ b/doc/binaries.rst
@@ -8,5 +8,8 @@
 and
 `windows <https://aubio.org/download#win>`_
 
+For Windows, aubio is also available from `vcpkg
+<https://vcpkg.readthedocs.io/en/latest/examples/installing-and-using-packages/>`_.
+
 To use aubio in a macOS or iOS application, see :ref:`xcode-frameworks-label`.
 
--- a/doc/statuslinks.rst
+++ b/doc/statuslinks.rst
@@ -9,10 +9,6 @@
    :target: https://ci.appveyor.com/project/piem/aubio/
    :alt: Appveyor build status
 
-.. image:: https://landscape.io/github/aubio/aubio/master/landscape.svg?style=flat
-   :target: https://landscape.io/github/aubio/aubio/master
-   :alt: Landscape code health
-
 .. image:: https://readthedocs.org/projects/aubio/badge/?version=latest
    :target: https://aubio.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation status
--- a/python/demos/demo_wav2midi.py
+++ b/python/demos/demo_wav2midi.py
@@ -63,7 +63,7 @@
         delta = frames2tick(total_frames) - last_time
         if new_note[2] > 0:
             track.append(Message('note_off', note=int(new_note[2]),
-                velocity=127, time=0)
+                velocity=127, time=delta)
                 )
         track.append(Message('note_on',
             note=int(new_note[0]),
--- a/python/ext/aubio-docstrings.h
+++ b/python/ext/aubio-docstrings.h
@@ -1,7 +1,7 @@
 #define PYAUBIO_dct_doc \
     "dct(size=1024)\n"\
     "\n"\
-    "Compute Discrete Fourier Transorms of Type-II.\n"\
+    "Compute Discrete Fourier Transforms of Type-II.\n"\
     "\n"\
     "Parameters\n"\
     "----------\n"\
--- a/python/ext/py-fft.c
+++ b/python/ext/py-fft.c
@@ -3,7 +3,7 @@
 static char Py_fft_doc[] = ""
 "fft(size=1024)\n"
 "\n"
-"Compute Fast Fourier Transorms.\n"
+"Compute Fast Fourier Transforms.\n"
 "\n"
 "Parameters\n"
 "----------\n"
--- a/python/ext/py-filter.c
+++ b/python/ext/py-filter.c
@@ -156,8 +156,18 @@
 
   err = aubio_filter_set_c_weighting (self->o, samplerate);
   if (err > 0) {
-    PyErr_SetString (PyExc_ValueError,
-        "error when setting filter to C-weighting");
+    if (PyErr_Occurred() == NULL) {
+      PyErr_SetString (PyExc_ValueError,
+          "error when setting filter to C-weighting");
+    } else {
+      // change the RuntimeError into ValueError
+      PyObject *type, *value, *traceback;
+      PyErr_Fetch(&type, &value, &traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
+    }
     return NULL;
   }
   Py_RETURN_NONE;
@@ -174,8 +184,18 @@
 
   err = aubio_filter_set_a_weighting (self->o, samplerate);
   if (err > 0) {
-    PyErr_SetString (PyExc_ValueError,
-        "error when setting filter to A-weighting");
+    if (PyErr_Occurred() == NULL) {
+      PyErr_SetString (PyExc_ValueError,
+          "error when setting filter to A-weighting");
+    } else {
+      // change the RuntimeError into ValueError
+      PyObject *type, *value, *traceback;
+      PyErr_Fetch(&type, &value, &traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
+    }
     return NULL;
   }
   Py_RETURN_NONE;
@@ -192,8 +212,18 @@
 
   err = aubio_filter_set_biquad (self->o, b0, b1, b2, a1, a2);
   if (err > 0) {
-    PyErr_SetString (PyExc_ValueError,
-        "error when setting filter with biquad coefficients");
+    if (PyErr_Occurred() == NULL) {
+      PyErr_SetString (PyExc_ValueError,
+          "error when setting filter with biquad coefficients");
+    } else {
+      // change the RuntimeError into ValueError
+      PyObject *type, *value, *traceback;
+      PyErr_Fetch(&type, &value, &traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
+    }
     return NULL;
   }
   Py_RETURN_NONE;
--- a/python/ext/py-filterbank.c
+++ b/python/ext/py-filterbank.c
@@ -326,7 +326,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }
     return NULL;
   }
@@ -351,7 +354,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }
     return NULL;
   }
@@ -380,7 +386,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }
     return NULL;
   }
@@ -409,7 +418,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }
     return NULL;
   }
@@ -463,7 +475,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }
     return NULL;
   }
@@ -493,7 +508,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }
     return NULL;
   }
--- a/python/ext/py-sink.c
+++ b/python/ext/py-sink.c
@@ -81,7 +81,7 @@
 "Close this sink now.\n"
 "\n"
 "By default, the sink will be closed before being deleted.\n"
-"Explicitely closing a sink can be useful to control the number\n"
+"Explicitly closing a sink can be useful to control the number\n"
 "of files simultaneously opened.\n"
 "";
 
--- a/python/lib/gen_code.py
+++ b/python/lib/gen_code.py
@@ -509,7 +509,10 @@
       // change the RuntimeError into ValueError
       PyObject *type, *value, *traceback;
       PyErr_Fetch(&type, &value, &traceback);
-      PyErr_Restore(PyExc_ValueError, value, traceback);
+      Py_XDECREF(type);
+      type = PyExc_ValueError;
+      Py_XINCREF(type);
+      PyErr_Restore(type, value, traceback);
     }}
     return NULL;
   }}
--- a/python/lib/gen_external.py
+++ b/python/lib/gen_external.py
@@ -120,23 +120,24 @@
     print("Running command: {:s}".format(" ".join(cpp_cmd)))
     proc = subprocess.Popen(cpp_cmd,
                             stderr=subprocess.PIPE,
-                            stdout=subprocess.PIPE)
+                            stdout=subprocess.PIPE,
+                            universal_newlines=True)
     assert proc, 'Proc was none'
     cpp_output = proc.stdout.read()
     err_output = proc.stderr.read()
     if err_output:
         print("Warning: preprocessor produced errors or warnings:\n%s" \
-                % err_output.decode('utf8'))
+                % err_output)
     if not cpp_output:
         raise_msg = "preprocessor output is empty! Running command " \
                 + "\"%s\" failed" % " ".join(cpp_cmd)
         if err_output:
-            raise_msg += " with stderr: \"%s\"" % err_output.decode('utf8')
+            raise_msg += " with stderr: \"%s\"" % err_output
         else:
             raise_msg += " with no stdout or stderr"
         raise Exception(raise_msg)
     if not isinstance(cpp_output, list):
-        cpp_output = [l.strip() for l in cpp_output.decode('utf8').split('\n')]
+        cpp_output = [l.strip() for l in cpp_output.split('\n')]
 
     return cpp_output
 
--- a/python/tests/test_hztomel.py
+++ b/python/tests/test_hztomel.py
@@ -4,6 +4,7 @@
 from numpy.testing import TestCase
 from numpy.testing import assert_equal, assert_almost_equal
 from _tools import assert_warns
+from utils import is32bit
 import numpy as np
 import aubio
 
@@ -10,7 +11,6 @@
 from aubio import hztomel, meltohz
 from aubio import hztomel_htk, meltohz_htk
 
-
 class aubio_hztomel_test_case(TestCase):
 
     def test_hztomel(self):
@@ -17,10 +17,15 @@
         assert_equal(hztomel(0.), 0.)
         assert_almost_equal(hztomel(400. / 3.), 2., decimal=5)
         assert_almost_equal(hztomel(1000. / 3), 5.)
-        assert_equal(hztomel(200.), 3.)
+        # on 32bit, some of these tests fails unless compiling with -ffloat-store
+        try:
+            assert_equal(hztomel(200.), 3.)
+        except AssertionError:
+            if not is32bit(): raise
+            assert_almost_equal(hztomel(200.), 3., decimal=5)
         assert_almost_equal(hztomel(1000.), 15)
-        assert_almost_equal(hztomel(6400), 42)
-        assert_almost_equal(hztomel(40960), 69)
+        assert_almost_equal(hztomel(6400), 42, decimal=5)
+        assert_almost_equal(hztomel(40960), 69, decimal=5)
 
         for m in np.linspace(0, 1000, 100):
             assert_almost_equal(hztomel(meltohz(m)) - m, 0, decimal=3)
@@ -28,7 +33,11 @@
     def test_meltohz(self):
         assert_equal(meltohz(0.), 0.)
         assert_almost_equal(meltohz(2), 400. / 3., decimal=4)
-        assert_equal(meltohz(3.), 200.)
+        try:
+            assert_equal(meltohz(3.), 200.)
+        except AssertionError:
+            if not is32bit(): raise
+            assert_almost_equal(meltohz(3.), 200., decimal=5)
         assert_almost_equal(meltohz(5), 1000. / 3., decimal=4)
         assert_almost_equal(meltohz(15), 1000., decimal=4)
         assert_almost_equal(meltohz(42), 6400., decimal=2)
--- a/python/tests/test_phasevoc.py
+++ b/python/tests/test_phasevoc.py
@@ -1,7 +1,7 @@
 #! /usr/bin/env python
 
 from numpy.testing import TestCase, assert_equal, assert_array_less
-from _tools import parametrize
+from _tools import parametrize, skipTest
 from aubio import fvec, cvec, pvoc, float_type
 import numpy as np
 
@@ -51,7 +51,7 @@
                 assert_equal (s.phas[s.phas > 0], +np.pi)
                 assert_equal (s.phas[s.phas < 0], -np.pi)
                 assert_equal (np.abs(s.phas[np.abs(s.phas) != np.pi]), 0)
-                self.skipTest('pvoc(fvec(%d)).phas != +0, ' % win_s \
+                skipTest('pvoc(fvec(%d)).phas != +0, ' % win_s \
                         + 'This is expected when using fftw3 on powerpc.')
             assert_equal ( r, 0.)
 
--- a/python/tests/utils.py
+++ b/python/tests/utils.py
@@ -3,10 +3,14 @@
 import os
 import re
 import glob
+import struct
 import numpy as np
 from tempfile import mkstemp
 
 DEFAULT_SOUND = '22050Hz_5s_brownnoise.wav'
+
+def is32bit():
+    return struct.calcsize("P") * 8 == 32
 
 def array_from_text_file(filename, dtype = 'float'):
     realpathname = os.path.join(os.path.dirname(__file__), filename)
--- a/scripts/get_waf.sh
+++ b/scripts/get_waf.sh
@@ -3,7 +3,7 @@
 set -e
 #set -x
 
-WAFVERSION=2.0.13
+WAFVERSION=2.0.17
 WAFTARBALL=waf-$WAFVERSION.tar.bz2
 WAFURL=https://waf.io/$WAFTARBALL
 WAFUPSTREAMKEY=https://gitlab.com/ita1024/waf/raw/master/utils/pubkey.asc
--- a/tests/src/spectral/test-mfcc.c
+++ b/tests/src/spectral/test-mfcc.c
@@ -33,9 +33,9 @@
   aubio_pvoc_t *pv = 0;
   aubio_mfcc_t *mfcc = 0;
 
-  fvec_t *in = new_fvec (win_s); // input buffer
-  cvec_t *fftgrain = new_cvec (win_s); // input buffer
-  fvec_t *out = new_fvec (n_coeffs); // output coefficients
+  fvec_t *in = new_fvec (hop_s);       // phase vocoder input
+  cvec_t *fftgrain = new_cvec (win_s); // pvoc output / mfcc input
+  fvec_t *out = new_fvec (n_coeffs);   // mfcc output
 
   if (!in || !fftgrain || !out) { err = 1; goto failure; }
 
--- a/wscript
+++ b/wscript
@@ -630,7 +630,7 @@
     ctx.excl += ' **/.pytest_cache'
     ctx.excl += ' **/.cache'
     ctx.excl += ' **/**.zip **/**.tar.bz2'
-    ctx.excl += ' **.tar.bz2'
+    ctx.excl += ' **.tar.bz2**'
     ctx.excl += ' **/doc/full/* **/doc/web/*'
     ctx.excl += ' **/doc/full.cfg'
     ctx.excl += ' **/python/*.db'
@@ -642,7 +642,6 @@
     ctx.excl += ' **/dist*'
     ctx.excl += ' **/.DS_Store'
     ctx.excl += ' **/.travis.yml'
-    ctx.excl += ' **/.landscape.yml'
     ctx.excl += ' **/.appveyor.yml'
     ctx.excl += ' **/.circleci/*'
     ctx.excl += ' **/azure-pipelines.yml'