[RESOLVED] OSX - Numpy float32 linear algebra bug

(Updated January 21, 2015 -- fix released)

Overview

We have identified a bug in Canopy Python in the interaction between Numpy, the Intel Math Kernel Library (MKL), and Apple's Accelerate library. The bug can affect Numpy's dot product when applied to float32 arrays.

This bug ONLY affects OS X users, and ONLY under specific conditions that are described below. However, if you are affected, the impact could be serious.

Numpy 1.8.1-2, released January 21, 2015, works around this bug. We recommend that all Canopy-64bit users on OS X, who use numpy, update immediately using the Canopy Package Manager. (Updating with enpkg numpy will also work but may re-install MKL unnecessarily because enpkg is not venv-aware).

 

Background (no need to read, now that fix has been released)

Test whether your system is vulnerable

We recommend that all OS X users who run numpy linear algebra code test their systems. Although we have only found the vulnerability on newer systems (some Mac OS 10.8, and all tested 10.9 and 10.10), we don't know whether the bug might exist on older systems after an Apple library update.

1) Canopy 32-bit installations are not vulnerable.

2) For Canopy 64-bit installations: open a Mac Terminal in which Canopy is the default Python. For most Canopy users, the easiest way to do this is to open a Canopy Terminal from the Canopy Tools menu.

4) Copy the following command exactly as shown, paste it into the Terminal, and press Enter to execute it:

python -c "import PySide;import numpy as np;f=np.ones(2,dtype=np.float32);print f.dot(f)"

4) If the result is 2.0, your system is not vulnerable and you do not need to read further.

If the result is 0.0, your system is vulnerable and could be affected under specific conditions.

Evaluate whether you are affected

Most OS X users will not be affected in practice, even if your system is vulnerable.

You will only be affected if all of the following are true:

  • You are running Canopy 64-bit.
  • The above vulnerability test gives the wrong answer (0.0).
  • Your program or its dependent packages use numpy to calculate the dot product of float32 arrays. (Note that on these systems, float64 is the default float type, so float32 would only be used when explicitly specified.)
  • Apple's Accelerate library is being loaded and used by Numpy. In our investigations with Canopy's numpy (which is MKL-linked), this only happens if your program imports a graphics library or GUI backend (e.g. PySide, PyQt, or wx) before it imports numpy. Importing matplotlib is ok because matplotlib imports numpy before it imports any graphics libraries.

    However, note that this problem sequence can also happen invisibly due to your program's running environment, i.e. not as a result of your own source code:
    • It will occur when Canopy was started from an icon or Spotlight (rather than Terminal), and your program is running in the Canopy Python shell. (The IPython notebook in Canopy is ok.)
    • It will occur when IPython was started from Mac Terminal with the --gui <backend> option. The most common values for <backend> are qt and wx. (Note that ipython --pylab <backend> and ipython --matplotlib <backend> are safe no matter what your own code does, because these IPython modes import Numpy first, during startup.)

Workarounds

The following workarounds are implicit in the "affected" criteria listed above.

1) If you are running your affected code from Canopy's Python pane, you must start Canopy not from an icon or Spotlight, but instead from Terminal, for example with one of the following commands:

open -a Canopy

or

open -a /Applications/Canopy.app

You can then verify that the problem no longer exists in Canopy's Python pane by confirming that the following command prints 2.0:

import PySide;import numpy as np;f=np.ones(2,dtype=np.float32);print f.dot(f)

2) If you are running your affected code from IPython in Terminal, start IPython with --matplotlib <backend> rather than --gui <backend>. (The most common values for <backend> are qt and wx.) Verify that the problem no longer exists, as just described.

3) If you are running your affected code in Terminal using plain Python, import numpy at the very start of your main module.

 

 

 

Have more questions? Submit a request

Comments

Powered by Zendesk