C++/Python: Import Eigen output to NumPy

At work our team runs Monte Carlo simulations of rather high-dimensional (900 complex dimensions) integrals over spaces of matrices. The actual simulation code running on the cluster is written in C++ and uses the Eigen library for various internal constructions. Data analysis is done in Python using Pandas and NumPy.

Bridging the two languages is easy since the C++ code outputs the simulation results in the HDF5 format, which is well supported by various Python libraries, including Pandas.

When we develop new features for the simulation, however, it is useful to debug by testing simple cases and sending the output directly to the command line. This brings me to a minor impracticality: the Eigen output is not directly readable by NumPy, so the debug output cannot easily be inspected using the existing analysis code.

A very simple conversion script in Python saves the day. It reads the Eigen output of a complex matrix (Eigen::MatrixXcd), copied or written directly to an intermediate file, and returns the resulting NumPy array.

import numpy as np

# std::complex is printed by default as (Re,Im)
convert = (lambda x: float(x[1:-1].split(b',')[0]) +
          float(x[1:-1].split(b',')[1])*1j)

conv = {}  

def read_as_ndarray(filename):
    # get number of columns
    with open(filename) as f:
        columns = len(next(f).split())
    # tell numpy to convert all columns
    for i in range(columns):
        conv[i] = convert
    n = np.genfromtxt(filename, dtype = complex,
                      converters = conv, autostrip = True)
    return n