====== 2011.07.07 - PyErr_Print() ======

[[wp>Python_(programming_language)|Python]] is known to cope well with [[wp>C_(programming_language)|C]] via [[http://docs.python.org/c-api|PythonC]]. thanks to [[http://www.boost.org/doc/libs/release/libs/python/doc|boost::python]] it integrates well with [[wp>C++]] as well.

this is theory. and we all know that in theory there is no difference between theory and practice but in practice there is.

===== how it started =====

it is not that bad actually, as long as you call C++ code from the inside of the Pyhton script. recently my need was to embed Python code to the C++ sources. while coding new component for [[http://www.acarm.wcss.wroc.pl|ACARM-ng]] project, that is going to be a code base for user plugins, written in Python, instead of C++ i started using boost::python to make things simpler((PythonC looks awful to me...)). then few new, interesting things came up. when my "hello (python) world" (a sort of prototype; PoC-alike code) started to work as i expected i created new component (named PythonAPI) and straight away came across an issue. how to solve exceptions thrown in Python code? i mean both user-generated and syntax error, since both of them may (and will) appear in real-life code, written by an end user.

how it worked in PoC? take a look at the following (simplified) example:

<code c++>
  try
  {
    PyImport_AppendInittab("mymod", PyInit_mymod);
    Py_Initialize();
    object mainModule=import("__main__");
    object mainNamespace=mainModule.attr("__dict__");
    exec_file("some_script.py", mainNamespace);
  }
  catch(const boost::python::error_already_set &)
  {
    PyErr_Print();
    return 42;
  }
</code>

when error is risen it is reported in C++ code as an exception of type //boost::python::error_already_set//, which does not mean anything more than "exception has been thrown in Python". the funny thing is that it does not have any information on exception details. looks shitty straight away, but ok - let's see how to extract error report.

PythonC has ready-to-use call to print error message directly on the screen, namely //PyErr_Print()//. nice feature - good for testing. used in my PoC as well. but how to do this in final code? ACARM-ng is a daemon by-design, thus writing anything on the screen does not make any sense. logging mechanism has to be used for that. but how the hell get error string?!


===== the (re)search =====

i've done a lot of goolging, searching forums, reading APIs, etc... this question/problem appears to be quite common on the net, though usable solution is hard to find, not mentioning - to self develop it, when you're new to both APIs having only basic knowledge and experience with Python.

while looking the net i came mostly across not answered topics and "interesting" solutions/code-monsters like [[http://objectmix.com/python/317969-how-get-string-printed-pyerr_print.html|replacing error stream with a string and reading from it]]! not only that i have to keep WHOLE stderr in some string (what for?! beside script author may want to use printf-like debugging, that i may interfere with this way) and then "guess" where is my error message. the solution is merely an UGLY HACK.

i was so desperate it was thinking about ripping //PyErr_Print()// call from the APIs sources and switching low-level calls that print on the screen. i even though about taking user script into one, big try-catch and saving error as some "magic" variable (imagine the state of mind when such a concepts can arose!).

luckily for me my holidays time had come and i could leave this topic for a while...


===== solution =====

few days ago i came back to work and after doing all tasks that have gathered over a time, i came back to the topic. this time i was far more lucky. i found excellent blog entry on this issue. [[http://thejosephturner.com/blog|Joseph Turner]] gave the working code example of possible solution in his [[http://thejosephturner.com/blog/2011/06/15/embedding-python-in-c-applications-with-boostpython-part-2|embedding Python in C++ with boost::python, part 2]] article.

this PoC of exception data extraction worked as a charm! what i did was to wrap this in a class and add some common functionalities, that are useful in this context. current (not final though) implementation is now part of [[http://www.acarm.wcss.wroc.pl|ACARM-ng's]] code, namely the [[http://repo.acarm.wcss.wroc.pl/main|development branch]], as GPLv2 sources. as of writing these words it is only available through [[http://repo.acarm.wcss.wroc.pl/devel/basz|my development branch]], since whole module is not done (i.e. fully functional) yet.

the class' draft looks like this

<code c++>
#include <string>
#include <stdexcept>
#include <boost/python.hpp>

using namespace std;
namespace py=boost::python;

class ExceptionHandle
{
public:
  ExceptionHandle(void):
    type_("???"),
    msg_("???"),
    backtrace_("???")
  {
    // maby there was no exception at all?
    if( PyErr_Occurred()==NULL )
      throw std::runtime_error("no exception reported by Python's interpreter");
 
    // read exception-related data from python
    PyObject *typePtr     =NULL;
    PyObject *valuePtr    =NULL;
    PyObject *backtracePtr=NULL;
    PyErr_Fetch(&typePtr, &valuePtr, &backtracePtr);
 
    // check if type is set
    if(typePtr!=NULL)
    {
      py::handle<>        h(typePtr);
      py::str             pyStr(h);
      py::extract<string> extrPyStr(pyStr);
      if( extrPyStr.check() )
        type_=extrPyStr();
    }
    // check if message is set
    if(valuePtr!=NULL)
    {
      py::handle<>        h(valuePtr);
      py::str             pyStr(h);
      py::extract<string> extrPyStr(pyStr);
      if( extrPyStr.check() )
        msg_=extrPyStr();
    }
    // check if backtrace is set
    if(backtracePtr!=NULL)
    {
      py::handle<>        h(backtracePtr);
      py::object          tb( py::import("traceback") );
      py::object          formatTb( tb.attr("format_tb") );
      py::object          tbList( formatTb(h) );
      py::object          tbStr( py::str("\n").join(tbList) );
      py::extract<string> extrPyStr(tbStr);
      if( extrPyStr.check() )
        backtrace_=extrPyStr();
    }
 
    // ok - this one has been read
    clearState();
  }
 
  void rethrow(void) const
  {
    throw std::runtime_error( str() );
  }
 
  std::string str(void) const
  {
    return getType()+": "+getMessage()+"; backtrace is:\n"+getBacktraceStr();
  }
 
  const std::string &getType(void) const
  {
    return type_;
  }
  const std::string &getMessage(void) const
  {
    return msg_;
  }
  // NOTE: backtrace should be also available as an vector of stack entries
  const std::string &getBacktraceStr(void) const
  {
    return backtrace_;
  }
 
  static void clearState(void)
  {
    PyErr_Clear();
  }
 
private:
  std::string type_;
  std::string msg_;
  std::string backtrace_;
}; // class ExceptionHandle
</code>

compilable sources, along with all dependencies can be downloaded from [[http://repo.acarm.wcss.wroc.pl/devel/basz/src/pythonapi/PythonAPI|PythonAPI on my development branch]]((will soon be available on main branch and next ACARM-ng's release)).

hope this will save you hours and nerves.