From 9c71ce8a03f89818b2e0bc73571da42c6ebfe5e0 Mon Sep 17 00:00:00 2001 From: Jean-Paul Chaput Date: Sun, 26 Sep 2021 17:27:57 +0200 Subject: [PATCH] Updated doc of the new Python/C++ template wrapper. * Change: In Isobar3::PyWrapper, C++ exceptions where not catcheds, forgot to call exceptionWrapper(). * Change: In Isobar3, remove unused pyToC<>() flavor where T is a simple pointer and the argument is also a simple pointer, does not make sense. --- .../pages/python-cpp-new/Implementation.rst | 393 +++++++++++----- .../pages/python-cpp-new/Introduction.rst | 6 +- .../output/pages/python-cpp-new.html | 430 ++++++++++++------ documentation/output/pdfs/CheckToolkit.pdf | Bin 241137 -> 241150 bytes documentation/output/pdfs/PythonCpp.pdf | Bin 174275 -> 174246 bytes documentation/output/pdfs/PythonTutorial.pdf | Bin 298243 -> 298213 bytes documentation/output/pdfs/RDS.pdf | Bin 117041 -> 117030 bytes documentation/output/pdfs/Stratus.pdf | Bin 537948 -> 537784 bytes documentation/output/pdfs/UsersGuide.pdf | Bin 673192 -> 673209 bytes hurricane/src/configuration/PyTypeManager.cpp | 78 ++-- .../hurricane/configuration/PyTypeManager.h | 67 +-- 11 files changed, 621 insertions(+), 353 deletions(-) diff --git a/documentation/content/pages/python-cpp-new/Implementation.rst b/documentation/content/pages/python-cpp-new/Implementation.rst index 67fabb7e..e76319fa 100644 --- a/documentation/content/pages/python-cpp-new/Implementation.rst +++ b/documentation/content/pages/python-cpp-new/Implementation.rst @@ -10,47 +10,77 @@ the Python/C API but we provides templates to automate as much as possible the boring tasks (and code duplication). This way, if we need a very specific feature at some point, we can still revert back to the pure Python/C API. -The key features of our wrapper are: +The wrapper basically provides support for three kind of operations: + +1. Encapsulate C++ object *in* Python ones, done by ``cToPy<>()`` template. + Template specializations are defined for the ``POD`` and basic ``STL`` + types like ``std::string``. + + To add more easily new specializations, they resides in the top level + scope (**not** inside ``Isobar``). + +2. Decapsulate a C++ object *from* a Python one, done by ``pyToC()``. + It's implementation is slightly different from the one of ``cToPy<>()`` + in the sense that it is a mix of normal C++ functions and template + functions. I was having trouble inside the ``callMethod()`` to access + to templates specialization defined *after* that point, so function be + it. + + There are two mutually exclusives versions of the ``pyToC<>()`` for + objects managed through the type manager. One is for value types and + the other for pointer types. + +3. Wrap C/C++ functions & methods inside C-linkage ``PyCFunction``, that is + ``PyOject* (*)(PyObject*, PyObject*)``. This is done respectively through + ``callFunction<>()`` and ``callMethod<>()``. ``callMethod<>()`` having + two variants, one for directly calling the function member, if there is + no overload and one for calling one given flavor of the overloaded + function member. + + In addition we provides special variants for Python unary, binary and + operator functions. + +In addition to those user exposed features, we have: * The ``PyTypeManager`` and it's derived classes to store and share informations about all our newly defined ``PyTypeObjects``. -* The ``callMethod<>()`` and ``callFunction<>()`` function templates to - automatically generate a wrapper around C++ object methods or functions. - -* The ``PyBoject* objectLink( CppT* )`` functions templates that provides C++ to Python - object translation. - -* The ``bool pyAs( PyObject*, CppT*& )`` functions templates that provides Python to C++ - object translation. +* Various wrapper *classes* to wrap functions & methods. They are not directly + exposed because the template class intanciation needs the template parameters + to be explicitely given, wich is tedious. Instead we create them *through* + a function template call, which will perform for us the template type + deduction. We creates only two kind of ``PyObject`` (but many ``PyTypeObject``): -* ``PyVoidpointer`` which encapsulate one void pointer to the C++ associated +* ``PyOneVoid`` which encapsulate one void pointer to the C++ associated object. .. code-block:: Python extern "C" { - typedef struct PyVoidPointer { + typedef struct PyOneVoid { PyObject_HEAD - void* _object; + void* _object1; }; } -* ``PyIteratorPointer`` which encapsulate one void pointer to the C++ associated - iterator and one another to the ``PyObject`` of the container. +* ``PyTwoVoid`` which encapsulate one void pointer to the C++ associated + object (an iterator) and one another to the ``PyObject`` of the container. .. code-block:: Python extern "C" { - typedef struct PyVoidPointer { + typedef struct PyTwoVoid { PyObject_HEAD - void* _object; // C++ iterator. - PyVoidPointer* _container; // Python wrapped container. + void* _object1; // C++ iterator. + PyObject* _object2; // Python wrapped container. }; } + +.. note:: A ``PyTwoVoid`` can be casted/accessed as a ``PyOneVoid``. + 2.1 PyTypeManager ~~~~~~~~~~~~~~~~~ @@ -62,42 +92,41 @@ We creates only two kind of ``PyObject`` (but many ``PyTypeObject``): * Provide a non-template abstract base class for all the derived ``PyTypeObject``. As said, it is not a template class but it supplies function member - template. Derived classes are provides for different kind of C++ - class. + template. Derived classes are provided to manage different kind of C++ + classes. * :cb:`PyTypeManagerVTrunk` - Is an intermediate between the non-template base class and all the - templatized others. + templatized others (do **not** use it directly). * :cb:`PyTypeManagerNonDBo` - Template for standalone C++ classes that are not derived from ``DBo``. For example ``Box`` or ``Parameter``. * :cb:`PyTypeManagerDBo` - Template for C++ classes that *are* not derived from ``DBo``. For example ``Cell`` or ``Instance``. + * :cb:`PyTypeManagerDerivedDBo` + Template for a ``CppT`` class derived derived from a ``BaseT`` class. + ``CppT`` doesn'y need to be a direct derived of ``BaseT``. ``BaseT`` + needs to derive from ``DBo`` + * :cb:`PyTypeManagerVector`, template for C++ ``std::vector``. * :cb:`PyTypeManagerVectorIterator` - Template for C++ ``std::vector::iterator``, automatically created from the vector registration. * :cb:`PyTypeManagerMap`, template for C++ ``std::map``. * :cb:`PyTypeManagerMapIterator` - Template for C++ ``std::vector::iterator``, automatically created from the map registration. * :cb:`PyTypeManagerCollection<,CppT>`, template for C++ ``Hurricane::Collection``. * :cb:`PyTypeManagerCollectionIterator<,CppT>` - Template for C++ ``Hurricane::Locator``, automatically created from the collection registration. @@ -108,7 +137,7 @@ We creates only two kind of ``PyObject`` (but many ``PyTypeObject``): Functions of a ``PyTypeObject`` like the *tp* methods (``tp_alloc``, ``tp_print``, ``tp_hash``, ...) must have a C-linkage. So we create *one* function per slot that we want to use, set that *same* function for all the created ``PyTypeObject``, and -perform a dispact in it. The drawback is that for each access we have to perform +perform a dispacth in it. The drawback is that for each access we have to perform a map lookup. Hope it is fast. Excerpt from the code: @@ -123,7 +152,7 @@ Excerpt from the code: extern long _tpHash ( PyObject* self ) { // Dispatch towards the relevant class, based on ob_type pointer. - return PyTypeManager::get( self->ob_type )->_getTpHash( asVPtr(self) ); + return PyTypeManager::get( Py_TYPE(self)->_getTpHash( self ); } } @@ -132,7 +161,7 @@ Excerpt from the code: public: void PyTypeManager::_setupPyType () // Derived classes must implement it as they see fit. - virtual long _getTpHash ( PyVoidPointer* ) = 0; + virtual long _getTpHash ( PyObject* ) = 0; template static PyTypeManager* _get(); private: @@ -158,63 +187,101 @@ Excerpt from the code: 2.3 Going From Python to C++ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To convert a C++ object (pointer) into a Python object, a set of -:cb:`pyAs<>` templates functions are supplieds. +To convert a C++ object (pointer) into a Python object, a mix of +:cb:`pyToC<>()` templates functions and real functions are supplieds. + +Templates functions are used for all types/classes managed through +the ``PyTypeManger``. They come in two flavors: + +2. **Value as pointer version** (C++ argment type is ``T*``): + The encapsulated C++ object is still a pointer, + but to a *stand-alone* one which has been created for the sole + purpose of this ``PyObject``. Typical example is the ``Box``. + Then, we *copy by value* the contents of the pointed object into + the contents of the pointer argument that we where given. + +3. **Pointer version** (C++ argument type is ``T**``): + The encapsulated C++ object is truly a pointer + to an element of the data-structure, then we just extract the + C++ pointer value. + +Normal function overload are used for ``POD`` types (``bool``, ``int``, +``long``, ``double``, ...) and basic ``STL`` types (``std::string``, ...). Specialization for all POD type that can be directly translated into Python types must be provideds (``bool``, ``int``, ``long``, ``double``, ``std::string``, ...). -Partial specialization for type availables through the ``PyTypeManager`` -is supplied. It checks the manager for the type's existence. +Those templates/functions are the ones the ``Isobar::parse_objects()`` recursive +template function call in turn for each ``PyObject*`` argument. + +.. note:: ``Hurricane::Name`` are *not* exposed to the Python interface, they + must be treated as ``std::string``. .. code-block:: C++ - // Most generic template, should *not* be used. - // So issue an error message and report a failed conversion. - template - inline bool pyAs ( PyObject* pyArg, T& arg ) + // Template/Pointer to a value flavor. + template< typename T + , typename std::enable_if< !std::is_pointer::value, bool >::type = true > + inline bool pyToC ( PyObject* pyArg, T* arg ) { - std::cerr << "Isobar3.pyAs(): Unsupported type \"" - << demangle(typeid(T).name()) << "\"" << endl; - return false; + typedef typename std::remove_cv::type NonConstT; + Isobar3::PyTypeManager* manager = Isobar3::PyTypeManager::_get(); + if (not manager) { + std::cerr << "Isobar3::pyToC<>(const T*): Unsupported type." << std::endl; + return false; + } + if (Py_TYPE(pyArg) != manager->_getTypeObject()) return false; + *(const_cast< NonConstT* >(arg)) = *(( T* )( Isobar3::object1( pyArg ))); + return true; } - // Template specialization for POD type "int". - template<> - inline bool pyAs ( PyObject* pyArg, int& arg ) + // Template/Pointer to a pointer flavor. + template + inline bool pyToC ( PyObject* pyArg, T** arg ) { - if (PyInt_Check (pyArg)) { arg = PyInt_AsLong ( pyArg ); } - else if (PyLong_Check(pyArg)) { arg = PyLong_AsLong( pyArg ); } + Isobar3::PyTypeManager* manager = Isobar3::PyTypeManager::_get(); + if (not manager) { + std::cerr << "Isobar3::pyToC(T*&): Unsupported type \"" << typeid(T).name() << "\"" << std::endl; + return false; + } + *arg = (T*)( Isobar3::object1( pyArg )); + return true; + } + + // True function overload for std::string. + inline bool pyToC ( PyObject* pyArg, std::string* arg ) + { + if (not PyUnicode_Check(pyArg)) return false; + PyObject* pyBytes = PyUnicode_AsASCIIString( pyArg ); + *arg = PyBytes_AsString( pyBytes ); + Py_DECREF( pyBytes ); + return true; + } + + // True function overload for bool. + inline bool pyToC ( PyObject* pyArg, bool* arg ) + { + if (not PyBool_Check(pyArg)) return false; + *arg = (pyArg == Py_True); + return true; + } + + // True function overload for int. + inline bool pyToC ( PyObject* pyArg, int* arg ) + { + if (PyLong_Check(pyArg)) { *arg = PyLong_AsLong( pyArg ); } else return false; return true; } - // Template partial specialization for any type registered in - // the PyTypeManager. - template - inline bool pyAs ( PyObject* pyArg, T*& arg ) - { - PyTypeManager* manager = PyTypeManager::_get(); - if (not manager) { - std::cerr << "Isobar3.pyAs(): Unsupported type \"" - << typeid(T).name() << "\"" << endl; - return false; - } - arg = (T*)( asVPtr( pyArg )->_object ); - return true; - } - - template - inline bool pyAs ( PyVoidPointer* pyArg, T*& arg ) - { return pyAs( (PyObject*)pyArg, arg ); } 2.4 Going From C++ to Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To convert a Python object into a C++ object, a set of -:cb:`objectLink<>` templates functions are supplieds. +:cb:`cToPy<>` templates functions are supplieds. We completely disable the partially specialized templates for objects that are non-POD as the compiler seems to be unable to @@ -230,48 +297,65 @@ method. **why** we need the intermediate ``PyTypeManagerVTrunk`` template class. +.. note:: **Different C++ templates.** You may notice that the two following templates + may look like specializations of the same one: + + * ``template PyObject* cToPy ( CppT object )`` + * ``template PyObject* cToPy ( CppT* object )`` + + Which would be illegal (function templates are not allowed to have *partial* + specialization), but they are *not*. The two pairs + ``(template parameter,function parameter)``, that is ``(CppT,CppT)`` and + ``(CppT,CppT*)`` cannot be made to be a specialization of each other. + .. code-block:: C++ - - // The most generic template, should *not* be used. So raise a Python exception. + + // Generic template for values. template< typename CppT > - inline PyObject* objectLink ( CppT object ) + inline PyObject* cToPy ( CppT object ) { - std::string message = "Overload for Isobar3::objectLink< " - + demangle(typeid(CppT).name()) + " >() is missing."; - PyErr_SetString( HurricaneError, message.c_str() ); - return NULL; + if (not Isobar3::PyTypeManager::hasType()) { + std::string message = "Overload for Isobar3::cToPy< " + + Hurricane::demangle(typeid(CppT).name()) + " >() Type not registered in the manager."; + PyErr_SetString( Isobar3::HurricaneError, message.c_str() ); + return NULL; + } + return Isobar3::PyTypeManager::link( new CppT (object) ); } - - // Disable this template if "CppT" is an already supported type. + + // Disabled for POD & STL types, pointer flavor. template< typename CppT , typename std::enable_if< !std::is_same::value && !std::is_same::value && !std::is_same::value - && !std::is_same::value - ,bool>::type = true > - inline PyObject* objectLink ( CppT* object ) - { return PyTypeManager::link( object ); } - - // Disable this template if "CppT" is an already supported type. + && !std::is_same::value,bool>::type = true > + inline PyObject* cToPy ( CppT* object ) + { return Isobar3::PyTypeManager::link( object ); } + + // Disabled for POD & STL types, const pointer flavor. template< typename CppT , typename std::enable_if< !std::is_same::value && !std::is_same::value && !std::is_same::value - && !std::is_same::value - ,bool>::type = true > - inline PyObject* objectLink ( const CppT* object ) - { return PyTypeManager::link( const_cast( object )); } - - // Specializations for POD type "int" . - template<> inline PyObject* objectLink ( int i ) - { return PyInt_FromLong( i ); } - - template<> inline PyObject* objectLink ( const int i ) - { return PyInt_FromLong( i ); } - - template<> inline PyObject* objectLink ( const int* i ) - { return PyInt_FromLong( *i ); } + && !std::is_same::value,bool>::type = true > + inline PyObject* cToPy ( const CppT* object ) + { return Isobar3::PyTypeManager::link( const_cast( object )); } + + // Specialization for booleans. + template<> + inline PyObject* cToPy ( bool b ) + { if (b) Py_RETURN_TRUE; Py_RETURN_FALSE; } + + // Specialization for STL std::string. + template<> inline PyObject* cToPy< std::string> ( std::string s ) { return PyUnicode_FromString( s.c_str() ); } + template<> inline PyObject* cToPy ( const std::string s ) { return PyUnicode_FromString( s.c_str() ); } + template<> inline PyObject* cToPy ( const std::string* s ) { return PyUnicode_FromString( s->c_str() ); } + + // Specialization for POD int. + template<> inline PyObject* cToPy< int > ( int i ) { return PyLong_FromLong( i ); } + template<> inline PyObject* cToPy ( const int i ) { return PyLong_FromLong( i ); } + template<> inline PyObject* cToPy ( const int* i ) { return PyLong_FromLong( *i ); } 2.5 Object Methods Wrappers @@ -309,8 +393,8 @@ Basically, we want to fit: to a sequence of ``PyObject*`` (that must match the number of arguments of it's C++ function conterpart). -So, the problem is to change a tuple which size is only kown at *runtime* -into a list of C/C++ parameters known at *compile time*. +So, the problem is to change a Python tuple which size is only kown at +*runtime* into a list of C/C++ parameters known at *compile time*. I am not such an expert in template programming so I can find a *generic* solution able to handle any number of parameters. Instead I did write @@ -320,7 +404,22 @@ to the C++ function call and the duplicated code needed for each template is kept to a minimum. To translate the Python tuple into an ordered list (vector like) of C++ -object *of different types*, the obvious choice is C++ ``std::tuple<>``. +object *of different types*, the obvious choice should have been ``std::tuple<>``, +but I did encouter problems when the functions signature did contains +references. So to manage that I did implement: + +* A ``BaseArg`` class and it's template derived ``Arg`` to hold + one value of a type (more or less like ``std::any<>``). + The type of the value attribute of ``Arg`` is ``T`` *stripped* + from reference and constness. This internal type is accessible + through ``Arg::ValueT``. + +* A template list of arguments ``Args`` analogous to + ``std::tuple<>`` which holds a table of ``BaseArg`` to convert all the + arguments. + +* A recursive template converter function ``parse_pyobjects<>``, which is + called through the ``Args<>::parse()`` function. Another challenge is the return type. I distinguish three flavor of return type: @@ -332,50 +431,96 @@ return type: To uniformize the return type we create four templates ``_callMethodReturn<>()`` that takes whatever the C++ return type is, and turn it into a ``PyObject*``. -Except for the functions returning ``void``, we call ``objectLink<>()`` to -wrap the value. +Except for the functions returning ``void``, we call ``cToPy<>()`` to +wrap the value. Given the return type of the method, only one template +will match. But as functions template do not allow partial specialization, +only one must be defined for that method (the one *matching* it's +return type), so we make the template mutually exclusives based on +the ``TR`` type (with the ``std::enable_if<>`` clause). + + +.. note:: In the various ``_callMethodReturn<>`` we have *two* sets for the + method parameters types : ``TArgsF...`` and ``TArgsW...``. This is to + allow a wider range of matching in the template as the type of the + arguments of the method (``TArgsF...``) may not *exactly* matches the + one passed by the wrapper (``TArgsW...``), typically the method has + a ``const`` parameter which is non-``const`` in the wrapper. Here is an excerpt of the code: .. code-block:: C++ - + + // Flavor for "return by value" (seems to match std::is_object<>) + template< typename TC, typename TR, typename... TArgsF, typename... TArgsW + , typename std::enable_if< !std::is_reference::value + && !std::is_pointer ::value + && !std::is_void ::value,bool>::type = true > + inline PyObject* _callMethodReturn ( TR(TC::* method)(TArgsF...), TC* cppObject, TArgsW... args ) + { + TR value = (cppObject->*method)( args... ); + return cToPy( value ); + } + + // Flavor for "return by reference" + template< typename TC, typename TR, typename... TArgsF, typename... TArgsW + , typename std::enable_if< std::is_reference::value + && !std::is_pointer ::value + && !std::is_void ::value,bool>::type = true > + inline PyObject* _callMethodReturn ( TR(TC::* method)(TArgsF...), TC* cppObject, TArgsW... args ) + { + TR rvalue = (cppObject->*method)( args... ); + return cToPy( rvalue ); + } + // Flavor for "return by pointer". - template< typename TC, typename TR, typename... TArgs + template< typename TC, typename TR, typename... TArgsF, typename... TArgsW , typename std::enable_if::value,bool>::type = true > - inline PyObject* _callMethodReturn ( TR(TC::* method)(TArgs...), TC* cppObject, TArgs... args ) + inline PyObject* _callMethodReturn ( TR(TC::* method)(TArgsF...), TC* cppObject, TArgsW... args ) { TR pvalue = (cppObject->*method)( args... ); - return objectLink( pvalue ); + return cToPy( pvalue ); } - + // Flavor for "return void". - template< typename TC, typename TR, typename... TArgs + template< typename TC, typename TR, typename... TArgsF, typename... TArgsW , typename std::enable_if::value,bool>::type = true > - inline PyObject* _callMethodReturn ( TR(TC::* method)(TArgs...), TC* cppObject, TArgs... args ) + inline PyObject* _callMethodReturn ( TR(TC::* method)(TArgsF...), TC* cppObject, TArgsW... args ) { (cppObject->*method)( args... ); Py_RETURN_NONE; } - - // Function without argument. + + // Make the translation call for a method without arguments. template< typename TC, typename TR > - inline PyObject* _callMethod ( TR(TC::* method)(), TC* cppObject, std::tuple<> ) + inline PyObject* _callMethod ( TR(TC::* method)(), TC* cppObject, Args<>& ) { return _callMethodReturn( method, cppObject ); } - - // Function with one argument. + + // Make the translation call for a method one argument. template< typename TC, typename TR, typename TA0 > - inline PyObject* _callMethod ( TR(TC::* method)(TA0), TC* cppObject, std::tuple args ) - { return _callMethodReturn( method, cppObject, std::get<0>(args) ); } - - // Function with two arguments. + inline PyObject* _callMethod ( TR(TC::* method)(TA0), TC* cppObject, Args& args ) + { return _callMethodReturn( method, cppObject, as( args[0] ) ); } + + // Make the translation call for a method two argument. template< typename TC, typename TR, typename TA0, typename TA1 > - PyObject* _callMethod ( TR(TC::* method)(TA0,TA1), TC* cppObject, std::tuple args ) - { return _callMethodReturn( method, cppObject, std::get<0>(args), std::get<1>(args) ); } + PyObject* _callMethod ( TR(TC::* method)(TA0,TA1), TC* cppObject, Args& args ) + { return _callMethodReturn( method, cppObject, as( args[0] ), as( args[1] ) ); } -The complete work of translating the Python tuple into a ``std::tuple<>`` and error -handling is done with a dedicated template class ``PyMethodWrapper`` and it's ``call()`` -method. +The complete work of translating the Python tuple into a ``Args<>`` is done inside +a dedicated template class ``PyWrapper`` and it's ``call()`` method. +C++ exceptions are catched and translated into Python ones. + +* ``PyWrapper`` the base class wich handle both C++ and Python exceptions. + Provides the ``call()`` function which in turn wraps the ``_call()`` that + must be overloaded in derived classes. + +* ``PyFunctionWrapper<>``, template derived class for C/C++ normal functions. + +* ``PyMethodWrapper<>``, template derived class for C++ class methods. + Two flavors are supported, the real method and a function build upon a + method (first argument beaing the object itself). The later is used when + we need to desambiguate overloaded functions, we must create one *function* + per overload. As a class template cannot guess the template parameters, we wrap them into a function template which can perform the guess. The ``callMethod<>`` template function. @@ -384,7 +529,7 @@ In the end, what the user can write is simply: .. code-block:: C++ - static PyObject* PyParameter_addValue ( PyVoidPointer* self, PyObject* args ) + static PyObject* PyParameter_addValue ( PyObject* self, PyObject* args ) { return callMethod("Parameter.addValue",&Parameter::addValue,self,args); } PyMethodDef PyParameter_Methods[] = @@ -400,7 +545,7 @@ In the end, what the user can write is simply: 2.6 Case of C++ overloaded functions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This apply to both overloaded functions and functions with arguments values. +This apply to both overloaded functions and functions with default arguments values. In that case, the only solution is to create a set of different functions with differents arguments to expose all the various signature of the function. @@ -433,7 +578,7 @@ support both case (through different constructors). , Parameter::Priority pri ) { return self->setString(value,pri,flags); } - static PyObject* PyParameter_setString ( PyVoidPointer* self, PyObject* args ) + static PyObject* PyParameter_setString ( PyObject* self, PyObject* args ) { PyObject* rvalue = callMethod("Parameter.setString",&setString3,self,args); if (not rvalue) rvalue = callMethod("Parameter.setString",&setString2,self,args); @@ -484,7 +629,7 @@ the ``Priority`` ``enum``. .. code-block:: C++ template<> - inline void pyTypePostModuleInit ( PyTypeObject* typeObject ) + inline void pyTypePostInit ( PyTypeObject* typeObject ) { PyTypeManagerNonDBo::create( (PyObject*)typeObject , Cfg::PyParameterPriority_Methods @@ -493,7 +638,7 @@ the ``Priority`` ``enum``. } template<> - inline void pyTypePostModuleInit ( PyTypeObject* typeObject ) + inline void pyTypePostInit ( PyTypeObject* typeObject ) { // Parameter::Priority enum. addConstant( typeObject, "UseDefault" , Cfg::Parameter::UseDefault ); diff --git a/documentation/content/pages/python-cpp-new/Introduction.rst b/documentation/content/pages/python-cpp-new/Introduction.rst index e0221a5a..10551fc0 100644 --- a/documentation/content/pages/python-cpp-new/Introduction.rst +++ b/documentation/content/pages/python-cpp-new/Introduction.rst @@ -12,7 +12,7 @@ directly your functions with the original API or any mix between. You only have to respect some naming convention. -* Coriolis is build against Python 2.7. +* Coriolis is build against Python 3.6. 1.1 About Technical Choices @@ -52,10 +52,10 @@ Some would say, why not use *off the shelf* wrappers like ``swig``, #. **Linking accross modules.** As far as I understand, the wrappers are for monolithic libraries. That is, you wrap the entire library - in one go. But Hurricane has a modular design, the core database + in one go. But Coriolis has a modular design, the core database then various tools. We do not, and cannot, have one gigantic wrapper that would encompass all the libraries in one go. We do one Python - module for one C++ library. + module for each C++ library. This brings another issue, at Python level this time. The Python modules for the libraries have to share some functions. Python diff --git a/documentation/output/pages/python-cpp-new.html b/documentation/output/pages/python-cpp-new.html index 4fe139b7..06c32538 100644 --- a/documentation/output/pages/python-cpp-new.html +++ b/documentation/output/pages/python-cpp-new.html @@ -153,7 +153,7 @@ the standard Python C/API. That is, you may not use them and write directly your functions with the original API or any mix between. You only have to respect some naming convention. -
  • Coriolis is build against Python 2.7.
    • +
  • Coriolis is build against Python 3.6.

    • 1.1 About Technical Choices

    @@ -189,10 +189,10 @@ by the PyTypeManager::_link()

  • Linking accross modules. As far as I understand, the wrappers are for monolithic libraries. That is, you wrap the entire library -in one go. But Hurricane has a modular design, the core database +in one go. But Coriolis has a modular design, the core database then various tools. We do not, and cannot, have one gigantic wrapper that would encompass all the libraries in one go. We do one Python -module for one C++ library.

    +module for each C++ library.

    This brings another issue, at Python level this time. The Python modules for the libraries have to share some functions. Python provides a mechanism to pass C function pointers accross modules, @@ -218,83 +218,107 @@ hides the Python/C API. Instead we keep mostly visible the classic structure of the Python/C API but we provides templates to automate as much as possible the boring tasks (and code duplication). This way, if we need a very specific feature at some point, we can still revert back to the pure Python/C API.

    -

    The key features of our wrapper are:

    +

    The wrapper basically provides support for three kind of operations:

    +
      +

    1. Encapsulate C++ object in Python ones, done by cToPy<>() template. +Template specializations are defined for the POD and basic STL +types like std::string.

      +

      To add more easily new specializations, they resides in the top level +scope (not inside Isobar).

      +
      2. +

    2. Decapsulate a C++ object from a Python one, done by pyToC(). +It's implementation is slightly different from the one of cToPy<>() +in the sense that it is a mix of normal C++ functions and template +functions. I was having trouble inside the callMethod() to access +to templates specialization defined after that point, so function be +it.

      +

      There are two mutually exclusives versions of the pyToC<>() for +objects managed through the type manager. One is for value types and +the other for pointer types.

      +
      3. +

    3. Wrap C/C++ functions & methods inside C-linkage PyCFunction, that is +PyOject* (*)(PyObject*, PyObject*). This is done respectively through +callFunction<>() and callMethod<>(). callMethod<>() having +two variants, one for directly calling the function member, if there is +no overload and one for calling one given flavor of the overloaded +function member.

      +

      In addition we provides special variants for Python unary, binary and +operator functions.

      +
      4. +
    +

    In addition to those user exposed features, we have:

    • The PyTypeManager and it's derived classes to store and share informations about all our newly defined PyTypeObjects.
      • -
    • The callMethod<>() and callFunction<>() function templates to -automatically generate a wrapper around C++ object methods or functions.
      • -
    • The PyBoject* objectLink<CppT>( CppT* ) functions templates that provides C++ to Python -object translation.
      • -
    • The bool pyAs<CppT>( PyObject*, CppT*& ) functions templates that provides Python to C++ -object translation.
      • +
    • Various wrapper classes to wrap functions & methods. They are not directly +exposed because the template class intanciation needs the template parameters +to be explicitely given, wich is tedious. Instead we create them through +a function template call, which will perform for us the template type +deduction.

    We creates only two kind of PyObject (but many PyTypeObject):

      -

    • PyVoidpointer which encapsulate one void pointer to the C++ associated +

    • PyOneVoid which encapsulate one void pointer to the C++ associated object.

      extern "C" {
-  typedef struct PyVoidPointer {
+  typedef struct PyOneVoid {
     PyObject_HEAD
-    void* _object;
+    void* _object1;
   };
 }
      • -

    • PyIteratorPointer which encapsulate one void pointer to the C++ associated -iterator and one another to the PyObject of the container.

      +

    • PyTwoVoid which encapsulate one void pointer to the C++ associated +object (an iterator) and one another to the PyObject of the container.

      extern "C" {
-  typedef struct PyVoidPointer {
+  typedef struct PyTwoVoid {
     PyObject_HEAD
-    void*          _object;     // C++ iterator.
-    PyVoidPointer* _container;  // Python wrapped container.
+    void*     _object1;  // C++ iterator.
+    PyObject* _object2;  // Python wrapped container.
   };
 }
    +
    +

    Note

    +

    A PyTwoVoid can be casted/accessed as a PyOneVoid.

    +

    2.1 PyTypeManager

    PyTypeManager has two usage:

    -