Best way to go about embedding python

[Kaylen Wheeler]

I wonder if Lua would be a better option.  Does it suffer from the same sandboxing issues that python does?

On Sunday, 13 November 2016 17:38:23 UTC-8, Nathan Ernst  wrote:
> In regards to performance of Lua vs Python, I don't have enough (near zero
> experience) with Lua to comment there.
> 
> But in regards to embedding in a game, the only experience I have w/ Python
> being embedded is while working on modding Civilization IV. What I saw
> there just made me nauseous.
> 
> The reasoning for moving to Lua in Civilization V was performance & memory
> usage - but they completely screwed up exposing the C++ internals to Python
> in Civ IV. They copied object instances & containers to the Python
> embedded, then marshalled them back to the C++ code when the Python code
> was finished. It was horribly inefficient.
> 
> I don't have much experience hosting Python inside of a C/C++ app, but I've
> 10 years of experience consuming C++ APIs from Python, mostly wrapped using
> Boost Python. I always did my best to minimize data marshalling. The Python
> objects exposed the C++ objects' interface, directly while holding a
> pointer (usually shared) to underlying C++ objects and only marshalling on
> method/property calls where needed (i.e. w/ strings, dates, datetimes).
> Containers were wrapped to expose a Python interface to an underlying C++
> container w/ no copying.
> 
> TLDR; Civilization IV shows an antipattern about how to embed Python in a
> native app and expecting anything resembling good performance. Civ IV used
> Boost Python, but they did it in an extremely poor, suboptimal fashion.
> Boost Python itself can be very performant, and I love the library. Boost
> Python, however does suffer from a learning curve that is akin to
> attempting to scale a 1000ft cliff with no rope or tools or even shoes. Due
> to the heavy use of template metaprogramming in C++, compiler errors if
> you're even just slightly off can be extremely hard to decipher. I've not
> used it lately or on modern compilers, so I don't know if it's gotten
> better, but tools like pretty_make.py (can find on github) can help filter
> out the compiler spewage from the real compiler error.
> 
> PS. I don't think that Python couldn't/shouldn't be used for scripting a
> game, I'm just offering up Civ IV as an example of how it can be done
> horribly wrong.
> 
> Regards,
> Nate
> 
> On Sun, Nov 13, 2016 at 7:03 PM, Chris Angelico <rosuav at gmail.com> wrote:
> 
> > On Mon, Nov 14, 2016 at 11:40 AM, Steve D'Aprano
> > <steve+python at pearwood.info> wrote:
> > > On Mon, 14 Nov 2016 12:23 am, Chris Angelico wrote:
> > >
> > >> Python without its stdlib is a disappointingly featureless
> > >> language :)
> > >
> > >
> > > I don't think that's true. You can do a lot in Python without any
> > imports in
> > > your code:
> > >
> > > - basic string processing
> > > - BigNum (int) and float arithmetic
> > > - lists and tuples
> > > - dicts
> > > - lazy processing of iterators
> > > - custom functions and classes
> > > - functional style zip, map and filter
> > >
> > > and more.
> > >
> > > Some parts of the standard library are required for built-in functions,
> > e.g.
> > > the io module is needed to open files. But I think you could probably
> > > reduce the standard library by, oh, 90% and still have a decent language
> > > for game scripting.
> >
> > Perhaps what would be more accurate is that Python without its stdlib
> > is as featureless as (say) Lua, but still without being safe to embed.
> > Yes, it's technically usable, but you really don't have a lot. And
> > remember, some of the stdlib is what we call builtins; if you embed
> > Python and engage "restricted mode", it's by emptying out the
> > builtins. You're pretty much down to three things:
> >
> > 1) Stuff that you could pass to ast.literal_eval
> > 2) Control flow statements
> > 3) Hacks that get you around the limitations, thus proving that
> > Python-without-stdlib is really really hard to actually test anything
> > with.
> >
> > ChrisA
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >