How to port Python to VxWorks

[Sofin Raskin]

I did port of Python to VxWorks. I wrote some additional modules, that
interface
with different libraries on VxWorks(memory, networking and etc.). I don't
have time to write few README and HOWTO documents. I tested it on R5000 and
ALPHA(64bits)  processors. Python works great and it saves a lot of time to
write prototype versions of the system, but you pay with additional amount
of memory.

I am ready to send it to anyone, who is interesting in it.

Regards,
Sofin


Paul Fernhout wrote:

> Pythonistas-
>
> I just got a bare bones port of Python running on VxWorks and thought
> I'd share some insights in how to do it (OK, and brag a bit too. :-).
> For various reasons (the work was done for a client) I can't make the
> code available. Still, I thought some general comments on this process
> might be useful. Previous postings indicate that there have been at
> least two other ports of Python to VxWorks. However, I haven't seen any
> code for this anywhere or a discussion of the specific issues for
> VxWorks.
>
> Background:
>
> VxWorks is a Real Time Operating System (RTOS), meaning that you can in
> theory make "real time" applications on it that respond to time critical
> events in a (guaranteed) timely fashion. It is typically used to develop
> applications for embedded computing devices like cell phones, Mars
> rovers, and other "invisible" computers. To the developer, VxWorks looks
> a lot like UNIX(tm). VxWorks has been called the "Cadillac" of embedded
> OSs.
>
> VxWorks development is often done using a cross compiler, typically
> using a product from WindRiver Systems called Tornado, which is a
> development shell around the GNU tool suite (GCC, GDB). The board you
> are developing for is commonly called the "target"; the workstation you
> develop on is called the "host". You download the code compiled on the
> host to the target (via serial, ethernet, or whatever).
>
> Why Python on VxWorks? Because there is quite a bit of things to know to
> compile under VxWorks, the developer license are expensive, and the
> turnaround time to download to a target is long. Python allows one to
> prototype under for example Windows and then run the prototype on the
> target. It also allows rapid development and refinement of test code,
> which can just be downloaded via a terminal emulator to the Python
> command prompt on the target (or via other means). Naturally, there is
> still lots of stuff to do in C (especially Interrupt Service Routines
> (ISRs) and code that must operate quickly).
>
> Note that Java and Smalltalk on the target would also offer these
> advantages. However, Python is a true OO language that is comfortable
> looking to C coders, and the Python license is easier to deal with in
> many ways for embedded systems than the Java or Smalltalk licenses, and
> of course Python has many other uses as well.
>
> Issues:
>
> The Python source for 1.5.2 comes complete with pretty much everything
> you need for the port. However, naturally you will have to already be
> able to do serial I/O on your hardware and have your own make files.
>
> Because of cross compiling approach (host != target), the normal Python
> "configure" script concept doesn't work. Under a typical UNIX, the
> configure script will create a bunch of C files, compile them, and
> evaluate the results to see how long integers are, what header files are
> present, and additional system dependent information -- all
> automatically. The output is a "configure.h" file, produced by that
> script modifying "configure.h.in".
>
> Since VxWorks is typically hosted on a machine other than the target
> board, this approach won't work, since at best you'd get settings for
> the host, not the target. So, you have to go in and manually change
> "configure.h", starting from "configure.h.in".
>
> My initial attempt a while back to start with the Windows version of
> configure.h file led to great confusion. I'd strongly suggest there is
> no substitute for going line by line down that file and set the proper
> flags. That's what I did that worked.
>
> I referred to the online documentation to determine what
> functions are supported. I looked for each header file in the Tornado
> directory. I put in type (int, long, long long) sizes from knowledge of
> the target processor. This process took several hours (after a first
> false start a while back). I commented out undefs I didn't want;
> otherwise I #define constants to 1 in most cases.
>
> I used an existing VxWorks makefile, and modified it to compile all the
> *.c files in a directory and link them together with a main program
> which called the code in "python.c".
>
> One of the problems with porting to VxWorks I had the first time I tried
> it was not knowing what modules to include. I had first tried a
> subtractive approach starting from including everything, and was
> overwhelmed with errors.
>
> This time I tried for the minimal set, by copying files as the compiler
> wanted me about missing items, starting from just having "Python.c". I
> faked the argv, argc arguments, passing an argc of 0 and an argv of an
> array with a "python.exe" string and a 0. (Just a null for argv ends up
> with a crash after starting.)
>
> After some experimentation, I ended up with a directory that had
> just about all the files from the python directory, the compiler
> directory, and a few from the module directory. A few files needed to be
> removed, because they duplicate VxWorks functionality; you'll see errors
> when you try to link.
>
> Now that I know what files are needed for a minimal build, I could
> create a VxWorks make file that would use the files in place in the
> conventional Python directory hierarchy. It would make sense to have a
> list like this somewhere.
>
> I was very surprised when it booted the second time. (The first time it
> crashed due to passing a null for argv and I had to debug it). It gave
> several error messages, due to not having file support on the target (I
> think). But it ran interactively on the serial port, and I was able to
> add numbers, define functions, and print results. I now can try adding
> in more modules. The first time I tried linking in all the modules a
> while back (when trying a subtractive approach to porting) I had all
> sorts of compile errors in various modules, so I expect getting other
> facilities like TCP/IP working may be a bit of a further adventure.
>
> What surprised me most was that I did not have to do anything special
> about "freezing" the application. I did not use freezemain.c or run any
> special converters. Of course, as I want more library support from *.py
> files, I may have to do this. I'm not even sure at this point what
> builtin modules I have to work with.
>
> Overall, this minimal port was doable in a day, but only after much
> previous time spent learning about the source, and also learning from a
> previous failed attempt. Obviously, getting more Python facilities
> operational under VxWorks will take longer.
>
> Particular stumbling blocks:
>
> I didn't know what thread choice to make. I ended up not including a
> thread file at all and turning the thread option off. I would like to
> improve on this, so I could have real Python threads under VxWorks.
>
> "dup2.c" gave me problems because I didn't have a fcntl function. I
> ended up making this just return BAD_RETURN (or whatever the error
> constant was in that function). Since I don't have file support anyway
> (I think the calls just fail), I figured this would not hurt.
>
> Note, this is written from home from memory. There may have been some
> other minor hurdles overcome in the process.
>
> Some architectural suggestions for Python 2 based on Squeak Smalltalk:
>
> Squeak is a portable Smalltalk. I have also put Squeak on embedded
> hardware. From this and other experience, I would suggest a few things
> to consider for Python architecture in the future (Python 2)?
>
> * Consider having a clear separation between C that does library calls
> (files, threads, network, any i/o) and code that does not. When writing
> code to do file stuff for example, have it call something like
> "PyOS_Open()", where that is defined elsewhere, (as opposed to just
> doing an ANSI "open"). This would allow the bulk of the files to be pure
> C, and only a few files to have these OS calls. This would localize
> porting issues and make the whole task more approachable. Platform
> specific modules might not sensibly be done this way; my emphasis here
> is on easily porting the core Python system. Obviously, in some cases
> this woudl have a perfomance impact; one apporach is to typedef some of
> these calls so they can be relaces by ANSI or POSIX calls in place.
>
> In the case of Squeak Smalltalk, the pure C code is actually generated
> from a Smalltalk simulator of the VM (written in a Smalltalk subset that
> mainly manipulates byte arrays and dispatches bytecodes). The actual
> machine dependent part of Squeak (as of about a year ago) is only about
> 2000 lines of C (implementing bitmap display, file handling, mouse
> handling, sound, and networking). This porting task is relatively
> straightforward -- just modify these 2000 lines for your system. Note
> that Squeak also provides a complete GUI, but with emulated widgets.
>
> * Consider having the Python compiler implemented in Python. It can then
> be compiled to Python byte code, so it doesn't have to be linked in.
> It can just be loaded at startup. Of course, this will be slower, but it
> will remove one more porting hurdle.
>
> * Size. Squeak can deliver an entire system with GUI, decompiled source
> code browsing, compiler, development tools, debugger, and more in a
> complete package of about 1 MB to 2 MB. Python seems to be much bigger
> (0.5 MB non-debug DLL for Windows, and many MB of source). I think the
> complete package was about 600K for the core Python (and VxWorks w/ WDB
> debugger agent) without extra modules. That is going to grow rapidly if
> I link in more modules. Clearly there is more that can be done to shrink
> the base Python system implementation for embedded targets. Some of this
> may be a time/space tradeoff. For example, the previous suggestion
> regarding the compiler might result in a smaller but slower compiler
> system.
>
> Anyhow, thanks to Guido and others for such a wonderful system!
>
> -Paul Fernhout
> Kurtz-Fernhout Software
> =========================================================
> Developers of custom software and educational simulations
> Creators of the GPL Garden with Insight(TM) garden simulator
> http://www.kurtz-fernhout.com