[ python-Bugs-1337400 ] Python.h should include system headers properly [POSIX]

Sat Sep 23 14:37:05 CEST 2006

Bugs item #1337400, was opened at 2005-10-25 14:38
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1337400&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
Status: Closed
Resolution: Works For Me
Priority: 5
Submitted By: Dimitri Papadopoulos (papadopo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python.h should include system headers properly [POSIX]

Initial Comment:
In Python 2.4.2, Python.h looks like this:

	#include <limits.h>
	[...]
	#include <stdio.h>
	[...]
	#include <string.h>
	#include <errno.h>
	#include <stdlib.h>
	#ifdef HAVE_UNISTD_H
	#include <unistd.h>
	#endif

On POSIX platforms <unistd.h> should be included first!

Indeed it includes headers such as
<sys/feature_tests.h> on Solaris, <standards.h> on
Irix, or <features.h> on GNU systems, which define
macros that specify the system interfaces to use,
possibly depending on compiler options, which in turn
may enable/disable/modify parts of other system headers
such as <limits.h> or <errno.h>.

By including <unistd.h>, you ensure consistent systems
interfaces are specified in all system headers included
by Python sources.

This may seem rather academic, but it actually breaks
my Solaris builds: I need to compile Python using Sun's
C compiler when building Python for performance and
GNU's C++ compiler when building Python modules written
in C++ for compatibility with C++ libraries used by
these modules that can't be compiled with Sun's C++
compiler. So the same Python.h is used by Sun's C
compiler (which it was created for in the first place)
and GNU's C++ compiler. GNU's C++ compiler fails to
compile some modules. Unfortunately I can't recall the
exact modules and error messages right now, but
including <unistd.h> fixes the problem.

----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2006-09-23 14:37

Message:
Logged In: YES 
user_id=21627

Skip, your problem is completely independent of the problem
reported here. It rather has to do with the way Sun non-C89 API.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2006-09-12 22:01

Message:
Logged In: YES 
user_id=44345

I'm coming late to this party (it seems the bar is closed),
however...

At work we just stumbled upon a similar problem when trying
to build
the latest release of matplotlib (0.87.5, avaialble at
http://matplotlib.sourceforge.net/) on Solaris 10 using gcc
3.4.1.  We
get errors like the following:

    In file included from
/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/bits/postypes.h:46,
		     from
/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/iosfwd:50,
		     from
/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/ios:44,
		     from
/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/ostream:45,
		     from
/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/iostream:45,
		     from swig/agg_buffer.h:7,
		     from src/agg.cxx:1584:

/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/cwchar:145:
error: `::btowc' has not been declared

/opt/app/g++lib6/gcc-3.4/lib/gcc/i386-pc-solaris2.10/3.4.1/../../../../include/c++/3.4.1/cwchar:150:
error: `::fwide' has not been declared

If I add

    #include <wchar.h>

at the top of Python.h it builds fine (modulo a couple
warnings).  One
warning which might be significant is:

    In file included from
/opt/app/g++lib6/python-2.4/include/python2.4/Python.h:10, 
               from src/agg.cxx:97:

/opt/app/g++lib6/python-2.4/include/python2.4/pyconfig.h:835:1:
warning: "_POSIX_C_SOURCE" redefined
    In file included from /usr/include/wchar.h:11,
		     from
/opt/app/g++lib6/python-2.4/include/python2.4/Python.h:7,
		     from src/agg.cxx:97:
    /usr/include/sys/feature_tests.h:266:1: warning: this is
the location of the previous definition

I don't have enough gcc smarts to understand what's
happening, or even
if it's the same problem Dimitri encountered on Solaris 8,
but it kind
of looks like the same sort of problem to me.

Skip

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2005-11-03 22:34

Message:
Logged In: YES 
user_id=21627

Thanks for the update. I believe nothing in the POSIX spec
mandates to include unistd.h before anything else (unlike
sys/types.h, which often is prerequisite to other headers),
so I'm closing this report.

----------------------------------------------------------------------

Comment By: Dimitri Papadopoulos (papadopo)
Date: 2005-11-03 11:56

Message:
Logged In: YES 
user_id=52414

Aaargh! I've rebuilt a version of Python 2.4.2 with Sun's C
compiler and GNU's C++ compiler but I'm unable to reproduce
the problem:

$ cat > foo.cpp
#include <Python.h>
#include <cwchar>
$
$ g++ -I/usr/local/python-2.4.2/include/python2.4 -c foo.cpp
$ 

Same Solaris 8 workstation, no OS updates, same GCC and same
Sun compilers. Oh well...

I think it's still a good idea to include <unistd.h> before
<limits.h>, <stdio.h>, <string.h>, <errno.h> and <stdlib.h>.
But that's your call, I don't mind as long as I'm able to
build Python.

----------------------------------------------------------------------

Comment By: Dimitri Papadopoulos (papadopo)
Date: 2005-11-02 09:42

Message:
Logged In: YES 
user_id=52414

Ah, I didn't explain myself clearly.

I meant to say that <unistd.h> must be included before other
system headers such as <limits.h>, <stdio.h>, <string.h>,
<errno.h> and <stdlib.h> in this specific case.

I totally agree it has to be included after "pyconfig.h".
For example if "pyconfig.h" defined _HPUX_SOURCE and
<unistd.h> was included before "pyconfig.h", then wrong
system APIs may be triggered (or at least system APIs that
were not intended to be specified).

Now why <unistd.h> should be included in front of other
system headers? This is because:
1) <unistd.h> triggers specific POSIX or Single UNIX
Specification APIs
2) most if not all system headers do not include <unistd.h>,
so different APIs may be triggered before including
<unistd.h> and after including <unistd.h>

I can't provide a section of the POSIX specification that
explictly states that <unistd.h> must be included before
<stdlib.h>. This is however implicit:
http://www.opengroup.org/onlinepubs/009695399/basedefs/unistd.h.html
As you can see <unistd.h> may or not define macros that
trigger a specific API (POSIX.1-1988, SUSv1, SUSv2, SUSv3,
etc.).

I'll investigate what happens in the case of this specific
failure and let you know.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2005-11-02 05:19

Message:
Logged In: YES 
user_id=21627

Can you please point to the relevant section of the POSIX
specification that states that unistdh.h must be included
before stdlib.h?

As for the specific problem: it may be that you are somehow
working around the real problem by including unistd.h before
Python.h. Python.h *must* be included before any system
headers, as pyconfig.h defines certain compilation options
which affect the feature tests. Including unistd.h before
can actually break things, as structs may get defined
differently depending on whether pyconfig.h was included
first or not.

So in the specific example, it would help if you could
determine why ::btowc is defined in one case but not in the
other.

----------------------------------------------------------------------

Comment By: Dimitri Papadopoulos (papadopo)
Date: 2005-10-25 15:57

Message:
Logged In: YES 
user_id=52414

Oops... Instead of
 including <unistd.h> fixes the problem.
please read
  including <unistd.h> first fixes the problem.

Here is an example to reproduce the problem:

$ cat > foo.cpp
#include <Python.h>
#include <cwchar>
$
$ g++ -I/usr/local/python/include/python2.4 -c foo.cpp
[...]
/usr/local/gcc-4.0.2/lib/gcc/sparc-sun-solaris2.8/4.0.2/../../../../include/c++/4.0.2/cwchar:145:
error: '::btowc' has not been declared
[...]
$
$ cat > foo.cpp
#include <unistd.h>
#include <Python.h>
#include <cwchar>
$
$ g++ -I/usr/local/python/include/python2.4 -c foo.cpp
[...]
$ 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1337400&group_id=5470