[SciPy-user] bdist_rpm build error in scipy_core from CVS

Thu Jan 27 19:08:38 EST 2005

Joe Cooper wrote:
> Hi all,
> 
> There's something gone awry in recent scipy_core CVS.  When I last 
> packaged it up, it worked OK with bdist_rpm, but now it fails with the 
> following traceback:

[...]

I actually don't have a solution to this, the last version I built an RPM for 
was CVS from a few weeks ago:

In [2]: scipy.__version__
Out[2]: '0.3.2_300.4526'

It worked fine.  But I want to mention something in this thread, b/c it's RPM 
related.  If anyone is going to have a look at the scipy rpm build process, it 
might be worth fixing a few (very minor) issues.  I had also promised to pass 
some of this info to others (esp. S. Walton), so here it goes.

I recently set up a group of Fedora3 repositories for our systems, with 
multiple architectures and therefore different ATLAS releases (which implies 
different Numeric/Scipy RPMs as well).  For the most part the process went 
quite well, I finally got it down to the following:

cd /path/to/scipy/source/dir
cd scipy_core
pybrpm-arch Scipy_core
cd ..
pybrpm-arch SciPy

where pybrpm-arch is a simple shell script (attached) which automates the 
building and installation of an rpm via bdist_rpm.  But the packages had to be 
split in Scipy_core and SciPy (with that funny capitalization), otherwise 
things won't install, since Scipy_core contains the critical scipy_distutils.

I'm not sure where the discussion stands on what to include into scipy_core 
and what should go in the main package.  Currently, scipy_core includes:

/scipy_base
/scipy_distutils
/scipy_test
/weave

and SciPy contains

/gui_thread
/scipy

(all of this in /usr/lib/python2.3/site-packages).  I don't have a problem 
with that, I just list it here for reference.

The only thing I'd like to see fixed is the awkward naming of the packages: 
why not be consistent and just name the two proper RPMs

scipy_core
scipy

This post is mainly to just ask for this minor change, and provide the install 
scripts which may be of use to others for managing Fedora-based environments. 
  They all rely on a few environment variables which need to be correctly 
configured.  This is from my root/.tcshrc:

##############################################################################
#
# Local yum configuration
#
alias yinst 'yum -y install'
alias yup   'yum -y update'

# Fedora Release version, as seen by YUM.
setenv RELEASEVER 3
# Architecture-specific flag, as per the ATLAS binaries convention
setenv ARCH P4SSE2

# Yum allows expanding $YUM0-9 variables in its .conf files, so by setting
# this variable we can manage architecture-specific repositories.

setenv YUM0 /usr/local/installers/yum/fc${RELEASEVER}
setenv YUM1 $ARCH
setenv YUM2 ${YUM0}-arch/${YUM1}
##############################################################################

My local yum repo is organized as:

root at planck[yum]# d
/usr/local/installers/yum
total 20
drwxr-xr-x  3 root 4096 Jan 27 15:47 fc3/
drwxr-xr-x  5 root 4096 Jan  7 12:00 fc3-arch/
drwxr-xr-x  3 root 4096 Jan 19 18:28 fc3-shared/
-rw-r--r--  1 root  984 Jan 10 16:24 README
-rwxr-xr-x  1 root   95 Jan  4 11:53 update-repo*

where fc3-arch contains the architecture-specific repositories:

root at planck[fc3-arch]# d
/usr/local/installers/yum/fc3-arch
total 12
drwxr-xr-x  3 root 4096 Jan 25 12:34 P4SSE2/
drwxr-xr-x  3 root 4096 Jan 19 18:29 P4SSE2_2HT/
drwxr-xr-x  3 root 4096 Jan 19 18:29 PIIISSE1/

and update-repo is a trivial one-liner I run whenever any repo changes (it's 
symlinked from all of them):

root at planck[yum]# cat update-repo
#!/bin/sh
# as of FedoraCore3, this is the proper command to update a repository

createrepo .

#########################

This setup allows me to manage multiple heterogeneous machines which fetch 
their updates from a single NFS-shared group of yum repos, without any 
conflicts and using the same config files (in /etc/yum.repos.d) for all.  Here 
are the relevant repo files:

root at planck[yum.repos.d]# cat local.repo
### Local packages

# These are directories with locally available rpms.  Remember to run
# 'createrepo .' when new rpms get added, so that the repodata/ subdir needed
# by yum is updated.

[local]
name = Local packages (Fedora Core $releasever)
baseurl = file://$YUM0
enabled=1

#########################
root at planck[yum.repos.d]# cat local-arch.repo
### Architecture-dependent local packages

# These are directories with locally available rpms.  Remember to run
# 'createrepo .' when new rpms get added, so that the repodata/ subdir needed
# by yum is updated.

[local-arch]
name = Architecture-dependent local packages (Arch: $YUM1)
baseurl = file://$YUM2
enabled=1

#########################
root at planck[yum.repos.d]# cat local-shared.repo
### Architecture-dependent local packages

# These are directories with locally available rpms.  Remember to run
# 'createrepo .' when new rpms get added, so that the repodata/ subdir needed
# by yum is updated.

# The -shared repo contain RPMs which were written to go into /usr/local/, and
# hence are effectively shared via NFS.  This repository should only be
# enabled on the host which hosts /usr/local.

[local-shared]
name = Shared local packages
baseurl = file://$YUM0-shared
enabled=1

####################

I've written also some code to auto-generate architecture-specific RPMs out of 
the scipy.org ATLAS binary tarballs, which I can provide if anyone is interested.

All this, combined with some more scripts to auto-generate kickstart install 
scripts, gives a pretty reasonable solution for managing the problem of 
multiple machines, with multiple architectures and in-house managed RPMs for 
rapidly changing code (like scipy, ipython and matplotlib), while keeeping my 
sanity and time to do my official job (research).

We can now go from blank hard disk to fully updated box in about 1 hour, with 
about 3 minutes of human intervention.  And the machines stay up to date via 
yum, including regarding our own managed packages.

Hopefully some of this is useful to others.

Cheers,

f.

ps. And Joe, good luck with your bug :)
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pybrpm-arch
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20050127/8d2c0eb2/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pybrpminst
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20050127/8d2c0eb2/attachment-0001.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pybrpm-noarch
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20050127/8d2c0eb2/attachment-0002.ksh>