Python Sybase slow compared to Perl?

[Chuck May]

I have always written my database scripts in Perl.  I am very interested 
in Python, and would like to start using it for all my scripting.  I am 
not trying to start a flame war, I'd just like to understand why a 
particular script I wrote runs so much faster under Perl.    The script in 
question takes roughly 3 times as long to run under Python.  It is very 
possible that there is a faster way to do this under Python.

I cut out most of the script, leaving only the part of the scrit that does 
all the work.  I am basically selecting all the fields from a table, 
looping over the results, and outputting the results in a fixed-width text 
file.  Seems simple enough.  

Here is the Perl code snippet:

   # output the actual data
   $sth = $dbh->prepare( "SELECT $select FROM $table->[0]" );
   $sth->execute();
   $num_recs = 0;
   $tempfile = "$dumpfile.tmp.gz";
   unlink $tempfile;
   open( DUMP_FILE, "| gzip -c > $tempfile" )
   while ( @data = $sth->fetchrow_array() ) {
       $line = "";
       for ( $i = 0; $i < scalar @lengths; $i++ ) {
           $line .= pack( "A$lengths[$i]", $data[$i] );
       }
       print DUMP_FILE "$line\n";
       $num_recs++;
   }
   close DUMP_FILE;
   $sth->finish();

and here is the Python code snippet:

   # output the actual data
   c = db.cursor()
   c.execute( "SELECT %s FROM %s" % ( select, table ) )
   num_recs = 0
   if os.path.isfile( "%s.gz" % dumpfile ):
       os.remove( "%s.gz" % dumpfile )
   file = os.popen( "gzip -c > %s.gz" % dumpfile, "w" )
   while 1:
       row = c.fetchone()
       if not row : break
    
       line = []
       for i in range( len( row ) ):
           line.append( "%-*s" % ( lengths[i], row[i] ) )

       print >> file, "".join( line )
       num_recs += 1

   file.close()
   c.close()

No matter what table how many records there are, the Python script always 
takes longer to run (usually by a facter of 3 or more).  

I am still a Python newbie, so I may be doing something wrong.  Has anyone 
else experienced similar results?  Is this due to a poorly optimized 
Sybase module?  I really want to use Python, and I don't mind giving up a 
little speed, but this is more than I can accept.

I ran the Python script using the profiler and the output is below.  Any 
help would be much appreciated.  Thanks.

Chuck May
IMS, Inc.

-------------

profile output:

 Ordered by: standard name

 ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      1    0.000    0.000 1230.540 1230.540 <string>:1(?)
      4    0.000    0.000    0.000    0.000 Sybase.py:114(_get_diag)
      2    0.000    0.000    0.000    0.000 Sybase.py:141(_ct_errors)
      3    0.000    0.000    0.000    0.000 Sybase.py:148(_row_bind)
 900345   96.240    0.000   96.240    0.000 Sybase.py:166(_extract_row)
 900348  177.670    0.000  273.910    0.000 Sybase.py:180(_fetch_rows)
      3    0.000    0.000    0.010    0.003 Sybase.py:229(__init__)
      3    0.000    0.000    0.000    0.000 Sybase.py:246(__del__)
 900361   83.230    0.000  267.090    0.000 Sybase.py:258(_lock)
 900361   86.720    0.000  269.000    0.000 Sybase.py:262(_unlock)
      3    0.000    0.000    0.010    0.003 Sybase.py:312(close)
      3    0.000    0.000    0.000    0.000 Sybase.py:328(execute)
 900348  178.280    0.000  988.260    0.001 Sybase.py:365(fetchone)
      1    0.000    0.000    0.000    0.000 Sybase.py:408(fetchall)
      3    0.000    0.000    0.000    0.000 Sybase.py:451(_fetch_rowcount)
      3    0.000    0.000    0.000    0.000 Sybase.py:466(_start_results)
      2    0.000    0.000    0.010    0.005 Sybase.py:494(__init__)
      2    0.000    0.000    0.000    0.000 Sybase.py:522(_raise_error)
      2    0.010    0.005    0.010    0.005 Sybase.py:527(connect)
      2    0.000    0.000    0.000    0.000 Sybase.py:564(__del__)
      4    0.000    0.000    0.000    0.000 Sybase.py:570(close)
      3    0.000    0.000    0.010    0.003 Sybase.py:613(cursor)
      2    0.000    0.000    0.000    0.000 Sybase.py:618(execute)
      2    0.000    0.000    0.000    0.000 Sybase.py:640(_fetch_results)
      2    0.000    0.000    0.010    0.005 Sybase.py:667(connect)
      1    0.000    0.000 1230.540 1230.540 bsi_dump.py:176(main)
      3    0.000    0.000    0.000    0.000 bsi_dump.py:238(outputMessage)
      1  242.210  242.210 1230.520 1230.520 bsi_dump.py:39(dumptable)
      1    0.030    0.030    0.030    0.030 commands.py:50
(getstatusoutput)
      1    0.000    0.000    0.000    0.000 getopt.py:121(do_shorts)
      1    0.000    0.000    0.000    0.000 getopt.py:136(short_has_arg)
      1    0.000    0.000    0.000    0.000 getopt.py:35(getopt)
      1    0.010    0.010    0.010    0.010 getpass.py:18(unix_getpass)
      1    0.000    0.000    0.000    0.000 getpass.py:71(_raw_input)
      2    0.000    0.000    0.000    0.000 posixpath.py:194(isfile)
      1    0.080    0.080 1230.620 1230.620 profile:0(main())
      0    0.000             0.000          profile:0(profiler)
      2    0.000    0.000    0.000    0.000 stat.py:29(S_IFMT)
      2    0.000    0.000    0.000    0.000 stat.py:54(S_ISREG)
      2    0.000    0.000    0.000    0.000 string.py:121(join)
      2    0.000    0.000    0.000    0.000 threading.py:39(__init__)
1800738   60.630    0.000   60.630    0.000 threading.py:44(_note)
1800738   60.440    0.000   60.440    0.000 threading.py:581
(currentThread)
      2    0.000    0.000    0.000    0.000 threading.py:64(RLock)
      2    0.000    0.000    0.000    0.000 threading.py:69(__init__)
 900369  123.010    0.000  183.860    0.000 threading.py:81(acquire)
 900369  122.060    0.000  182.280    0.000 threading.py:99(release)