Memory usage problem using SWIG-wrapped library

Ray Van Dolson rayvd at nospam.bludgeon.org
Wed May 23 11:44:46 EDT 2001


I've used SWIG to wrap some functions in libsmbclient.so (included in the 
latest version of Samba).  I am using a recursive program to span through 
shares and directories on a Windows 2000 server and making a map of the 
files on it.

#!/usr/bin/python



#import MySQLdb

import sys

import string

import pysmb



computers = ["BLUDGEON","TULCATE","PATRIOT","LEFTY"]



def insertDB(url):

	#db = MySQLdb.connect
(db='samba',host='localhost',user='rayvd',passwd='')

	#cursor = db.cursor()

	sqlStatement = """\

INSERT INTO smbFile (fileName,fileSize,computerName,path) VALUES ("%s","%
s","%s","%s")\

""" % (string.split(url,"/")[-1],pysmb.size("smb://"+url),string.split
(url,"/")[0],string.join(string.split(url,"/")[1:-1],"/"))

	#cursor.execute(sqlStatement)

	#db.close()



def descend(topURL,s=""):

	print "Descending into smb://%s" % topURL

	dirList = pysmb.directory_list("smb://%s" % topURL)[1]

	for entry in dirList:

		if pysmb.is_directory("smb://%s" % topURL + "/" + entry):

			if entry != "." and entry != "..":

				descend("%s" % topURL + "/" + entry, "\t")

		elif pysmb.is_file("smb://%s" % topURL + "/" + entry):

			#print s+entry, pysmb.size("smb://%s" % topURL + "/" + 
entry)

			insertDB(topURL+"/"+entry)

			#print topURL

		else:

			pass

	print sys.getrefcount(dirList)

	return 1

	

def main():

	global computers



	sys.setrecursionlimit(10000)



	for computer in computers:

		descend(computer)



if __name__ == '__main__':

	main()


Kind of a mess, but as you can see, descend is the function that gets called 
recursively.  The problem I am having is that after this program runs for a 
while, it gets Killed -- I assume by the system.  Watching the process with 
'top', I see its memory usage shoots up above 80% right before it is killed 
leading me to think it's a memory problem.

The function directory_list() is the C function being wrapped and is passed 
a char *dirs[512] array which is malloc'd for each entry of the array to 
have a strlen of 127.  Here is the .i file I am using (along with the 
typemap):

http://www.bludgeon.org/~rayvd/pysmb/c/pysmb.i

You can see where the memory is allocated, but I am wondering if perhaps I 
need to use free() somewhere to free this memory once this list is returned?  
I stuck a free($source[i]) in the argout typemap, but this only caused the 
function to Segmentation Fault.

Any ideas what I can do to reduce memory usage?  Besides writing a non-
recursive version of this program :)  The C code I am wrapping is at

http://www.bludgeon.org/~rayvd/pysmb/c/pysmb.c

Please feel free to take a look--thanks for any help.

Ray Van Dolson



More information about the Python-list mailing list