Memory usage problem using SWIG-wrapped library
Ray Van Dolson
rayvd at nospam.bludgeon.org
Wed May 23 11:44:46 EDT 2001
I've used SWIG to wrap some functions in libsmbclient.so (included in the
latest version of Samba). I am using a recursive program to span through
shares and directories on a Windows 2000 server and making a map of the
files on it.
#!/usr/bin/python
#import MySQLdb
import sys
import string
import pysmb
computers = ["BLUDGEON","TULCATE","PATRIOT","LEFTY"]
def insertDB(url):
#db = MySQLdb.connect
(db='samba',host='localhost',user='rayvd',passwd='')
#cursor = db.cursor()
sqlStatement = """\
INSERT INTO smbFile (fileName,fileSize,computerName,path) VALUES ("%s","%
s","%s","%s")\
""" % (string.split(url,"/")[-1],pysmb.size("smb://"+url),string.split
(url,"/")[0],string.join(string.split(url,"/")[1:-1],"/"))
#cursor.execute(sqlStatement)
#db.close()
def descend(topURL,s=""):
print "Descending into smb://%s" % topURL
dirList = pysmb.directory_list("smb://%s" % topURL)[1]
for entry in dirList:
if pysmb.is_directory("smb://%s" % topURL + "/" + entry):
if entry != "." and entry != "..":
descend("%s" % topURL + "/" + entry, "\t")
elif pysmb.is_file("smb://%s" % topURL + "/" + entry):
#print s+entry, pysmb.size("smb://%s" % topURL + "/" +
entry)
insertDB(topURL+"/"+entry)
#print topURL
else:
pass
print sys.getrefcount(dirList)
return 1
def main():
global computers
sys.setrecursionlimit(10000)
for computer in computers:
descend(computer)
if __name__ == '__main__':
main()
Kind of a mess, but as you can see, descend is the function that gets called
recursively. The problem I am having is that after this program runs for a
while, it gets Killed -- I assume by the system. Watching the process with
'top', I see its memory usage shoots up above 80% right before it is killed
leading me to think it's a memory problem.
The function directory_list() is the C function being wrapped and is passed
a char *dirs[512] array which is malloc'd for each entry of the array to
have a strlen of 127. Here is the .i file I am using (along with the
typemap):
http://www.bludgeon.org/~rayvd/pysmb/c/pysmb.i
You can see where the memory is allocated, but I am wondering if perhaps I
need to use free() somewhere to free this memory once this list is returned?
I stuck a free($source[i]) in the argout typemap, but this only caused the
function to Segmentation Fault.
Any ideas what I can do to reduce memory usage? Besides writing a non-
recursive version of this program :) The C code I am wrapping is at
http://www.bludgeon.org/~rayvd/pysmb/c/pysmb.c
Please feel free to take a look--thanks for any help.
Ray Van Dolson
More information about the Python-list
mailing list