Python not freeing memory (?)

Chris Fonnesbeck fonnesbeck at gmail.com
Sun Aug 12 21:22:40 EDT 2007


Martin v. Löwis <martin <at> v.loewis.de> writes:

> If you want others to help you in finding the bug, you
> need to provide more detail, e.g. a specific piece of
> code that reproducibly wastes memory. If you want to
> study how Python objects are allocated and released,
> you need to create a debug build of Python (and all
> extension modules), and start, e.g., with looking at the
> value of sys.gettotalrefcount() over time.

I tried monitoring the refcount at every iteration, but it
does not change; at the same time, the memory use by the 
python process increases. This is why I suspected that 
python was not returning memory.

Below is the method that gets called iteratively; the *_like methods
are statistical likelihoods implemented in f2py. I dont see
anything that is obviously responsible:

    def model(self):
        # Specification of joint log-posterior
        
        #alpha3 = concatenate(([0], self.alpha3))
        
        # Linear model for surfacing wait time
        #self.lamda = exp(self.alpha0 + self.alpha1 * self.wind + self.alpha2 * 
self.air + alpha3[self.series-1])
        gamma3 = concatenate(([0], self.gamma3))
        
        # Linear model for at-surface probability
        self.theta = invlogit(self.gamma0 + self.gamma1 * self.wind + self.gamma
2 * self.air + gamma3[self.series-1])
        
        x, n, theta = transpose([[z[1], sum(z), t] for z, t in zip(self.downup, 
self.theta) if type(z)!=type(0.0)])
        
        # Binomial likelihood of available animals
        self.binomial_like(x, n, theta, name='theta')
        
        # Probability of availability (per survey)
        self.pa = 1.0 - (1 - self.theta)**10
        
        beta3 = concatenate(([0], self.beta3))
        
        # Linearmodel for probability of detection
        self.pd = invlogit(self.beta0 + self.beta1 * self.wind + self.beta2 * (s
elf.cloud>0) + beta3[self.series-1])
        
        # Binomial likelihood of detection
        self.binomial_like(self.obs, self.present, self.pd * self.pa, name='pd')
        
        zeta1 = concatenate(([0], self.zeta1))
        
        # Probability of presence
        self.pp = invlogit(self.zeta0 + zeta1[self.series-1] + self.zeta2 * self
.intake + self.zeta3 * (self.discharge - self.intake))
        
        # Binomial likelihood of presence
        self.binomial_like(self.present, self.absent + self.present, self.pp, na
me='pp')
        
        # Correct flight counts for detection
        self.N_flight = self.count / (self.pd * self.pa)
        
        # Aggregate counts by series
        N_series = [self.N_flight[self.series==i+1] for i in range(6)]
        
        # Log-normal likelihood for N
        #sum(self.lognormal_like(X, log(N), T) for X, N, T in zip(N_series, self
.N, self.T))
        for N, mu in zip(N_series, self.N):
            self.poisson_like(N, mu)

Thanks in advance for anything that you might suggest.

cf




More information about the Python-list mailing list