Reference count access or better method?

[Philip Swartzleonard]

Alex Martelli || Mon 14 Jan 2002 07:21:21a:

> "Philip Swartzleonard" <starx at pacbell.net> wrote in message
> news:Xns9195AC442CC11RASXnewsDFE1 at 130.133.1.4...
>     ...
>> > Almost.  weakref.ref and .proxy do let you specify a callback, but
>> > that happens when the referent is no longer available.  You could
>> > wrap the referent into a highly-transparent-proxy, have all client
>> > code refer to the proxy, and when the proxy goes away and your
>> > weakref.ref to it triggers your callback, you still have the real
>> > object in hand.  I.e., "there's no ill that can't be cured by
>> > another level of indirection":-). 
>> >
>> > The highly transparent proxy can be built by automatic delegation,
>> > cfr http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52295 
>>
>> But what if it is then referenced again, how do i determine that
>> change in state? 
> 
> It's not hard to keep track of that, since other code is asking your
> system ("the registry") for references to "the referent" -- and you
> hand out very-thin-proxies instead.

This sounds reasonable, certanly easier than it did last time i thought 
about it =). It will probably make sense after i get some more rest.
 
>> If it helps, the intent is to sort of 'de-cache' things, so that if an
>> object isn't used for some amount of time, it's memory (gl display
>> list) is destroied, and then it's recreated if it's needed again. But
>> i need the 
> 
> The "amount of time" is another issue; you'll need to have other kinds
> of "events" that cause you to check what's too-old in your (registry)
> cache -- nothing in this scheme implies such events, they just let you
> USE such events to clean up what parts of your cache you want.

Well, since i'm working on a turn-based game, I figured i can just use 
turns as the measure of time - if something hasen't been used in 100 turns 
or whatever, delete it. Also, scan only a few resources every turn, 
cycling, so that it dosen't create a single huge noticible GC cycle. (I'm 
planning on doing a lot of things like this mainly in-depth AI sessions 
(bacground orgs, goverments and stuff like that especially))
 
>> object--- for the textures example, their identity is determined by
>> thier pathname, so i'm going to make a customized list that attempts
>> to construct on first mention, but i can't create these objects from
>> such simple data. Or... maybe i can... hm... i already store the
>> paramters for the object, i could store them in a meta-object of some
>> sort... 
> 
> If you can hold other pieces, rather than the true-object, it makes
> your task simpler.  But, let's assume that "building the object" is a
> VERY time-consuming task.  If you have determined that very often the
> last client-code reference to an enregistered object is dropped and
> then soon after the reference is requested again, then giving some
> hysteresis to the drop-object/rebuild-object pair can substantially
> increase your overall performance.
> 
> Let's build a sample (and simple) scenario.
> 
> Say that you have a function loadit(S) from module M that takes a
> string S, presumably a pathname, and does a lot of work then returns
> object x of type X.  Leaving loadit bare produces lots of duplicates of
> object x as different parts of client-code try to load S, and thus,
> performance (as measured by profiling, of course) is unacceptable.

Well, this is pretty much the idea, generator functions and all that. I'm 
really pretty clueless about how to go about profiling python code though, 
what do you use to do that?

> Memoizing in the simplest way, e.g (2.2 syntax for concision &c):
> [Examples...]

Memoizing makes sense, but what is concision? Niether python.org or 
dictionary.com help...


> I hope the basic idea, at least, is clear, even though the code and
> design of this registry are likely to be substantially improvable.

Yes, i think it all makes sense, though i'll wait untill i'm fully awake to 
try and fully understand it =)

 
> Is this worth it?  Only under truly extreme needs of performance
> according to the above-sketched scenario, of course.  But, if one
> really must...

Hm, well i'm not going for performance per se, although it's not something 
i wish to impede. The main thing i'm worried about is the possibility of 
dealing with lots and lots of different things like this, and moving them 
in and out of 'scope', as it were, over a long running period. But really, 
this is my first real attempt at any kind of real 'resource management', 
and my needs are simpler at this time. I'm just trying to understand where 
i'm going with this kind of a thing. The ultimate evolution of this project 
to the state that I can only imagine would probably require advanced or new 
methods of managing things like this, but... that's a long long way away if 
it's anywhere :)

Ok, without further polish, here's my current plan. I'm going to use a 
customized dictionary under py2.2. This dictionary, when passed a key for a 
resource not already available, will load it out of the file (key ~= file). 
So let's say we're making a monster, and we need the 3d representation for 
it. We write something like "my_rep = tdrep.reps['rep/mon/goblin']". That's 
not available, so the reps dictionary loads 'data/rep/mon/goblin.xml' (or 
whatever). This contains the name of the generator function to use and a 
dictionary of paramters for that generator. The generator also looks up the 
required resourses in the dictionarys for textures, materials, meshes, and 
what have you, which results in futher automatic loading if necessary. I'm 
not even going to worry about deleting things at the time being, except 
maybe for the simple case of 'purge every damn thing and start from 
scratch'.

Thanks for the most enlighening responses, and any input on this method is 
fine as well. =)

-- 
Philip Sw "Starweaver" [rasx] :: <nothing>
*Waiting to see a zork/enchanter referece in Harry Potter... not going to 
happen, but a frotz or so would fit in so well..."