[SciPy-user] Efficient update of dictionary holding arrays

[Jose Gómez-Dans]

Hi,
This is probably something that is very python-centric and not very 
scipy-centric, but maybe the folk here can shed light on it. I am storing 
numpy arrays in dictionaries. The keys to such dicionaries are tuples, and 
hence fairly complex. The dictionaries are huge (they are around 100000 
elements, but potentially I'd like to process around 6M elements). Each 
dictionary element, in itself, points to another dictionary, with some 10 
keys, where each of them is a numpy array. 

What I want to achieve is to "trim" these numpy arrays (remove the first TRIM 
elements for all the arrays). My attempt goes like this:

for k in self.data_dictionary.iterkeys():
   for w in self.data_dictionary[k].iterkeys():
		if w<>'array2d':
			self.data_dictionary[k][w] = self.data_dictionary[k][w][TRIM:]
		else:
			for b in xrange(7):
				self.data_dictionary[k][w][b] = self.data_dictionary[k][w][b][TRIM:]

This is very slow. I guess that I have two nested loops, and then a further 
test. Is there some way to speed it up?

Thanks!
Jose