Convert string to command..

[Bruno Desthuilliers]

sjdevnull at yahoo.com a écrit :
> On Oct 18, 1:38 pm, Bruno Desthuilliers <bruno.
> 42.desthuilli... at wtf.websiteburo.oops.com> wrote:
>> Abandoned a écrit :
>> (snip)
>>
>>> I'm very confused :(
>>> I try to explain main problem...
>>> I have a table like this:
>>> id-1 | id-2 | value
>>> 23     24       34
>>> 56     68       66
>>> 56     98       32455
>>> 55     62       655
>>> 56     28       123
>>> .... ( 3 millions elements)
>>> I select where id=56 and 100.000 rows are selecting but this took 2
>>> second. (very big for my project)
>> Not to bad in the absolute.
>>
>>> I try cache to speed up this select operation..
>>> And create a cache table:
>>> id-1 | all
>>> 56    {68:66, 98:32455, 62:655}
>> I really doubt this is the right way to go.
>>
>>> When i select where id 56 i select 1 row and its took 0.09 second but
>>> i must convert text to dictionary..
>>> Have you got any idea what can i do this conver operation ?
>> Other alread answered
>>
>>> Have you got any idea what can i do cache for this table ?
>> Depends on your RDBMS. And as far as I'm concerned, I'd start by trying
>> to find out how to optimize this query within the RDBMS - good ones are
>> usually highly optimized softwares with provision for quite a lot of
>> performance tuning.
> 
> Just the overhead of the query is a killer compared to a dictionary
> lookup in Python, even if all you're doing is selecting an integer
> from a 1-row, 1-column table.

Indeed. But then why use a RDBMS at all ? Please understand that I'm not 
saying that a RDBMS will beat a plain dict lookup not that a RDBMS will 
solve world's problem, but that storing pickled Python's dicts into a 
RDBMS is certainly not the best thing to do. It will *still* have the db 
connection overhead anyway, and will be a nightmare to maintain in sync 
with the real state of the db. Which is why I suggest *first* looking 
for RDBMS-side tuning and optimization - which may include third-part 
cache systems FWIW.