[melbourne-pug] django db race conditions

[Richard Jones]

Ahem, my apologies. On re-reading your email not on a tiny phone I see
that's an option you looked into. I'd still advocate use of database
constraints, even if it means modifying your data model to make a multiple
column uniqueness constraint if one column is not enough (because if it's
not enough then something else in your data model should say why you're not
allowed to have two values created simultaneously.)


     Richard



On 16 October 2013 13:13, Richard Jones <r1chardj0n3s at gmail.com> wrote:

> Have you considered using a database constraint?
>
> Sent from my mobile - please excuse any brevity.
> On 16 Oct 2013 12:27, "Brian May" <brian at microcomaustralia.com.au> wrote:
>
>> Hello All,
>>
>> I have a reasonable amount of Django code that follows this general model:
>>
>> try:
>>    object = Model.objects.get(name="woof")
>> except Model.DoesNotExist:
>>    object = Model()
>>    init_object(update)
>>    object.save()
>>
>> Or, in some cases:
>>
>> object = Model.objects.get_or_create(name="woof")
>>
>> In both cases the resultant code is very similar.
>>
>> In both cases there is a race condition. Depending on the flow of
>> execution, I can end up with two or more db objects with name="woof". There
>> are many forum posts discussing this race condition.
>>
>> As an example, for the first case happens when displaying a webpage. Lets
>> assume init_object() is relatively slow. As the web page takes a while to
>> load, the user clicks reload. This results in two (or more) objects being
>> created with name="woof" in error.
>>
>> Another example, for the second case occurs when a JavaScript app makes
>> concurrent calls to the web service.
>>
>> Some people have suggested that if I I want name to be unique, I should
>> make it a database constraint. However that is not always the case that I
>> want these values to be strictly unique, I just want to reuse an existing
>> entry or create it if it doesn't exist. Also, the database constraint would
>> mean the code fails instead of committing two objects, which is not really
>> helpful.
>>
>> Other people have suggested locking the db table, while doing the
>> get_or_create. Seems to require possible db specific SQL code, am I bit
>> reluctant to do this.
>>
>> Django's select_for_update method is interesting, however as the object
>> doesn't actually exist yet, not really applicable.
>>
>> Another solution I have considered, at least for some cases, is
>> moving init_object to a celery task. This would provide the user with
>> faster feedback as to what is happening, and for some slow tasks is
>> probably a good thing.  Ideally I would only want one task to initialize
>> the object, not sure how I would check this without introducing new race
>> conditions very similar to the one I am trying to remove. e.g.:
>>
>> if task not created:
>>     create task
>>
>> In theory create task could be called multiple times.
>>
>> Another solution, that would work in some places is to make sure that the
>> object exists by some other means beforehand. So I can safely do a get
>> instead of a get_or_create.
>>
>> Any other ideas?
>>
>> Quite possibly I will have to try and find a solution on a case by case
>> basis :-(.
>>
>> Shame we didn't realize this before we wrote this code.
>> --
>> Brian May <brian at microcomaustralia.com.au>
>>
>> _______________________________________________
>> melbourne-pug mailing list
>> melbourne-pug at python.org
>> https://mail.python.org/mailman/listinfo/melbourne-pug
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/melbourne-pug/attachments/20131016/1c8d8150/attachment-0001.html>