Locking around

[Nikolaus Rath]

Tobiah <toby at tobiah.org> writes:
> On Mon, 04 Aug 2008 15:30:51 +0200, Nikolaus Rath wrote:
>
>> Hello,
>> 
>> I need to synchronize the access to a couple of hundred-thousand
>> files[1]. It seems to me that creating one lock object for each of the
>> files is a waste of resources, but I cannot use a global lock for all
>> of them either (since the locked operations go over the network, this
>> would make the whole application essentially single-threaded even
>> though most operations act on different files).
>
> Do you think you could use an SQL database on the network to
> handle the locking?

Yeah, I could. It wouldn't even have to be over the network (I'm
synchronizing access from within the same program). But I think that
is even more resource-wasteful than my original idea.

> Hey, maybe the files themselves should go into blobs.

Nope, not possible.  They're on Amazon S3.

Best,

   -Nikolaus

-- 
 »It is not worth an intelligent man's time to be in the majority.
  By definition, there are already enough people to do that.«
                                                         -J.H. Hardy

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C