multiple processes, private working directories

[Michael Palmer]

On Sep 25, 8:16 am, "Tim Arnold" <tim.arn... at sas.com> wrote:
> "Tim Arnold" <a_j... at bellsouth.net> wrote in message
>
> news:57cdd3f1-cde8-45f5-b9d9-3eef32010bda at l43g2000hsh.googlegroups.com...
>
> >I have a bunch of processes to run and each one needs its own working
> > directory. I'd also like to know when all of the processes are
> > finished.
>
> Thanks for the ideas everyone--I now have some news tools in the toolbox.
> The task is to use pdflatex to compile a bunch of (>100)  chapters and know
> when the book is complete (i.e. the book pdf is done and the separate
> chapter pdfs are finished. I have to wait for that before I start some
> postprocessing and reporting chores.
>
> My original scheme was to use a class to manage the builds with threads,
> calling pdflatex within each thread. Since pdflatex really does need to be
> in the directory with the source, I had a problem.
>
> I'm reading now about python's multiprocessing capabilty, but I think I can
> use Karthik's suggestion to call pdflatex in subprocess with the cwd set.
> That seems like the simple solution at this point, but I'm going to give
> Cameron's pipes suggestion a go as well.
>
> In any case, it's clear I need to rethink the problem. Thanks to everyone
> for helping me get past my brain-lock.
>
> --Tim Arnold

I still don't see why this should be done concurrently? Do you have >
100 processors available? I also happen to be writing a book in Latex
these days. I have one master document and pull in all chapters using
\include, and pdflatex is only ever run on the master document. For a
quick preview of the chapter I'm currently working on, I just use
\includeonly - compiles in no time at all.

How do you manage to get consistent page numbers and cross-referencing
if you process all chapters separately, and even in _parallel_ ? That
just doesn't look right to me.