__import__ fails randomly due to missing file

[Arkadiusz Bulski]

I inserted importlib.invalidate_caches() instead of os.sync() and at a
glance, it seems to have fixed it.

Big thanks Paul!


pon., 12 mar 2018 o 11:00 użytkownik Paul Moore <p.f.moore at gmail.com>
napisał:

> On 12 March 2018 at 09:12, Arkadiusz Bulski <arek.bulski at gmail.com> wrote:
> > Hi,
> >
> > For the record, I am the developer of Construct library. And the question
> > relates to it.
> > https://construct.readthedocs.io/en/latest/#
> >
> > I have an issue of Travis failing randomly at test cases. The cause is
> > always __import__ failing to import a dynamically generated code. Library
> > generates a string, saves it to a file (closes the file too), then
> > __import__ it. I dont think there is anything wrong with the (generated
> > imported) code itself. It seems like its some sort of issue with the
> Python
> > runtime (or filesystem) not detecting the file because it was created
> only
> > a moment ago. I tried to solve it by adding os.sync() between writing the
> > file and importing it, but it seems to have only decreased the error
> rate,
> > not solved it. Frankly, this is a heisenbug. Re-running the Travis job
> > usually fixes it. Its random and happens very rarely. I havent pinpointed
> > it to any particular version of CPython or Pypy (although I dont remember
> > it ever happening on Pypy).
> >
> > An example:
> > https://travis-ci.org/construct/construct/jobs/352213556
> >
> > The code in question:
> >
> https://github.com/construct/construct/blob/d9c645ed0a75b5a2b64318113cf69cac822fd8f5/construct/core.py#L402-L414
> >
> https://github.com/construct/construct/blob/d9c645ed0a75b5a2b64318113cf69cac822fd8f5/construct/core.py#L179-L180
>
> The import system caches os.stat information, so it's possible that if
> you're creating the file and then relatively quickly doing the import,
> there may be some cache effect going on (granularity of file
> modification timestamps?) I don't know the details, but it might be
> worth looking in that area...
>
> Paul
>
-- 
~ Arkadiusz Bulski
~ tel 503 357 111 lub WhatsApp/Telegram