[AstroPy] Level ODF downloads using the astroquery.XMMNewton module do not progress when run from a docker container.

[Peter Dzwig]

Try the obvious things as well if you haven't done so already. (I am
sure that you will have)

Run

sudo apt-get update
sudo apt-get upgrade

or whatever you use for updating.

Check that you have the newest version (or at least a recent version) of
astropy.

Peter

On 22/06/2021 10:43, Eduardo Ojero Pascual wrote:
> Dear Peter,
> 
> Thanks for your reply!
> 
> Indeed, it could be what you suggest, some type of obscure time-out,
> applicable to ipython and/or Jup. Notebook. In our case we have used
> ipython for the tests.
> 
> The XMM-Newton observation data files (ODFs) that I am attempting to
> download are small (~70 MB) compared to other data missions, e.g. SDSS
> or Gaia, etc. Thus, these files should not be so sensible to these
> time-outs as larger files.
> 
> Besides, I have already tested it using the requests from ipython, using
> directly the URL and getting the response into a binary file. It worked
> fine.
> 
> It seems that astroquery gets stuck where requests succeeds. Why?
> 
> Anyhow, I am going to try your suggestion and will let you know.
> 
> Thanks again!
> 
> Regards,
> 
> Eduardo
> 
> On 21/06/2021 21:32, Peter Dzwig wrote:
>> Eduardo,
>>
>> another possibility is Jupyter notebook/ipython itself.
>>
>> I too had an issue after going to 20.04 LTS.
>>
>> In my case it turned out to be a timeout issue caused by an internal
>> default message rate that was too low, principally with large (in my
>> case SDSS) datasets and/or records.
>>
>> I now invoke jupyter notebook using:
>>
>> $ jupyter notebook --NotebookApp.iopub.msg_rate_limit = 100000
>>
>> which resolved the issue for me.
>>
>> And, yes, I didn't get the issue in a terminal either.
>>
>> Hope that this helps.
>>
>> Regards,
>>
>> Peter
>>
>> PS I think the issue is highlighted somewhere on Ubuntu's own message
>> boards, or possibly on stackexchange somewhere. If it was here I
>> apologise to the community wholeheartedly ;-)
>>
>> On 21/06/2021 17:32, Eduardo Ojero Pascual wrote:
>>> Dear all,
>>>
>>> We are using the astroquery XMMNewton module to download ODF level data
>>> from a docker container running the latest Ubuntu 20.04 image.
>>>
>>> When invoking from within ipython the XMMNewton.download_data for any
>>> given ODF, as
>>>
>>>      In [x]: XMMNewton.download_data(odfid='nnnnnnnnnn', level='ODF',
>>> verbose=True)
>>>
>>>      INFO:
>>> http://nxsa.esac.esa.int/nxsa-sl/servlet/data-action-aio?obsno=nnnnnnnnnn&level=ODF
>>>
>>> [astroquery.esa.xmm_newton.core]
>>>
>>> the connection is effectively established but it does not progress.
>>> After a while, the connection is dropped but ipython does not return to
>>> its prompt.
>>>
>>> The same request used in a standard Ubuntu terminal, progresses
>>> apropriately and downloads the requested observation file.
>>>
>>> Is anyone having any idea on what is going on here?
>>>
>>> Best regards,
>>>
>>> Eduardo
>>>
>>>
>>> This message is intended only for the recipient(s) named above. It may
>>> contain proprietary information and/or
>>> protected content. Any unauthorised disclosure, use, retention or
>>> dissemination is prohibited. If you have received
>>> this e-mail in error, please notify the sender immediately. ESA applies
>>> appropriate organisational measures to protect
>>> personal data, in case of data privacy queries, please contact the ESA
>>> Data Protection Officer (dpo at esa.int).
>>>
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at python.org
>>> https://mail.python.org/mailman/listinfo/astropy
> 
> This message is intended only for the recipient(s) named above. It may
> contain proprietary information and/or
> protected content. Any unauthorised disclosure, use, retention or
> dissemination is prohibited. If you have received
> this e-mail in error, please notify the sender immediately. ESA applies
> appropriate organisational measures to protect
> personal data, in case of data privacy queries, please contact the ESA
> Data Protection Officer (dpo at esa.int).
> 
> 

-- 

Dr. Peter Dzwig