problem with netCDF4 OpenDAP

Fri Aug 14 11:26:27 EDT 2015

On 08/14/2015 03:15 PM, Jason Swails wrote:
>
>> On Aug 14, 2015, at 3:18 AM, Tom P <werotizy at freent.dd> wrote:
>>
>> Thanks for the reply but that is not what the documentation says.
>>
>> http://unidata.github.io/netcdf4-python/#section8
>> "Remote OPeNDAP-hosted datasets can be accessed for reading over http if a URL is provided to the netCDF4.Dataset constructor instead of a filename. However, this requires that the netCDF library be built with OPenDAP support, via the --enable-dap configure option (added in version 4.0.1).”
>
> Huh, so it does.  Your error message says "file not found", though, which suggested to me that it's trying to interpret the NetCDF file as a local file instead of a URL.  Indeed, when I run that example, the traceback is more complete (the traceback you printed had omitted some information):
>
>>>> netCDF4.Dataset('http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc')
> syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
> context: <!DOCTYPE^ HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL /pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc.dds was not found on this server.</p></body></html>
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "netCDF4/_netCDF4.pyx", line 1547, in netCDF4._netCDF4.Dataset.__init__ (netCDF4/_netCDF4.c:9551)
> RuntimeError: NetCDF: file not found
>
> So it’s clear that netCDF4 is at least *trying* to go online to look for the file, but it simply can’t find it.  Since the docs say it’s linking to libcurl, I tried using curl to download the file (curl -# http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc > test.nc) and it worked fine.  What’s more, it *seems* like the file (/pub/.../ersst.201507.nc.dds) was decorated with the ‘.dds’ suffix for some reason (not sure if the server redirected the request there or not).  But this looks like a netCDF4 issue.  Perhaps you can go to their project page on Github and file an issue there -- they will be more likely to have your answer than people here.
>
> HTH,
> Jason

Hi,
   yes the file does appear to be there, I can download it and I can 
open and read the URL using urllib. Since there are a whole bunch of 
files in the directory, I really need MFDataset, but according to the 
documentation that doesn't work with URLs. Maybe the solution really is 
to D/L them all into a temporary folder and use MFDataset.

>
>>
>> and for the Anaconda package -
>> http://docs.continuum.io/anaconda/changelog
>> "2013-05-08: 1.5.0:
>> Highlights:
>>   updates to all important packages: python, numpy, scipy, ipython, matplotlib, pandas, cython
>>   added netCDF4 (with OpenDAP support) on Linux and MacOSX"
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
> --
> Jason M. Swails
> BioMaPS,
> Rutgers University
> Postdoctoral Researcher
>