[SciPy-User] pylab

[Benjamin Root]

On Tue, Jul 20, 2010 at 12:24 PM, <PHobson at geosyntec.com> wrote:

> > > Ben and Pierre,
> > >
> > > Thanks for the tips! Glad I chimed in here b/c I've definitely learned
> > something. I'm often pulling data out from a database, so null records
> > come back to me as None's. I'll be sure to set a CASE statement now that
> > fills the NULLs in with an obviously junk value that I can mask from now
> > on.
> > >
> > > -paul
>
> > Why not user your None as said junk value?
> >
> > --Josh
>
> All my existing routines just throw values from the database (Postgres or
> MS SQL) cursor directly into numpy arrays. As Pierre pointed out, the None
> values force the array's dtype to object.
>
> In [40]: x
> Out[40]: array([2.5, 12.2, 5, None], dtype=object)
>
> I could easily mask the Nones, but switching it over to a value such as
> -99999 would let it me keep the dtype as float or integer. Probably not a
> big deal at the moment, but I'm trying to adopt as many best practices as
> possible so nothing comes back to bite me later on. (Negative values are
> pretty rare in environmental data, so I think it's a safe bet).
> -paul
>
>
As a general rule, magic values can be very tricky to work with, and it is a
good idea to think out conventions.  I was just bitten by a bug where a
previous developer wanted coordinate values of exactly 0.0 (yes, floating
point) to mean something special, but had no constraints for the domain.
 So, once in a blue moon, a data point would get processed differently than
all the others, causing slightly odd analysis results.

So, my personal rule is to use NaNs whenever/where-ever possible because if
they sneak into a calculation unexpectedly, they will usually propagate such
that your program breaks -- which is a good thing.  Bad results that appear
to be good is a bad thing.  When you have to do integers, pick a magic value
that is impossible (e.g., negative value for temperature in Kelvins), and
document that magic value.

I hope this is helpful.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100720/06e3f01d/attachment.html>