efficient way to process data

[Chris Angelico]

On Wed, Jan 15, 2014 at 1:18 AM, Larry Martell <larry.martell at gmail.com> wrote:
> if you're interested in what the application is, this is data
> collected with an electron microscope from semiconductor wafers as
> they are being manufactured. The x and y are the position on the wafer
> that the data was collected, in microns. If 2 data points are
> collected within 1 micron of each other they need to be combined when
> being analyzed.

As far as I'm concerned, you won geek cred the moment you said
"electron microscope", and "semiconductor wafers as they are being
manufactured" is just gravy I don't suppose you want to hire another
programmer? :)

Do you actually mean here that the two points need to be within 1
micron, or that data gets combined if it's nearby in *either*
coordinate? There are libraries for figuring out if two things are
near each other - I'm not 100% sure, but you might be able to do this
inside PostgreSQL (though that just gets back to the previous rule:
can't move off MySQL). Treat every data point as a circle or square,
and then look for overlap.

ChrisA