[melbourne-pug] mirror app [Was: Project time?]

Sat Feb 6 07:12:09 CET 2010

Google's visualisation API is actually quite good, and we have used it
before when presenting some complex data to DPCD - in particular, the Motion
Charts<http://code.google.com/apis/visualization/documentation/gallery/motionchart.html>interface
allows up to 5 dimensions of data to be displayed at once: one on
the time axis, two on the X/Y axis (which need to be continuous or at least
numeric dimensions), one which defines each ball's relative size (also
numeric) and another for each ball's colour (which may be a discrete
dimension). With no programming beyond formatting the input data, end-users
can fish for relationships with only minimal training. Patterns which would
never be seen in tables or static graphs can be divined very quickly when
movement and colour are used...

<IMHO>
My two cents would be to build a web service which acts as a central
catalogue of the available data sources, along with some metadata on each
one. Then use client-side code (javascript) to connect directly to the
datasources. The more standards-compliant the organisations are with their
data, the easier it will be for the clients to use it. This is similar to
OpeNDAP, and the OGC geospatial web services such as WMS/WFS.

This approach would also encourage other government agencies to contribute,
as the overheads are very low - just provide access to the data using one of
a number of supported protocols. No need for them to build a front-end, and
no need to advertise it; it will immediately appear in the list of available
data sources.

Python is likely a good tool for the central web service, and may also be
useful for the data providers. When it comes to the core work here though,
I'm not sure that python is the best language, unless we'll use something
like pyjamas to convert python code to something which can run client-side
in browsers. To provide a tool for working with all this heterogeneous data,
and potentially more in the future, it would be unwise to try to cherry-pick
a subset of the data and discard the rest as useless. Let the users decide
for themselves.
</IMHO>

nick

2010/2/6 Mike Dewhirst <miked at dewhirst.com.au>

> On 5/02/2010 3:36pm, Irae Hueck Costa wrote:
>
>>
>>
>> How about a little brainstorming and then we meet at some McDonalds?
>> Here an idea:
>> Feed the data into an artificial neural network (and make the found
>> patterns usefull for the everyday use?).
>>
>
> Let's accept that the data sets currently available are less than great. If
> we assume (invent) some good data then we can write a winning application.
>
> We need to look past current datasets. If what we do is compelling enough
> it will eventually force the proper collection of useful data just so our
> application can reveal insights otherwise unattainable.
>
> I had discussions in 2007 with the Vic government about exactly this
> requirement. We agreed then that the objective should be to correlate
> information so that it is possible for policy makers to adjust something (a
> law, a tax, a handout, whatever) with a predicted outcome and for the system
> to demonstrate success or failure down the track.
>
> For example, let's say water consumption is particularly low in a
> partularly crime-ridden area. A policy maker might ease watering
> restrictions there with the expected outcome of greener gardens hoping
> community morale might pick up and the crime rate may drop. BTW if it worked
> it might have nothing to do with morale but rather the increase in casual
> employment for young gardeners.
>
> All of that is just ordinary. What we really need is the equivalent of
> time-lapse photography and data collected in lots of (de-identified) streams
> which are tied to geographic locations.
>
> Patterns of information in the environment indicate trends in society. We
> just need to see them in context to draw the correct insights. Remember New
> York's zero tolerance of graffiti co-inciding with a drop in homicides?
>
> So, I think our competition entry should be an API for a data visualisation
> app. Or a wrapper for an existing data visualisation tool. I want to see
> "time-lapse" mountains and valleys flowing across the screen so our policy
> makers can spot connections, freeze the frame, drill down and ponder. A
> mirror on society.
>
> We should be able to assume the existence of the NBN and widespread VPN
> access to databases hither and yon. For the competition we can manufacture
> whatever data we want for demonstration (actually testing) purposes.
>
> Mike
>
>
>
>  I dont think that this will give better results than other stochastic
>> methods, but it sounds fancy.
>>
>> Here some shorter python specific introduction to neural networks:
>> http://www.ibm.com/developerworks/library/l-neural/
>>
>> ps:
>> Maybe we should use some other way to communicate about this project so
>> we don`t flood the email incomming box of people who are not interested.
>>
>>  > From: r1chardj0n3s at gmail.com
>>  > Date: Thu, 4 Feb 2010 10:05:44 +1100
>>  > To: melbourne-pug at python.org
>>  > Subject: [melbourne-pug] Project time?
>>  >
>>  > Hi all,
>>  >
>>  > So this competition was just announced:
>>  >
>>  > http://www.premier.vic.gov.au/component/content/article/9300.html
>>  >
>>  > The short story being:
>>  >
>>  > 1. take 50 datasets like
>>  > • Maps of local hospitals (http://www.health.vic.gov.au/maps/index.htm
>> )
>>  > • Crime statistics (http://www.vicpolicenews.com.au/myplace.html)
>>  > • Water resources data (http://www.vicwaterdata.net/)
>>  > 2. crank up a Pinax website (allowing us to focus on point #3)
>>  > 3. present information from #1 in "interesting" ways
>>  > 4. profit!
>>  >
>>  > So, who's interested?
>>  >
>>  >
>>  > Richard
>>  >
>>  > _______________________________________________
>>  > melbourne-pug mailing list
>>  > melbourne-pug at python.org
>>  > http://mail.python.org/mailman/listinfo/melbourne-pug
>>
>> ------------------------------------------------------------------------
>> Quer navegar na Internet sem medo? Instale grátis o Internet Explorer 8.
>> <http://go.microsoft.com/?linkid=9707132>
>>
>>
>>
>> _______________________________________________
>> melbourne-pug mailing list
>> melbourne-pug at python.org
>> http://mail.python.org/mailman/listinfo/melbourne-pug
>>
>
> _______________________________________________
> melbourne-pug mailing list
> melbourne-pug at python.org
> http://mail.python.org/mailman/listinfo/melbourne-pug
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/melbourne-pug/attachments/20100206/0ae5de20/attachment-0001.htm>