[omaha] FeedJack

[Jeff Hinrichs - DM&T]

Well on the plus side, it is using feedparser, and etags but on the
downside ...

I don't see where it setting the last-modified header when it goes after a
URI. The comments make it seem that it does, but I didn't see it in the
code until after the fetch, which is too late to save you and the publisher
any bandwidth.  (feedparser looks to be a good citizen, but you have to
tell it what it needs to know to act intelligently)

It should be written as a django management command --- maybe  it is but I
just don't recognize the old form.   It is using threads -- ick. I would
seriously think about ripping it out, use multiprocessing or messaging / mq
to give it some scalability.  The way it sits, it doesn't appear to be
useful for more than 1 user.

I'd break it out to a command / fetcher / bizlogic-dbwriter structure.
 Then you could spool up as many fetchers (feedparser + data) as you need,
which then hand off to a bizlogic-writer pool.  Do it right and you could
scale from a db based queue to a real mq if you were going to go big.

If going from scratch, I'd keep the model and client view separated via
json comm only.  Lean towards an AngularJS front-end and do the heavy
back-end lifting with python.  Django would be exploited for it's orm and
middle wear for accounts, auth and access.

I like hypothetical solutions to problems, it's more fun than even writing
tests.  :)

The other conceptual problem that I am having, is size.  I like the idea of
everyone having their own, but in reality it is going to be at least small
groups on the low end (like Mike was saying about him and his wife.  I have
a similar situation here but with 4 people.)

Although it is php based, wordpress as a personal feed reader does make
sense in the abstract.  Not that I'm a fan of wordpress, but it is a
widespread solution, supported by many hosting providers.  That solves a
number of problems for wide adoption.

On more wish for my reader -- find other similar stories in feeds.  How
many times have you see essentially the same store in multiple feeds?  Ever
wish you could filter to them and mark them all read at once?  I do.

A question, how do you read your articles?  I tend to group mine by topic:
 Python, Hard Tech, Soft Tech, Local News & Sports, etc.  I have coworkers
who strictly do the "river" always looking at all.  For me, that is just
too chaotic.  Also, some feeds just get stale faster than others.  Soft
Tech and Local news -- If I don't have the time, I just mark all as read
and move on.  While Python and Hard Tech, I will leave until I have some
real time and those I give a harder look.  What about you?

Best,

Jeff


On Thu, Mar 14, 2013 at 7:49 PM, Mike Hostetler
<mike at squarepegsystems.com>wrote:

> On Mar 14, 2013 6:04 PM, "Matthew Nuzum" <newz at bearfruit.org> wrote:
> >
> >
> > On Mar 14, 2013, at 1:29 PM, Mike Hostetler wrote:
> >
> > > And now I looked at feedjack_update.py. Holy crap
> > >
> > >
>
> https://github.com/squarepegsys/feedjack/blob/master/feedjack/bin/feedjack_update.py
> >
> > Looks like a good place to have some fun learning how to refactor!
> >
> >
>
> Yup, certainly. First thing is to write a unit test.
>
> _______________________________________________
> > Omaha Python Users Group mailing list
> > Omaha at python.org
> > http://mail.python.org/mailman/listinfo/omaha
> > http://www.OmahaPython.org
> _______________________________________________
> Omaha Python Users Group mailing list
> Omaha at python.org
> http://mail.python.org/mailman/listinfo/omaha
> http://www.OmahaPython.org
>



-- 
Best,

Jeff Hinrichs
402.218.1473