[omaha] FeedJack

Fri Mar 15 14:41:51 CET 2013

As a wise man once said: It's all talk til the code runs.

After I looked at the feedjack_update.py, I started to do a thinking and
research, and my thoughts were like like Jeff's stream of consciousnesses.
The main thing I came up with is that there is no open-source RSS Reader.
Not just in Django, but period. FeedJack isn't a reader as much as a
aggregator: it displays the contents of feeds and does some manipulation.
There are quite a few of those, but no projects that I can find that has a
user authentication for them to manage their own subscriptions and what
they have read.

Example: Bob and Susan both subscribe to Feed1. Bob has caught up on Feed1
so his articles are marked as read while Susan hasn't logged on in a while,
so hers are not. FeedJack (nor any other open source solution) does this.

I'm going to agree with what Jeff was hinting at: keep FeedJack's models
and extend from that. I honestly don't like to design too far ahead of the
game, but I also understand that you have to have a roadmap of what to use.
Honestly there are a lot of problems to solve before approaching  the
front-end.

Big items:
   o Only seeing the feeds that you are subscribed to
   o Searching
   o fetching (this could be a cronjob for the time being)

Libraries I have thought about:
  o django-orm-extensions: Easy way interface PostgreSQL full text search
(which I would known about this before)
  o django-user-accounts: handles sign-ups, password resets, etc. I would
actually prefer OpenID/Google/Facebook ID integration instead. Who needs
another password to deal with?

Thanks for letting me ramble . . .

On Thu, Mar 14, 2013 at 11:37 PM, Jeff Hinrichs - DM&T
<jeffh at dundeemt.com>wrote:

> Well on the plus side, it is using feedparser, and etags but on the
> downside ...
>
> I don't see where it setting the last-modified header when it goes after a
> URI. The comments make it seem that it does, but I didn't see it in the
> code until after the fetch, which is too late to save you and the publisher
> any bandwidth.  (feedparser looks to be a good citizen, but you have to
> tell it what it needs to know to act intelligently)
>
> It should be written as a django management command --- maybe  it is but I
> just don't recognize the old form.   It is using threads -- ick. I would
> seriously think about ripping it out, use multiprocessing or messaging / mq
> to give it some scalability.  The way it sits, it doesn't appear to be
> useful for more than 1 user.
>
> I'd break it out to a command / fetcher / bizlogic-dbwriter structure.
>  Then you could spool up as many fetchers (feedparser + data) as you need,
> which then hand off to a bizlogic-writer pool.  Do it right and you could
> scale from a db based queue to a real mq if you were going to go big.
>
> If going from scratch, I'd keep the model and client view separated via
> json comm only.  Lean towards an AngularJS front-end and do the heavy
> back-end lifting with python.  Django would be exploited for it's orm and
> middle wear for accounts, auth and access.
>
> I like hypothetical solutions to problems, it's more fun than even writing
> tests.  :)
>
> The other conceptual problem that I am having, is size.  I like the idea of
> everyone having their own, but in reality it is going to be at least small
> groups on the low end (like Mike was saying about him and his wife.  I have
> a similar situation here but with 4 people.)
>
> Although it is php based, wordpress as a personal feed reader does make
> sense in the abstract.  Not that I'm a fan of wordpress, but it is a
> widespread solution, supported by many hosting providers.  That solves a
> number of problems for wide adoption.
>
> On more wish for my reader -- find other similar stories in feeds.  How
> many times have you see essentially the same store in multiple feeds?  Ever
> wish you could filter to them and mark them all read at once?  I do.
>
> A question, how do you read your articles?  I tend to group mine by topic:
>  Python, Hard Tech, Soft Tech, Local News & Sports, etc.  I have coworkers
> who strictly do the "river" always looking at all.  For me, that is just
> too chaotic.  Also, some feeds just get stale faster than others.  Soft
> Tech and Local news -- If I don't have the time, I just mark all as read
> and move on.  While Python and Hard Tech, I will leave until I have some
> real time and those I give a harder look.  What about you?
>
> Best,
>
> Jeff
>
>
> On Thu, Mar 14, 2013 at 7:49 PM, Mike Hostetler
> <mike at squarepegsystems.com>wrote:
>
> > On Mar 14, 2013 6:04 PM, "Matthew Nuzum" <newz at bearfruit.org> wrote:
> > >
> > >
> > > On Mar 14, 2013, at 1:29 PM, Mike Hostetler wrote:
> > >
> > > > And now I looked at feedjack_update.py. Holy crap
> > > >
> > > >
> >
> >
> https://github.com/squarepegsys/feedjack/blob/master/feedjack/bin/feedjack_update.py
> > >
> > > Looks like a good place to have some fun learning how to refactor!
> > >
> > >
> >
> > Yup, certainly. First thing is to write a unit test.
> >
> > _______________________________________________
> > > Omaha Python Users Group mailing list
> > > Omaha at python.org
> > > http://mail.python.org/mailman/listinfo/omaha
> > > http://www.OmahaPython.org
> > _______________________________________________
> > Omaha Python Users Group mailing list
> > Omaha at python.org
> > http://mail.python.org/mailman/listinfo/omaha
> > http://www.OmahaPython.org
> >
>
>
>
> --
> Best,
>
> Jeff Hinrichs
> 402.218.1473
> _______________________________________________
> Omaha Python Users Group mailing list
> Omaha at python.org
> http://mail.python.org/mailman/listinfo/omaha
> http://www.OmahaPython.org
>

-- 
Mike Hostetler
SquarePeg Systems
http://www.squarepegsystems.com