Django broken pipe error

[justin walters]

On Wed, Dec 7, 2016 at 1:08 AM, <dr.roman.graf at gmail.com> wrote:

> Thank you Justin,
>
> I'm on the dev server and should present results in this way.
>
> Yes, I use manage.py runserver --insecure to start the server (from
> PyCharm).
>
> My views.py call:
>
> @detail_route(methods=['post'], permission_classes=[permissions.AllowAny])
> def export_report(request):
>     body_unicode = request.body.decode('utf-8')
>     body_str = body_unicode.encode('ascii','ignore')
>     attr_list = body_str.split('&')
>     attr_dict = {}
>     if (len(attr_list) > 0):
>         for attr in attr_list:
>            ...
>            key = key_value_pair[0]
>            attr_dict[key] = key_value_pair[1]
>     trend = trends.calculate_trend(
>         attr_dict['search_phrase']
>         , attr_dict['time_from']
>         , attr_dict['time_to']
>         , attr_dict['time_scale']
>     )
>     attr_dict['trend'] = trend
>     res = str(json.dumps(attr_dict))
>     return HttpResponse(res, content_type="text/plain; charset=utf-8")
>
> and trend calculation in trends.py with database calls:
>
> def calculate_trend(query_phrase, time_from, time_to, time_scale):
>     # check in database if trend already exists
>     try:
>         db_trend = Trend.objects.get(pk=query_phrase)
>         if db_trend.from_time.strftime("%Y-%m-%d") == time_from \
>                 and db_trend.to_time.strftime("%Y-%m-%d") == time_to \
>                 and db_trend.granularity == time_scale:
>             logger.info("trend already exists.")
>             existing_trend_dict = ast.literal_eval(db_trend.content)
>             return existing_trend_dict
>     except Trend.DoesNotExist:
>         logger.info("It is a new trend search.")
>     trend_dict = {}
>     start_time = pd.Timestamp(value[0])
>     end_time = pd.Timestamp(value[-1])
>     freq = ... get frequency using pandas lib
>     trend_dict[key] = freq
>     json_trend_content = trend_dict_to_sorted_json_str(trend_dict)
>     trend = Trend(
>         phrase=query_phrase,
>         content=json_trend_content,
>         from_time=time_from,
>         to_time=time_to,
>         granularity=time_scale,
>     )
>     if trend is not None:
>         try:
>             db_trend = Trend.objects.get(pk=query_phrase)
>             db_trend.delete()
>             logger.info("delete old trend: %s. " % trend)
>         except Trend.DoesNotExist:
>             logger.info("create trend: %s. " % trend)
>         trend.save()
>     return trend_dict
>
> Thank you in advance!
>
> Roman
> --
> https://mail.python.org/mailman/listinfo/python-list
>


It looks like you can probably get rid of the try/except block at the end
of the calculate_trend
method as any existing Trend object will have already been caught in the
first try/except block.

>From what I'm reading here:
http://stackoverflow.com/questions/11866792/how-to-prevent-errno-32-broken-pipe
,
this issue can be caused by the client closing the connection before
sendall() finishes writing. Can you estimate
how long it takes for a request to this endpoint takes to resolve? If it's
a long time(maybe due to the pandas call?),
your browser/client may be timing out. It could be because it takes a while
for the Db to find an existing Trend object
as well.

I can't give you any advice on how to fix it exactly, but I can tell you
what the problem is: The client is closing the
connection before socket.sendall() has finished writing to the socket. My
guess is that the calculate_trend() method
takes a long time to complete and the client is timing out.