From riteshn at gmail.com Tue Oct 3 14:57:29 2017 From: riteshn at gmail.com (Ritesh Nadhani) Date: Tue, 3 Oct 2017 11:57:29 -0700 Subject: [Flask] Generic way of input filtering and output encoding to prevent XSS Message-ID: Hello I inherited a Flask + Angular app where all the APIs are basically API calls. As is usual, through some forms allow users to provide names for object and we save it to our DB which can be retreived back. This happens with multiple apis. E.g. HTTP POST: POST /api/v2/flags HTTP/1.1 .... Content-Type: application/json;charset=utf-8 Content-Length: 109 ... { "description":"foobar", "name":"test99", "permission":"NO_ACCESS", "type":"watchlist" } HTTP RESPONSE: HTTP/1.1 200 OK Server: nginx/1.6.2 .. Content-Type: application/json Content-Length: 1195 { "flags":[ { "description":"?}foobar{?", "end_time":9999999999000, "entities":0, "id":4, "name":"test99", "permission":"NO_ACCESS", "start_time":0, "type":"watchlist", "user_id":3 } ], "limit":100, "offset":0, "status":"Ok", "total":4 } ... we recently had a security inspection and this was deemed to be not properly encoding of output and vulnerable to XSS attack. >>> What is the canonical way to handle this? Since the name is used as reference and other place, ideally I would not like to keep encoded version into the DB. Is there a generic library that can encode the output before sending it back to the client? I suspect in that case UI will have to decode the data in every case to show the correct string to user? I tested against some website e.g. digitalocean and I noticed that if you enter such string in the form, its blocked by cloudflare itself. Since we dont use cloudflare, what is our other options? -- Ritesh From scott.werner.vt at gmail.com Tue Oct 3 17:50:32 2017 From: scott.werner.vt at gmail.com (Scott Werner) Date: Tue, 3 Oct 2017 17:50:32 -0400 Subject: [Flask] Generic way of input filtering and output encoding to prevent XSS In-Reply-To: References: Message-ID: Why is description being converted from "foobar" to ?}foobar{?"? Are you using flask jsonify when creating your API response? >>> import json >>> form_data = json.loads('{"description":"foobar"}') {'description': 'foobar'} >>> json.dumps(form_data) '{"description": "foobar"}' - You should serialize (dumping) and de-serialize (loading) using a library like https://marshmallow.readthedocs.io/en/latest/. - If the user can input html, like a rich editor, and you display it in Angular (not escaped), then you should sanitize it with a library like https://github.com/mozilla/bleach. - A quick search shows that you can also sanitize on the client side using angular: https://docs.angularjs.org/api/ngSanitize/service/$sanitize. In summary, never trust the user and always validate on the server. On Tue, Oct 3, 2017 at 2:57 PM, Ritesh Nadhani wrote: > Hello > > I inherited a Flask + Angular app where all the APIs are basically API > calls. As is usual, through some forms allow users to provide names > for object and we save it to our DB which can be retreived back. This > happens with multiple apis. > > E.g. > > HTTP POST: > POST /api/v2/flags HTTP/1.1 > .... > Content-Type: application/json;charset=utf-8 > Content-Length: 109 > ... > > { > "description":"foobar", > "name":"test99", > "permission":"NO_ACCESS", > "type":"watchlist" > } > > HTTP RESPONSE: > HTTP/1.1 200 OK Server: nginx/1.6.2 .. > Content-Type: application/json > Content-Length: 1195 > > { > "flags":[ > { > "description":"?}foobar{?", > "end_time":9999999999000, > "entities":0, > "id":4, > "name":"test99", > "permission":"NO_ACCESS", > "start_time":0, > "type":"watchlist", > "user_id":3 > } > ], > "limit":100, > "offset":0, > "status":"Ok", > "total":4 > } > > ... > > we recently had a security inspection and this was deemed to be not > properly encoding of output and vulnerable to XSS attack. > > >>> > > What is the canonical way to handle this? Since the name is used as > reference and other place, ideally I would not like to keep encoded > version into the DB. Is there a generic library that can encode the > output before sending it back to the client? I suspect in that case UI > will have to decode the data in every case to show the correct string > to user? > > I tested against some website e.g. digitalocean and I noticed that if > you enter such string in the form, its blocked by cloudflare itself. > Since we dont use cloudflare, what is our other options? > > > -- > Ritesh > _______________________________________________ > Flask mailing list > Flask at python.org > https://mail.python.org/mailman/listinfo/flask > -- Scott Werner scott.werner.vt at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From riteshn at gmail.com Tue Oct 3 18:11:52 2017 From: riteshn at gmail.com (Ritesh Nadhani) Date: Tue, 3 Oct 2017 15:11:52 -0700 Subject: [Flask] Generic way of input filtering and output encoding to prevent XSS In-Reply-To: References: Message-ID: It seems when I cut-n-paste into GMAIL it does its own stripping. The input output is exactly like you showed and its using flask jsonify. I will look into the libraries. Seems like we will have to go back and look into each API call to handle it correctly. On Tue, Oct 3, 2017 at 2:50 PM, Scott Werner wrote: > Why is description being converted from "foobar" to > ?}foobar{?"? Are you using flask jsonify when creating your API response? > >>>> import json >>>> form_data = >>>> json.loads('{"description":"foobar"}') > {'description': 'foobar'} >>>> json.dumps(form_data) > '{"description": "foobar"}' > > You should serialize (dumping) and de-serialize (loading) using a library > like https://marshmallow.readthedocs.io/en/latest/. > If the user can input html, like a rich editor, and you display it in > Angular (not escaped), then you should sanitize it with a library like > https://github.com/mozilla/bleach. > A quick search shows that you can also sanitize on the client side using > angular: https://docs.angularjs.org/api/ngSanitize/service/$sanitize. > > In summary, never trust the user and always validate on the server. > > On Tue, Oct 3, 2017 at 2:57 PM, Ritesh Nadhani wrote: >> >> Hello >> >> I inherited a Flask + Angular app where all the APIs are basically API >> calls. As is usual, through some forms allow users to provide names >> for object and we save it to our DB which can be retreived back. This >> happens with multiple apis. >> >> E.g. >> >> HTTP POST: >> POST /api/v2/flags HTTP/1.1 >> .... >> Content-Type: application/json;charset=utf-8 >> Content-Length: 109 >> ... >> >> { >> "description":"foobar", >> "name":"test99", >> "permission":"NO_ACCESS", >> "type":"watchlist" >> } >> >> HTTP RESPONSE: >> HTTP/1.1 200 OK Server: nginx/1.6.2 .. >> Content-Type: application/json >> Content-Length: 1195 >> >> { >> "flags":[ >> { >> "description":"?}foobar{?", >> "end_time":9999999999000, >> "entities":0, >> "id":4, >> "name":"test99", >> "permission":"NO_ACCESS", >> "start_time":0, >> "type":"watchlist", >> "user_id":3 >> } >> ], >> "limit":100, >> "offset":0, >> "status":"Ok", >> "total":4 >> } >> >> ... >> >> we recently had a security inspection and this was deemed to be not >> properly encoding of output and vulnerable to XSS attack. >> >> >>> >> >> What is the canonical way to handle this? Since the name is used as >> reference and other place, ideally I would not like to keep encoded >> version into the DB. Is there a generic library that can encode the >> output before sending it back to the client? I suspect in that case UI >> will have to decode the data in every case to show the correct string >> to user? >> >> I tested against some website e.g. digitalocean and I noticed that if >> you enter such string in the form, its blocked by cloudflare itself. >> Since we dont use cloudflare, what is our other options? >> >> >> -- >> Ritesh >> _______________________________________________ >> Flask mailing list >> Flask at python.org >> https://mail.python.org/mailman/listinfo/flask > > > > > -- > Scott Werner > scott.werner.vt at gmail.com -- Ritesh From jeff1evesque at yahoo.com Wed Oct 18 20:15:45 2017 From: jeff1evesque at yahoo.com (Jeffrey Levesque) Date: Wed, 18 Oct 2017 20:15:45 -0400 Subject: [Flask] Jeff Levesque: need help unit testing flask redis session Message-ID: <20E45963-0CE3-45ED-9C12-3A1B13D69A9A@yahoo.com> Hi guys, I've implemented Redis with my flask login session. I've verified that it all works: - https://github.com/jeff1evesque/machine-learning/pull/3073#issuecomment-337446563 However, I am trying to write a unit test (via pytest 'live_server'). So, immediately after '/login', and before the 'live_server' terminates its unit test, I query redis for all hash keys. Unfortunately, no keys are stored in Redis. This is strange because my application, just general use will store a cookie in the browser, containing the key, in Redis. Can anyone help me, or better organize my thoughts? Thank you, Jeff Levesque https://github.com/jeff1evesque -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff1evesque at yahoo.com Thu Oct 19 16:04:45 2017 From: jeff1evesque at yahoo.com (Jeffrey Levesque) Date: Thu, 19 Oct 2017 16:04:45 -0400 Subject: [Flask] Jeff Levesque: flask 'session' via POST Message-ID: <99D01463-DFEF-4090-91A8-C39EE1A652EC@yahoo.com> Hi guys, Wondering if flask authentication (i.e. 'from flask import session'), will return a cookie back to the client (non-browser), if the client sent a custom post request. If this is possible, can the same client, on successive post request, provide that same cookie (given to them from the first POST request they made) to maintain the status of being authenticated (logged in), if they send a successive custom POST request, not through a traditional browser action? Thank you, Jeff Levesque https://github.com/jeff1evesque From riteshn at gmail.com Thu Oct 19 18:07:37 2017 From: riteshn at gmail.com (Ritesh Nadhani) Date: Thu, 19 Oct 2017 15:07:37 -0700 Subject: [Flask] Jeff Levesque: flask 'session' via POST In-Reply-To: <99D01463-DFEF-4090-91A8-C39EE1A652EC@yahoo.com> References: <99D01463-DFEF-4090-91A8-C39EE1A652EC@yahoo.com> Message-ID: Yes. This is kind of what we are doing at my current place for our tests etc. We just send back the same cookie that we get after login to any further request using the requests/httpie combination. On Thu, Oct 19, 2017 at 1:04 PM, Jeffrey Levesque via Flask wrote: > Hi guys, > Wondering if flask authentication (i.e. 'from flask import session'), will return a cookie back to the client (non-browser), if the client sent a custom post request. If this is possible, can the same client, on successive post request, provide that same cookie (given to them from the first POST request they made) to maintain the status of being authenticated (logged in), if they send a successive custom POST request, not through a traditional browser action? > > Thank you, > > Jeff Levesque > https://github.com/jeff1evesque > _______________________________________________ > Flask mailing list > Flask at python.org > https://mail.python.org/mailman/listinfo/flask -- Ritesh From jeff1evesque at yahoo.com Thu Oct 19 18:23:03 2017 From: jeff1evesque at yahoo.com (Jeffrey Levesque) Date: Thu, 19 Oct 2017 18:23:03 -0400 Subject: [Flask] Jeff Levesque: flask 'session' via POST In-Reply-To: References: <99D01463-DFEF-4090-91A8-C39EE1A652EC@yahoo.com> Message-ID: Hi Ritesh, Are you by chance using the generic 'session' module from flask, or 'flask-session'? I need to find the syntax, to query the returned cookie, when using 'from flask import session'. Thank you, Jeff Levesque https://github.com/jeff1evesque > On Oct 19, 2017, at 6:07 PM, Ritesh Nadhani wrote: > > Yes. > > This is kind of what we are doing at my current place for our tests > etc. We just send back the same cookie that we get after login to any > further request using the requests/httpie combination. > > On Thu, Oct 19, 2017 at 1:04 PM, Jeffrey Levesque via Flask > wrote: >> Hi guys, >> Wondering if flask authentication (i.e. 'from flask import session'), will return a cookie back to the client (non-browser), if the client sent a custom post request. If this is possible, can the same client, on successive post request, provide that same cookie (given to them from the first POST request they made) to maintain the status of being authenticated (logged in), if they send a successive custom POST request, not through a traditional browser action? >> >> Thank you, >> >> Jeff Levesque >> https://github.com/jeff1evesque >> _______________________________________________ >> Flask mailing list >> Flask at python.org >> https://mail.python.org/mailman/listinfo/flask > > > > -- > Ritesh From riteshn at gmail.com Thu Oct 19 18:28:37 2017 From: riteshn at gmail.com (Ritesh Nadhani) Date: Thu, 19 Oct 2017 15:28:37 -0700 Subject: [Flask] Jeff Levesque: flask 'session' via POST In-Reply-To: References: <99D01463-DFEF-4090-91A8-C39EE1A652EC@yahoo.com> Message-ID: We are using flask-session backed by Redis. Our tests are really more: resp = requests.post('/api/ogin', ...) cookies = resp.cookies requests.post('/api/foo', cookies=cookies) .... somethingh like that pattern. On Thu, Oct 19, 2017 at 3:23 PM, Jeffrey Levesque wrote: > Hi Ritesh, > Are you by chance using the generic 'session' module from flask, or 'flask-session'? I need to find the syntax, to query the returned cookie, when using 'from flask import session'. > > Thank you, > > Jeff Levesque > https://github.com/jeff1evesque > >> On Oct 19, 2017, at 6:07 PM, Ritesh Nadhani wrote: >> >> Yes. >> >> This is kind of what we are doing at my current place for our tests >> etc. We just send back the same cookie that we get after login to any >> further request using the requests/httpie combination. >> >> On Thu, Oct 19, 2017 at 1:04 PM, Jeffrey Levesque via Flask >> wrote: >>> Hi guys, >>> Wondering if flask authentication (i.e. 'from flask import session'), will return a cookie back to the client (non-browser), if the client sent a custom post request. If this is possible, can the same client, on successive post request, provide that same cookie (given to them from the first POST request they made) to maintain the status of being authenticated (logged in), if they send a successive custom POST request, not through a traditional browser action? >>> >>> Thank you, >>> >>> Jeff Levesque >>> https://github.com/jeff1evesque >>> _______________________________________________ >>> Flask mailing list >>> Flask at python.org >>> https://mail.python.org/mailman/listinfo/flask >> >> >> >> -- >> Ritesh > -- Ritesh From sam.samkit1 at gmail.com Fri Oct 20 00:11:56 2017 From: sam.samkit1 at gmail.com (Samkit Jain) Date: Fri, 20 Oct 2017 09:41:56 +0530 Subject: [Flask] [PyBITS][HydPy] Invitation for PyBITS Conference and Workshops 2017 Message-ID: Hi, We are proud to announce that the tickets for PyBITS Conference 2017 are live now. PyBITS is the python conference for students conducted by BITS Pilani Hyderabad Campus as a part of its techfest ATMOS and was started last year with the support of Python Software Foundation and this year we are growing bigger with the support of HydPy as well. The aim of the conference is to bring the Python Student Community together and give them a platform to learn, explore and network. As a part of PyBITS we are having the conference on 28th October with keynote, general and lightning talks. Sunil Mohan Adapa is the keynote speaker for the conference and he will be talking about the Freedom Box project and also about our current dystopian world with ubiquitous data collection. The general talks are as follows - Building a chatbot from scratch using open-source tools by Saikat Kumar Dey - IOT and Python by Satish Rao - Data Science and Sentimental Analysis. by Madhu Vadlamani - Introduction to Continuous Integration in SW Development by Ramanathan Mutahiah The workshops will be conducted on 29th October. We would be conducting a large-scale basic python workshop for local school-children, which we intend to let them attend for free. We also plan to have two intermediate level Python workshops which have a limit of 300 registrations. - Basic Workshop(6 hours) by Crux - For python beginners and school students. - Flask Workshop (3 hours) by Malhar Lathkar who is a Programming Technologies Trainer - Scrapy(3 hours) by Mohammad Sajid who is the founder of Step Up Analytics. The tickets for attending only conference are priced at Rs. 250 while for attending both the conference and workshops it is Rs. 400. To register for the event and know more about the speakers please visit the PyBITS Website. For any queries regarding the event, mail to crux at hyderabad.bits-pilani.ac.in. Regards, Samkit Jain Coordinator PyBITS, 9705601217 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff1evesque at yahoo.com Fri Oct 20 08:23:27 2017 From: jeff1evesque at yahoo.com (Jeffrey Levesque) Date: Fri, 20 Oct 2017 08:23:27 -0400 Subject: [Flask] Jeff Levesque: flask 'session' via POST In-Reply-To: References: <99D01463-DFEF-4090-91A8-C39EE1A652EC@yahoo.com> Message-ID: <37609872-EC79-4224-A484-9A302ABD243B@yahoo.com> Hi Ritesh, That's helpful. I was trying to query the same Redis instance, for an entry corresponding to the session. Unfortunately, that always returned 'None', during unit testing. But, I will use your general cookie pattern, you've just mentioned, unless I can also find out how to query Redis as well. Thank you, Jeff Levesque https://github.com/jeff1evesque > On Oct 19, 2017, at 6:28 PM, Ritesh Nadhani wrote: > > We are using flask-session backed by Redis. > > Our tests are really more: > > resp = requests.post('/api/ogin', ...) > > cookies = resp.cookies > > requests.post('/api/foo', cookies=cookies) > > .... somethingh like that pattern. > > On Thu, Oct 19, 2017 at 3:23 PM, Jeffrey Levesque > wrote: >> Hi Ritesh, >> Are you by chance using the generic 'session' module from flask, or 'flask-session'? I need to find the syntax, to query the returned cookie, when using 'from flask import session'. >> >> Thank you, >> >> Jeff Levesque >> https://github.com/jeff1evesque >> >>> On Oct 19, 2017, at 6:07 PM, Ritesh Nadhani wrote: >>> >>> Yes. >>> >>> This is kind of what we are doing at my current place for our tests >>> etc. We just send back the same cookie that we get after login to any >>> further request using the requests/httpie combination. >>> >>> On Thu, Oct 19, 2017 at 1:04 PM, Jeffrey Levesque via Flask >>> wrote: >>>> Hi guys, >>>> Wondering if flask authentication (i.e. 'from flask import session'), will return a cookie back to the client (non-browser), if the client sent a custom post request. If this is possible, can the same client, on successive post request, provide that same cookie (given to them from the first POST request they made) to maintain the status of being authenticated (logged in), if they send a successive custom POST request, not through a traditional browser action? >>>> >>>> Thank you, >>>> >>>> Jeff Levesque >>>> https://github.com/jeff1evesque >>>> _______________________________________________ >>>> Flask mailing list >>>> Flask at python.org >>>> https://mail.python.org/mailman/listinfo/flask >>> >>> >>> >>> -- >>> Ritesh >> > > > > -- > Ritesh From rclayton at uw.edu Fri Oct 27 13:03:07 2017 From: rclayton at uw.edu (Richard Clayton) Date: Fri, 27 Oct 2017 17:03:07 +0000 Subject: [Flask] Jeff Levesque: need help unit testing flask redis session In-Reply-To: <20E45963-0CE3-45ED-9C12-3A1B13D69A9A@yahoo.com> References: <20E45963-0CE3-45ED-9C12-3A1B13D69A9A@yahoo.com> Message-ID: <71B90939-25BD-48D0-B4A0-80E86AC67D5F@uw.edu> I am currently out of the office. I will be out of the office until Monday 11/6/2017. If you need a response before 11/6 please contact the Lab Med computer staff at lmsrcomp at uw.edu. On Oct 18, 2017, at 5:15 PM, Jeffrey Levesque via Flask wrote: Hi guys, I've implemented Redis with my flask login session. I've verified that it all works: - https://github.com/jeff1evesque/machine-learning/pull/3073#issuecomment-337446563 However, I am trying to write a unit test (via pytest 'live_server'). So, immediately after '/login', and before the 'live_server' terminates its unit test, I query redis for all hash keys. Unfortunately, no keys are stored in Redis. This is strange because my application, just general use will store a cookie in the browser, containing the key, in Redis. Can anyone help me, or better organize my thoughts? Thank you, Jeff Levesque https://github.com/jeff1evesque _______________________________________________ Flask mailing list Flask at python.org https://mail.python.org/mailman/listinfo/flask -------------- next part -------------- An HTML attachment was scrubbed... URL: