From garcia.marc at gmail.com Wed Apr 8 06:32:12 2020 From: garcia.marc at gmail.com (Marc Garcia) Date: Wed, 8 Apr 2020 11:32:12 +0100 Subject: [Pandas-dev] Domain and hosting In-Reply-To: References: Message-ID: An update on the pandas hosting. We kept having problems with the cloud infrastructure (object storage) we were trying to use. We initially couldn't set up the naked domain to it, and we had to get a load balancer. We couldn't find an easy way to create symlinks (like from `docs/` to `versions/1.0.3`, and many others we use). We've also got problems with the synchronization, which was failing for corruption errors (apparently the corruption didn't exist, but it failed our builds). And some other minor problems. So, I finally set everything up in the existing NumFOCUS/rackspace server, under the pandas.pydata.org domain. That also includes the development docs, and the blog, which has been moved together with the website (we don't use Pelican now, the posts are rendered as "normal" website pages). The development docs are now at: https://pandas.pydata.org/docs/dev/ Couple of things are pending: - We discarded the cloud infrastructure, but OVH can give us a regular server. Do we want it? Would having two servers be useful for running benchmarks or something? - I think most people was in favor of using pandas.pydata.org. I guess we should remove the dns entries from pandas.io, to avoid confussion there. Wes, can you do that please? - The next repos are not used anymore, and I guess they can be removed: * https://github.com/pandas-dev/pandas-dev.github.io (was used to publish the development docs in GitHub pages, now that's push to the production server, at https://pandas.pydata.org/docs/dev/) * https://github.com/pandas-dev/pandas-blog (was used for the blog, that's now in the website, new posts can be simply added to `web/pandas/community/blog`) * https://github.com/pandas-dev/pandas-website (old website, not being used anymore) Most of the links to the old domains have been replaced, and we've got a PR open to change a pending one. If you find any broken link, or anything not working as expected, please let me know. Cheers! On Fri, Mar 20, 2020 at 4:49 PM Marc Garcia wrote: > I finally moved the old docs to the OVH server, and synchronized the dev > docs and the website automatically from the CI there. > > We had problems with GitHub actions and the ssh key, and found it was > easier to move forward with OVH. Also, if we use OVH, then we could > potentially use the rackspace server for benchmarks or other things (the > OVH infrastructure is just a storage for static files, we could not do the > opposite). > > There is one issue pending, looks like these cloud storages do not support > symlinks. I guess there is a workaround, and asked OVH about it, but still > haven't heard from them. Once that is done, we can point pandas.pydata.org > to OVH, and update the release script that deploys the documentation, and > we should be done. > > On Fri, Mar 20, 2020 at 4:37 PM Joris Van den Bossche < > jorisvandenbossche at gmail.com> wrote: > >> Hi Marc, >> >> Can you give an update on this thread? >> (and thanks for your work on this!) >> >> Joris >> >> On Mon, 17 Feb 2020 at 20:17, Andy Ray Terrel >> wrote: >> >>> >>> >>> On Mon, Feb 17, 2020 at 12:24 PM Marc Garcia >>> wrote: >>> >>>> The OVH hosting is also free. I thought Rackspace started sending >>>> invoices now, I saw on Twitter comments from some other projects about it. >>>> >>> >>> NumFOCUS has negotiated a 2 year extension. The twitter comments seems >>> to be mostly tied to personal accounts. >>> >>> >>>> >>>> Part of the idea of moving to OVH was because they should provide us >>>> with the Binder infrastructure if we make the examples in the docs runnable. >>>> >>> >>> OVH is great but as far as I know there is no formal relationship. Sorry >>> if there are details elsewhere that I've missed. Anywho, I just wanted to >>> make sure you know I had resources to help if needed. >>> >>> >>>> Not sure how we should move forward then. Should we simply disable or >>>> redirect pandas.io, and we set up the CI to update the website and the >>>> dev docs there? The current server seems to work well enough, and probably >>>> not worth moving things for now if we still have it for two more years. >>>> Does anyone have a different idea or a preference? >>>> >>> >>>> On Mon, Feb 17, 2020 at 1:09 PM Andy Ray Terrel >>>> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Feb 17, 2020 at 4:13 AM Marc Garcia >>>>> wrote: >>>>> >>>>>> We've got a bit of a mess at the moment with the pandas domain and >>>>>> hosting. I'll try to leave things in a more reasonable way, but there are >>>>>> some decisions pending. >>>>>> >>>>>> My understanding from a thread in this list was that everybody was >>>>>> happy with using pandas.io, and we set up the domain for the new >>>>>> hosting, and also dev.pandas.io for the development version of the >>>>>> website (and the blog and anything published in our GitHub organization >>>>>> pages). >>>>>> >>>>>> From the discussion in >>>>>> https://github.com/pandas-dev/pandas/issues/28528 seems like the >>>>>> preference is to keep the old pandas.pydata.org instead. Given that, >>>>>> I think we can get rid of the dev.pandas.io, and point >>>>>> pandas.pydata.org to the new server once it's ready. >>>>>> >>>>>> For the website, I think the agreement is to update it with the >>>>>> latest version from master. So, no dev.pandas.io. For the >>>>>> development (master) documentation, I think it can live in >>>>>> pandas.pydata.org/docs/dev/. >>>>>> >>>>>> The blog, I think the best is to have the posts as pages on the >>>>>> website, in a directory blog/, so we don't need to maintain separaterly, >>>>>> and it has the look and feel of the website. >>>>>> >>>>>> For the new hosting, I'll move everything (all old documentation >>>>>> versions) from the current server to the new one, and then set up that the >>>>>> website and the development docs are automatically updated. Tom, can you >>>>>> give me access to the current web server so I can fetch the data please? >>>>>> >>>>> >>>>> Marc, send me your ssh-key and preferred login, I can get you access. >>>>> NumFOCUS just got a deal with AWS and we have the Rackspace servers for >>>>> another two years, so unless you are just dying to pay fees, let me get you >>>>> free servers. >>>>> >>>>> >>>>>> >>>>>> Once everything is working in the new server, we'll be able to see it >>>>>> in pandas.io, and when we're happy we can change the domain >>>>>> pandas.pydata.org to point to it, and disable pandas.io. >>>>>> >>>>>> Please let me know if there are objections to any of the above, >>>>>> otherwise I'll move forward. >>>>>> >>>>>> Cheers! >>>>>> _______________________________________________ >>>>>> Pandas-dev mailing list >>>>>> Pandas-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/pandas-dev >>>>>> >>>>> _______________________________________________ >>> Pandas-dev mailing list >>> Pandas-dev at python.org >>> https://mail.python.org/mailman/listinfo/pandas-dev >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy.terrel at gmail.com Wed Apr 8 07:44:02 2020 From: andy.terrel at gmail.com (Andy Ray Terrel) Date: Wed, 8 Apr 2020 06:44:02 -0500 Subject: [Pandas-dev] Domain and hosting In-Reply-To: References: Message-ID: Sorry I?be been out of it. I?ve been overwhelmed at work these days. Do you have a script that sets the server up, so if the server goes down we can restore it easily. Rackspace isn?t the most reliable cloud, moving things to OVH seems fine to me, but I?ve never used them so I can?t comment on stability. NumFOCUS now has an AWS account structure to, and I now have around $1000 per project. ? Andy On Wed, Apr 8, 2020 at 5:32 AM Marc Garcia wrote: > An update on the pandas hosting. > > We kept having problems with the cloud infrastructure (object storage) we > were trying to use. We initially couldn't set up the naked domain to it, > and we had to get a load balancer. We couldn't find an easy way to create > symlinks (like from `docs/` to `versions/1.0.3`, and many others we use). > We've also got problems with the synchronization, which was failing for > corruption errors (apparently the corruption didn't exist, but it failed > our builds). And some other minor problems. > > So, I finally set everything up in the existing NumFOCUS/rackspace server, > under the pandas.pydata.org domain. That also includes the development > docs, and the blog, which has been moved together with the website (we > don't use Pelican now, the posts are rendered as "normal" website pages). > > The development docs are now at: https://pandas.pydata.org/docs/dev/ > > Couple of things are pending: > - We discarded the cloud infrastructure, but OVH can give us a regular > server. Do we want it? Would having two servers be useful for running > benchmarks or something? > - I think most people was in favor of using pandas.pydata.org. I guess we > should remove the dns entries from pandas.io, to avoid confussion there. > Wes, can you do that please? > - The next repos are not used anymore, and I guess they can be removed: > * https://github.com/pandas-dev/pandas-dev.github.io (was used to > publish the development docs in GitHub pages, now that's push to the > production server, at https://pandas.pydata.org/docs/dev/) > * https://github.com/pandas-dev/pandas-blog (was used for the blog, > that's now in the website, new posts can be simply added to > `web/pandas/community/blog`) > * https://github.com/pandas-dev/pandas-website (old website, not being > used anymore) > > Most of the links to the old domains have been replaced, and we've got a > PR open to change a pending one. If you find any broken link, or anything > not working as expected, please let me know. > > Cheers! > > On Fri, Mar 20, 2020 at 4:49 PM Marc Garcia wrote: > >> I finally moved the old docs to the OVH server, and synchronized the dev >> docs and the website automatically from the CI there. >> >> We had problems with GitHub actions and the ssh key, and found it was >> easier to move forward with OVH. Also, if we use OVH, then we could >> potentially use the rackspace server for benchmarks or other things (the >> OVH infrastructure is just a storage for static files, we could not do the >> opposite). >> >> There is one issue pending, looks like these cloud storages do not >> support symlinks. I guess there is a workaround, and asked OVH about it, >> but still haven't heard from them. Once that is done, we can point >> pandas.pydata.org to OVH, and update the release script that deploys the >> documentation, and we should be done. >> >> On Fri, Mar 20, 2020 at 4:37 PM Joris Van den Bossche < >> jorisvandenbossche at gmail.com> wrote: >> >>> Hi Marc, >>> >>> Can you give an update on this thread? >>> (and thanks for your work on this!) >>> >>> Joris >>> >>> On Mon, 17 Feb 2020 at 20:17, Andy Ray Terrel >>> wrote: >>> >>>> >>>> >>>> On Mon, Feb 17, 2020 at 12:24 PM Marc Garcia >>>> wrote: >>>> >>>>> The OVH hosting is also free. I thought Rackspace started sending >>>>> invoices now, I saw on Twitter comments from some other projects about it. >>>>> >>>> >>>> NumFOCUS has negotiated a 2 year extension. The twitter comments seems >>>> to be mostly tied to personal accounts. >>>> >>>> >>>>> >>>>> Part of the idea of moving to OVH was because they should provide us >>>>> with the Binder infrastructure if we make the examples in the docs runnable. >>>>> >>>> >>>> OVH is great but as far as I know there is no formal relationship. >>>> Sorry if there are details elsewhere that I've missed. Anywho, I just >>>> wanted to make sure you know I had resources to help if needed. >>>> >>>> >>>>> Not sure how we should move forward then. Should we simply disable or >>>>> redirect pandas.io, and we set up the CI to update the website and >>>>> the dev docs there? The current server seems to work well enough, and >>>>> probably not worth moving things for now if we still have it for two more >>>>> years. Does anyone have a different idea or a preference? >>>>> >>>> >>>>> On Mon, Feb 17, 2020 at 1:09 PM Andy Ray Terrel >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Mon, Feb 17, 2020 at 4:13 AM Marc Garcia >>>>>> wrote: >>>>>> >>>>>>> We've got a bit of a mess at the moment with the pandas domain and >>>>>>> hosting. I'll try to leave things in a more reasonable way, but there are >>>>>>> some decisions pending. >>>>>>> >>>>>>> My understanding from a thread in this list was that everybody was >>>>>>> happy with using pandas.io, and we set up the domain for the new >>>>>>> hosting, and also dev.pandas.io for the development version of the >>>>>>> website (and the blog and anything published in our GitHub organization >>>>>>> pages). >>>>>>> >>>>>>> From the discussion in >>>>>>> https://github.com/pandas-dev/pandas/issues/28528 seems like the >>>>>>> preference is to keep the old pandas.pydata.org instead. Given >>>>>>> that, I think we can get rid of the dev.pandas.io, and point >>>>>>> pandas.pydata.org to the new server once it's ready. >>>>>>> >>>>>>> For the website, I think the agreement is to update it with the >>>>>>> latest version from master. So, no dev.pandas.io. For the >>>>>>> development (master) documentation, I think it can live in >>>>>>> pandas.pydata.org/docs/dev/. >>>>>>> >>>>>>> The blog, I think the best is to have the posts as pages on the >>>>>>> website, in a directory blog/, so we don't need to maintain separaterly, >>>>>>> and it has the look and feel of the website. >>>>>>> >>>>>>> For the new hosting, I'll move everything (all old documentation >>>>>>> versions) from the current server to the new one, and then set up that the >>>>>>> website and the development docs are automatically updated. Tom, can you >>>>>>> give me access to the current web server so I can fetch the data please? >>>>>>> >>>>>> >>>>>> Marc, send me your ssh-key and preferred login, I can get you access. >>>>>> NumFOCUS just got a deal with AWS and we have the Rackspace servers for >>>>>> another two years, so unless you are just dying to pay fees, let me get you >>>>>> free servers. >>>>>> >>>>>> >>>>>>> >>>>>>> Once everything is working in the new server, we'll be able to see >>>>>>> it in pandas.io, and when we're happy we can change the domain >>>>>>> pandas.pydata.org to point to it, and disable pandas.io. >>>>>>> >>>>>>> Please let me know if there are objections to any of the above, >>>>>>> otherwise I'll move forward. >>>>>>> >>>>>>> Cheers! >>>>>>> _______________________________________________ >>>>>>> Pandas-dev mailing list >>>>>>> Pandas-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev >>>>>>> >>>>>> _______________________________________________ >>>> Pandas-dev mailing list >>>> Pandas-dev at python.org >>>> https://mail.python.org/mailman/listinfo/pandas-dev >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at ganssle.io Sat Apr 18 13:46:00 2020 From: paul at ganssle.io (Paul Ganssle) Date: Sat, 18 Apr 2020 13:46:00 -0400 Subject: [Pandas-dev] Request for Comments on PEP 615 Message-ID: Hi Pandas devs, If you are not already aware, I proposed PEP 615 , which adds IANA time zone support to the standard library, almost two months ago. I know that pandas makes very heavy use of time zones, so I probably should have explicitly requested comments earlier in this process, but I suppose it's better late than never. The discussion is mostly centralized to the Python discourse . You are welcome to comment on any aspect of the PEP, though one of the few remaining "open questions" is the question of equality and hash semantics , which I figured may be relevant for this crowd, since I know you have to do some funky footwork to in some ways normalize time zones and adapt the time zone information to your own time model. Thanks! Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: