[Pandas-dev] Domain and hosting

Wed Apr 8 07:44:02 EDT 2020

Sorry I‘be been out of it. I’ve been overwhelmed at work these days.

Do you have a script that sets the server up, so if the server goes down we
can restore it easily. Rackspace isn’t the most reliable cloud, moving
things to OVH seems fine to me, but I’ve never used them so I can’t comment
on stability. NumFOCUS now has an AWS account structure to, and I now have
around $1000 per project.

— Andy

On Wed, Apr 8, 2020 at 5:32 AM Marc Garcia <garcia.marc at gmail.com> wrote:

> An update on the pandas hosting.
>
> We kept having problems with the cloud infrastructure (object storage) we
> were trying to use. We initially couldn't set up the naked domain to it,
> and we had to get a load balancer. We couldn't find an easy way to create
> symlinks (like from `docs/` to `versions/1.0.3`, and many others we use).
> We've also got problems with the synchronization, which was failing for
> corruption errors (apparently the corruption didn't exist, but it failed
> our builds). And some other minor problems.
>
> So, I finally set everything up in the existing NumFOCUS/rackspace server,
> under the pandas.pydata.org domain. That also includes the development
> docs, and the blog, which has been moved together with the website (we
> don't use Pelican now, the posts are rendered as "normal" website pages).
>
> The development docs are now at: https://pandas.pydata.org/docs/dev/
>
> Couple of things are pending:
> - We discarded the cloud infrastructure, but OVH can give us a regular
> server. Do we want it? Would having two servers be useful for running
> benchmarks or something?
> - I think most people was in favor of using pandas.pydata.org. I guess we
> should remove the dns entries from pandas.io, to avoid confussion there.
> Wes, can you do that please?
> - The next repos are not used anymore, and I guess they can be removed:
>   * https://github.com/pandas-dev/pandas-dev.github.io (was used to
> publish the development docs in GitHub pages, now that's push to the
> production server, at https://pandas.pydata.org/docs/dev/)
>   * https://github.com/pandas-dev/pandas-blog (was used for the blog,
> that's now in the website, new posts can be simply added to
> `web/pandas/community/blog`)
>   * https://github.com/pandas-dev/pandas-website (old website, not being
> used anymore)
>
> Most of the links to the old domains have been replaced, and we've got a
> PR open to change a pending one. If you find any broken link, or anything
> not working as expected, please let me know.
>
> Cheers!
>
> On Fri, Mar 20, 2020 at 4:49 PM Marc Garcia <garcia.marc at gmail.com> wrote:
>
>> I finally moved the old docs to the OVH server, and synchronized the dev
>> docs and the website automatically from the CI there.
>>
>> We had problems with GitHub actions and the ssh key, and found it was
>> easier to move forward with OVH. Also, if we use OVH, then we could
>> potentially use the rackspace server for benchmarks or other things (the
>> OVH infrastructure is just a storage for static files, we could not do the
>> opposite).
>>
>> There is one issue pending, looks like these cloud storages do not
>> support symlinks. I guess there is a workaround, and asked OVH about it,
>> but still haven't heard from them. Once that is done, we can point
>> pandas.pydata.org to OVH, and update the release script that deploys the
>> documentation, and we should be done.
>>
>> On Fri, Mar 20, 2020 at 4:37 PM Joris Van den Bossche <
>> jorisvandenbossche at gmail.com> wrote:
>>
>>> Hi Marc,
>>>
>>> Can you give an update on this thread?
>>> (and thanks for your work on this!)
>>>
>>> Joris
>>>
>>> On Mon, 17 Feb 2020 at 20:17, Andy Ray Terrel <andy.terrel at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Feb 17, 2020 at 12:24 PM Marc Garcia <garcia.marc at gmail.com>
>>>> wrote:
>>>>
>>>>> The OVH hosting is also free. I thought Rackspace started sending
>>>>> invoices now, I saw on Twitter comments from some other projects about it.
>>>>>
>>>>
>>>> NumFOCUS has negotiated a 2 year extension. The twitter comments seems
>>>> to be mostly tied to personal accounts.
>>>>
>>>>
>>>>>
>>>>> Part of the idea of moving to OVH was because they should provide us
>>>>> with the Binder infrastructure if we make the examples in the docs runnable.
>>>>>
>>>>
>>>> OVH is great but as far as I know there is no formal relationship.
>>>> Sorry if there are details elsewhere that I've missed. Anywho, I just
>>>> wanted to make sure you know I had resources to help if needed.
>>>>
>>>>
>>>>> Not sure how we should move forward then. Should we simply disable or
>>>>> redirect pandas.io, and we set up the CI to update the website and
>>>>> the dev docs there? The current server seems to work well enough, and
>>>>> probably not worth moving things for now if we still have it for two more
>>>>> years. Does anyone have a different idea or a preference?
>>>>>
>>>>
>>>>> On Mon, Feb 17, 2020 at 1:09 PM Andy Ray Terrel <andy.terrel at gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 17, 2020 at 4:13 AM Marc Garcia <garcia.marc at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> We've got a bit of a mess at the moment with the pandas domain and
>>>>>>> hosting. I'll try to leave things in a more reasonable way, but there are
>>>>>>> some decisions pending.
>>>>>>>
>>>>>>> My understanding from a thread in this list was that everybody was
>>>>>>> happy with using pandas.io, and we set up the domain for the new
>>>>>>> hosting, and also dev.pandas.io for the development version of the
>>>>>>> website (and the blog and anything published in our GitHub organization
>>>>>>> pages).
>>>>>>>
>>>>>>> From the discussion in
>>>>>>> https://github.com/pandas-dev/pandas/issues/28528 seems like the
>>>>>>> preference is to keep the old pandas.pydata.org instead. Given
>>>>>>> that, I think we can get rid of the dev.pandas.io, and point
>>>>>>> pandas.pydata.org to the new server once it's ready.
>>>>>>>
>>>>>>> For the website, I think the agreement is to update it with the
>>>>>>> latest version from master. So, no dev.pandas.io. For the
>>>>>>> development (master) documentation, I think it can live in
>>>>>>> pandas.pydata.org/docs/dev/.
>>>>>>>
>>>>>>> The blog, I think the best is to have the posts as pages on the
>>>>>>> website, in a directory blog/, so we don't need to maintain separaterly,
>>>>>>> and it has the look and feel of the website.
>>>>>>>
>>>>>>> For the new hosting, I'll move everything (all old documentation
>>>>>>> versions) from the current server to the new one, and then set up that the
>>>>>>> website and the development docs are automatically updated. Tom, can you
>>>>>>> give me access to the current web server so I can fetch the data please?
>>>>>>>
>>>>>>
>>>>>> Marc, send me your ssh-key and preferred login, I can get you access.
>>>>>> NumFOCUS just got a deal with AWS and we have the Rackspace servers for
>>>>>> another two years, so unless you are just dying to pay fees, let me get you
>>>>>> free servers.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Once everything is working in the new server, we'll be able to see
>>>>>>> it in pandas.io, and when we're happy we can change the domain
>>>>>>> pandas.pydata.org to point to it, and disable pandas.io.
>>>>>>>
>>>>>>> Please let me know if there are objections to any of the above,
>>>>>>> otherwise I'll move forward.
>>>>>>>
>>>>>>> Cheers!
>>>>>>> _______________________________________________
>>>>>>> Pandas-dev mailing list
>>>>>>> Pandas-dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>
>>>>>> _______________________________________________
>>>> Pandas-dev mailing list
>>>> Pandas-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pandas-dev/attachments/20200408/4a867f45/attachment-0001.html>