[Python.NET] Working group notes for November 21, 2019

[Mark Visser]

Attendees: Victor, Benoit, Mark, Amos, Benedikt

Agenda
Review action items from last meeting:
Mark - got in touch with .NET foundation
Benoit - document how to use pythonnet when you expect to run multithreaded python code (BeginAllowThreads/EndAllowThreads) in an embedded interpreter 
Victor - will create Github issues for items in the Quantconnect PR and figure out plan for bringing more generic solutions to master.
? - can someone research how hard it would be to move the CI to Github Actions?
Benoit - will break out SlotsHolder and __import__ restore into smaller PRs 
Amos - will continue to break https://github.com/pythonnet/pythonnet/pull/958 <https://github.com/pythonnet/pythonnet/pull/958> into smaller PRs following Benoit's example
All - please discuss changing the meeting time on the list

Notes
.NET foundation - Benedikt working on it with contact & maintainer of IronPython

Multithreaded python code - Benoit still working on it, has a deadlock issue that's probably unrelated. WPF has a similar issue? There are old tickets around where people tried this before. Possibility of making it automatic, but requires a major version bump. https://github.com/pythonnet/pythonnet/issues/990 <https://github.com/pythonnet/pythonnet/pull/958> for example. 

Quantconnect PR - perf PR was OK, but is a fairly small part of the larger PR. Need to get .NET standard / core stuff on the way so that they don't fork off.

Github actions - currently have appveyer and travis CI implementations, but in theory github actions can do both and also work on mac. Could cover all platforms. 
One of the unit tests crashes on Mac - ongoing problem, would be good to have CI for that. Possible plan is to start with github actions on Mac, then extend to Linux, finally Windows.

__import__ restore - merged (moved out of pull 958)

SlotsHolder in progress - goal is to make it so that PyFinalize doesn't need to be called to disconnect C# from Python (moved out of pull 958)

General note: please make PRs as small and targeted as possible - large PRs are very difficult to review and judge the impact and risk. Large PRs won't likely be merged.
Discussion around time to accommodate Amos - we will move the meeting 3 hours earlier.

Need to start planning for a point release (2.4.1) including Python 3.8 support.
PyBuffer change - good addition but is crashing in a non-obvious way. Maybe not a candidate for the point release. If someone needs it for the next release, they could try to fix it in branch. We shouldn't have any breaking changes since 2.4.0, only bug fixes.

Benedikt - Haven't been able to integrate CLR loader yet (ripping out old stuff). Will try to have the interface ready for the CLR loader. Python.NET exposes a clr module inside Python which exposes .NET VM startup functions. Mono vs. Core have different code paths currently. Current approach in latest .NET core seems to be stable. New CLR loader abstracts three different ways of loading .NET. Technical discussion. Proposal: pass parameters as JSON. Alternatives? Would also enable choosing runtime version of Python.

Benchmarking - Still in progress. PR will be open today.

Could CLR loader be completely separate from Python.NET? 
Benedikt - sure, it's useful on its own as a low-level mechanism to talk to .NET. On the other hand Python interop is interconnected with how Python.NET works and should stay in the project. No plans to have more libraries than CLR loader (which isn't actually needed if you're embedding Python within .NET) and pure interop + abstractions on top.

Action Items
Benedikt - continue working on joining .NET foundation 
Amos - continue breaking up pull 958 into smaller PRs to make it easier to review and merge.  e.g. have a single PR that gets reference counts right and doesn't touch GC chains. See Benoit's example of the __import__ restore PR for best practices.
Amos - same for other older PRs
Mark - move meeting 3 hours earlier
Benedikt - will try to reproduce PyBuffer crash locally
Benedikt - will check if there are any breaking changes since 2.4.0
Benedikt - continue exploring CLR loader, splitting parts into separate Python interop DLLs, have main Python runtime implement layers on top

The meeting notes google doc is here <https://docs.google.com/document/d/1rJVU84B_dgx58-_EopjRtOJVFAI2WfHJYV0n7uE1Oak/edit#>. Feel free to correct or add additional information.

The next working group meeting will be held on December 5th at 12pm EST, 9am PST, 6pm CET, 1am China. Please note that this is three hours earlier than previous meetings.


Mark Visser
Tooling Dev Manager
Unity Technologies - www.unity3d.com <http://www.unity3d.com/>




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20191121/b86745fd/attachment-0001.html>