Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-23 1:56 PM, Frank Millman wrote:
> On 2020-03-23 12:57 PM, Chris Angelico wrote:
>> On Mon, Mar 23, 2020 at 8:03 PM Frank Millman <frank at chagford.com> wrote:
>>>
>>> On 2020-03-22 12:11 PM, Chris Angelico wrote:
>>>> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman <frank at chagford.com> 
>>>> wrote:
>>>>>
>>>>> On 2020-03-22 10:45 AM, Chris Angelico wrote:
>>>>
>>>> If you can recreate the problem with a single socket and multiple
>>>> requests, that would be extremely helpful. I also think it's highly
>>>> likely that this is the case.
>>>>
>>>
>>> I am working on a stripped-down version, but I realise there are a few
>>> things I have not grasped.
>>>
>>> Hope you don't mind, but can you scan through what follows and tell me
>>> if I am on the right lines?
>>
>> No probs!
>>
> 
> [...]
> 
> Really appreciate the one-on-one tuition. I am learning a lot!
> 
>>
>>> If this all makes sense, I should write two versions of the client
>>> program, one using a single connection, and one using a pool of 
>>> connections.
>>>
>>
>> Possibly! I think you'll most likely see that one of those behaves
>> perfectly normally, and you only trigger the issue in the other. So
>> you could move forward with just one test program.
>>
> 
> Well, I have got the first one working - single connection - and so far 
> it has not gone wrong.
> 
> However, it is difficult to be sure that I am comparing apples with 
> apples. I have written my test server to handle 'Keep-Alive' correctly, 
> but as I mentioned earlier, my live program closes the connection after 
> each transfer. So now I have to make my test server do the same, and 
> change my test client to react to that and re-open the connection each 
> time. I will make the changes and see how that behaves.
> 
> Of course now I am in the murky waters of trying to second-guess how 
> Edge reacts to that. Presumably that is where Wireshark will be useful. 
> I will keep you posted.

Here is a progress report.

I decided to concentrate on using Wireshark to detect the difference 
between a Python3.7 session and a Python3.8 session. Already I can see 
some differences.

There is only one version of my program. I am simply running it with 
either 'py -3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, 
as it is only that Edge shows the problem.

First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess 
is that this is caused by my 'protocol violation' of sending a 
'Keep-Alive' header and then closing the connection. Python3.7 does not 
suffer from dropping files, so I now think this is a sidetrack. I will 
fix my program when this is all over, but for now I don't want to touch it.

When I send the response to the initial connection, I write a status 
line, then multiple header lines, then an html file. I then close the 
connection. Python3.7 sends a packet with the status, then a separate 
packet for each header, then a packet with the file. Python3.8 sends a 
packet with the status, then merges everything else into a single packet 
and sends it in one go. I just mention this as an indication that quite 
a lot has changed between my two versions of Python.

I have one frustration with Wireshark. I will mention it in case anyone 
has a solution.

I can see that Edge opens multiple connections. I am trying to track the 
activity on each connection separately. I can export the data to csv, 
which makes it easier to work on. But while the TCP lines include the 
source and destination ports, the HTTP lines do not, so I don't know 
which connection they belong to. If I view the data in Wireshark's gui 
it does show the ports, so the data is there somewhere. Does anyone know 
how to include it in the csv output?

That's all for now. I will keep you posted.

Frank