From konstantin at shemyak.com  Fri Apr  2 15:46:51 2021
From: konstantin at shemyak.com (Konstantin Shemyak)
Date: Fri, 2 Apr 2021 22:46:51 +0300
Subject: [Cryptography-dev] Replicating CI of the project?
Message-ID: <1f8a7417-33f9-bb11-d657-98e9ee88daf8@shemyak.com>

In my attempt to contribute to the project, I have bumped into numerous 
"corners" of the CI checks and have caused massive force-push spam. What 
would be your suggestion to replicate the CI of the project to be able 
to run all the checks before actually creating a PR?

Konstantin Shemyak

From alex.gaynor at gmail.com  Fri Apr  2 18:14:11 2021
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 2 Apr 2021 18:14:11 -0400
Subject: [Cryptography-dev] Replicating CI of the project?
In-Reply-To: <1f8a7417-33f9-bb11-d657-98e9ee88daf8@shemyak.com>
References: <1f8a7417-33f9-bb11-d657-98e9ee88daf8@shemyak.com>
Message-ID: <CAFRnB2VswdQnGCzFqzv-U-nnA8vQ-wm-xpvFd3TGNN9tTyctPw@mail.gmail.com>

I haven't tried it myself, but https://github.com/nektos/act may literally
let you just run our github actions locally.

To answer your question more specifically: there's usually one of two
reasons our CI fails that aren't caught locally: either due to the many
many versions of OpenSSL we support, or due to flake8/black/mypy/etc. tests.

I haven't bothered trying to replicate our zillions of OpenSSL versions
locally, thought one _can_ compile them all themselves if they're
motivated. For the other issue, it's possible to run them under tox, but I
generally just run black+flake8+mypy by hand, since it's faster.

Does that answer what you're looking for?

Alex

On Fri, Apr 2, 2021 at 6:09 PM Konstantin Shemyak <konstantin at shemyak.com>
wrote:

> In my attempt to contribute to the project, I have bumped into numerous
> "corners" of the CI checks and have caused massive force-push spam. What
> would be your suggestion to replicate the CI of the project to be able
> to run all the checks before actually creating a PR?
>
> Konstantin Shemyak
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev
>


-- 
All that is necessary for evil to succeed is for good people to do nothing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210402/bedfd015/attachment.html>

From mass.shock57 at gmail.com  Sat Apr  3 23:54:06 2021
From: mass.shock57 at gmail.com (Masoud Shokohi)
Date: Sun, 4 Apr 2021 08:24:06 +0430
Subject: [Cryptography-dev] Submission
Message-ID: <CAEn=tBWPp0AvzWrhrAR0fp6KZWaqDRH9mV-Nu2AmoYcvOXne=A@mail.gmail.com>

Submission
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210404/3b8a3d63/attachment.html>

From hanuszczak at google.com  Thu Apr 15 06:06:42 2021
From: hanuszczak at google.com (=?UTF-8?Q?=C5=81ukasz_Hanuszczak?=)
Date: Thu, 15 Apr 2021 12:06:42 +0200
Subject: [Cryptography-dev] Problems with importing OpenSSL on Windows
Message-ID: <CAAUp-Cjm=RAce_3MjYiWhDeVyVzXd_J7=z2US0dkiwn3tJ6K8Q@mail.gmail.com>

Hello,

Our team uses two continuous integration systems that build and package the
Windows distribution of our project. One of them is internal and runs on
Windows 7, the other one is AppVeyor (running Windows 2012, which is based
on Windows 8 best to my knowledge) that works with our open-source
repository on GitHub.

Both systems use the same script that initializes the virtual environment,
upgrades pip, installs dependencies. The only significant difference is
that the internal pipeline does not have internet access and uses vendored
wheels (though downloaded from PyPI upfront).

Both systems use exactly the same version of Python and pip, as witnessed
by the output of `pip debug --verbose`:

```
(...)
pip version: pip 21.0.1 from c:\buildenv\python_64\lib\site-packages\pip
(python 3.6)
sys.version: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC
v.1916 64 bit (AMD64)]
(...)
```

Unfortunately, after upgrading `cryptography` from 2.9.2 to 3.3.2, the
AppVeyor task started to fail with the following error (during runtime, the
package installation process itself works fine):

```
(...)
  File
"c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\__init__.py",
line 7, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File
"c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py",
line 117, in <module>
    from cryptography.hazmat.bindings.openssl import binding
  File
"c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py",
line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: DLL load failed: The specified module could not be found.
```

I have verified that on both systems there is the `cryptography` OpenSSL
DLL available
(`c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\_openssl.pyd`)
and that it has exactly the same size (3,107,328 bytes). I have also
checked and tried using a newer version of `cryptography` (3.4.7) on
AppVeyor but the problem does not go away.

Has there been any change in handling of the OpenSSL DLL between 2.9.2 and
3.3.2? Or is there any other reason that might hint the source of issues?
Is it possible that the DLL itself introduced some new system dependency
that has to be installed first?

Thank you,
?ukasz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210415/25387d14/attachment.html>

From alex.gaynor at gmail.com  Thu Apr 15 08:30:23 2021
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Thu, 15 Apr 2021 08:30:23 -0400
Subject: [Cryptography-dev] Problems with importing OpenSSL on Windows
In-Reply-To: <CAAUp-Cjm=RAce_3MjYiWhDeVyVzXd_J7=z2US0dkiwn3tJ6K8Q@mail.gmail.com>
References: <CAAUp-Cjm=RAce_3MjYiWhDeVyVzXd_J7=z2US0dkiwn3tJ6K8Q@mail.gmail.com>
Message-ID: <CAFRnB2UN-6F7js+XgznWfyRtju5z7AH5Q0oMow4t0ZXVTUYD=A@mail.gmail.com>

Hi Lukasz,

Unfortunately this issue doesn't ring a bell at all for me. I don't believe
we made any fundamental changes to our DLLs -- all we've done is update
OpenSSL versions over this period. We have definitely not introduced any
new system dependencies.

Alex

On Thu, Apr 15, 2021 at 8:10 AM ?ukasz Hanuszczak via Cryptography-dev <
cryptography-dev at python.org> wrote:

> Hello,
>
> Our team uses two continuous integration systems that build and package
> the Windows distribution of our project. One of them is internal and runs
> on Windows 7, the other one is AppVeyor (running Windows 2012, which is
> based on Windows 8 best to my knowledge) that works with our open-source
> repository on GitHub.
>
> Both systems use the same script that initializes the virtual environment,
> upgrades pip, installs dependencies. The only significant difference is
> that the internal pipeline does not have internet access and uses vendored
> wheels (though downloaded from PyPI upfront).
>
> Both systems use exactly the same version of Python and pip, as witnessed
> by the output of `pip debug --verbose`:
>
> ```
> (...)
> pip version: pip 21.0.1 from c:\buildenv\python_64\lib\site-packages\pip
> (python 3.6)
> sys.version: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC
> v.1916 64 bit (AMD64)]
> (...)
> ```
>
> Unfortunately, after upgrading `cryptography` from 2.9.2 to 3.3.2, the
> AppVeyor task started to fail with the following error (during runtime, the
> package installation process itself works fine):
>
> ```
> (...)
>   File
> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\__init__.py",
> line 7, in <module>
>     from cryptography.hazmat.backends.openssl.backend import backend
>   File
> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py",
> line 117, in <module>
>     from cryptography.hazmat.bindings.openssl import binding
>   File
> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py",
> line 14, in <module>
>     from cryptography.hazmat.bindings._openssl import ffi, lib
> ImportError: DLL load failed: The specified module could not be found.
> ```
>
> I have verified that on both systems there is the `cryptography` OpenSSL
> DLL available
> (`c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\_openssl.pyd`)
> and that it has exactly the same size (3,107,328 bytes). I have also
> checked and tried using a newer version of `cryptography` (3.4.7) on
> AppVeyor but the problem does not go away.
>
> Has there been any change in handling of the OpenSSL DLL between 2.9.2 and
> 3.3.2? Or is there any other reason that might hint the source of issues?
> Is it possible that the DLL itself introduced some new system dependency
> that has to be installed first?
>
> Thank you,
> ?ukasz
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev
>


-- 
All that is necessary for evil to succeed is for good people to do nothing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210415/3f20e138/attachment.html>

From alex.gaynor at gmail.com  Thu Apr 15 09:42:28 2021
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Thu, 15 Apr 2021 09:42:28 -0400
Subject: [Cryptography-dev] codecov security incident
Message-ID: <CAFRnB2XZWzzJgDpwFP+0Q3E+paEbBCmYyxTJYL_+ZR20qZ=EQw@mail.gmail.com>

This morning codecov disclosed a security incident:

https://about.codecov.io/security-update/

This incident allowed an attacker to run code in environments that use
codecov
for uploading coverage. Since the Python Cryptographic Authority uses
codecov
across our projects for tracking coverage, we wanted to provide an update on
how we were impacted by this incident.

tl;dr; we're not impacted

We use codecov across many many testing jobs. However, none of these jobs
contain access to any secrets or tokens or any sort. Further, we do not not
use
codecov in any jobs that generate release artifacts (e.g. built wheels).
Because
our CI infrastructure relies on ephemeral environments; jobs are isolated
from
each other -- gaining access to a job that runs tests cannot be pivoted to
access to a job that generates a release wheel. 100% of our source code is
open
source, including all release infrastructure, so there was no source code to
steal.

The fact that we were not impacted reflects deliberate decisions to minimize
the attack surface of the parts of our release infrastructure that could
impact
the integrity of our artifacts.

Nevertheless, we are going to be investigating whether there exist good
alternatives to codecov -- this reflects not just this security incident,
but
also a long running pattern of instability in codecov's service. Our
constraints are our coverage infrastructure needs to be able to merge
coverage
results from multiple jobs and languages and compute aggregate coverage and
reports and it needs to be highly reliable. We encourage folks to send
recommendations our way.

Regards,
Alex & Paul

-- 
All that is necessary for evil to succeed is for good people to do nothing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210415/13f247f6/attachment-0001.html>

From hanuszczak at google.com  Wed Apr 21 12:30:58 2021
From: hanuszczak at google.com (=?UTF-8?Q?=C5=81ukasz_Hanuszczak?=)
Date: Wed, 21 Apr 2021 18:30:58 +0200
Subject: [Cryptography-dev] Problems with importing OpenSSL on Windows
In-Reply-To: <CAFRnB2UN-6F7js+XgznWfyRtju5z7AH5Q0oMow4t0ZXVTUYD=A@mail.gmail.com>
References: <CAAUp-Cjm=RAce_3MjYiWhDeVyVzXd_J7=z2US0dkiwn3tJ6K8Q@mail.gmail.com>
 <CAFRnB2UN-6F7js+XgznWfyRtju5z7AH5Q0oMow4t0ZXVTUYD=A@mail.gmail.com>
Message-ID: <CAAUp-CgXCXGOnHpKxXSz-45Hzg5b-DFuXDsdHeMSRDPP8yW4=w@mail.gmail.com>

Hello again,

On Thu, Apr 15, 2021 at 2:30 PM Alex Gaynor <alex.gaynor at gmail.com> wrote:

> I don't believe we made any fundamental changes to our DLLs -- all we've
> done is update OpenSSL versions over this period.
>

There is one thing I noticed that changed between 2.9.2 and 3.3.2: all DLL
files in `cryptography\hazmat\bindings` (`_constant_time`, `_openssl`,
`_padding`) in 2.9.2 are suffixed with platform compatibility tags (so it
is `_openssl.cp36-win_amd64.pyd`), whereas in 3.3.2 they are not (it is
just `_openssl.pyd`). While this might not be the culprit here, it is
definitely suspicious?do you have any ideas what caused this change and
whether this can affect how the DLL is loaded?

Thank you,
?ukasz


>
We have definitely not introduced any new system dependencies.
>
> Alex
>
> On Thu, Apr 15, 2021 at 8:10 AM ?ukasz Hanuszczak via Cryptography-dev <
> cryptography-dev at python.org> wrote:
>
>> Hello,
>>
>> Our team uses two continuous integration systems that build and package
>> the Windows distribution of our project. One of them is internal and runs
>> on Windows 7, the other one is AppVeyor (running Windows 2012, which is
>> based on Windows 8 best to my knowledge) that works with our open-source
>> repository on GitHub.
>>
>> Both systems use the same script that initializes the virtual
>> environment, upgrades pip, installs dependencies. The only significant
>> difference is that the internal pipeline does not have internet access and
>> uses vendored wheels (though downloaded from PyPI upfront).
>>
>> Both systems use exactly the same version of Python and pip, as witnessed
>> by the output of `pip debug --verbose`:
>>
>> ```
>> (...)
>> pip version: pip 21.0.1 from c:\buildenv\python_64\lib\site-packages\pip
>> (python 3.6)
>> sys.version: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC
>> v.1916 64 bit (AMD64)]
>> (...)
>> ```
>>
>> Unfortunately, after upgrading `cryptography` from 2.9.2 to 3.3.2, the
>> AppVeyor task started to fail with the following error (during runtime, the
>> package installation process itself works fine):
>>
>> ```
>> (...)
>>   File
>> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\__init__.py",
>> line 7, in <module>
>>     from cryptography.hazmat.backends.openssl.backend import backend
>>   File
>> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py",
>> line 117, in <module>
>>     from cryptography.hazmat.bindings.openssl import binding
>>   File
>> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py",
>> line 14, in <module>
>>     from cryptography.hazmat.bindings._openssl import ffi, lib
>> ImportError: DLL load failed: The specified module could not be found.
>> ```
>>
>> I have verified that on both systems there is the `cryptography` OpenSSL
>> DLL available
>> (`c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\_openssl.pyd`)
>> and that it has exactly the same size (3,107,328 bytes). I have also
>> checked and tried using a newer version of `cryptography` (3.4.7) on
>> AppVeyor but the problem does not go away.
>>
>> Has there been any change in handling of the OpenSSL DLL between 2.9.2
>> and 3.3.2? Or is there any other reason that might hint the source of
>> issues? Is it possible that the DLL itself introduced some new system
>> dependency that has to be installed first?
>>
>> Thank you,
>> ?ukasz
>> _______________________________________________
>> Cryptography-dev mailing list
>> Cryptography-dev at python.org
>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>
>
>
> --
> All that is necessary for evil to succeed is for good people to do nothing.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210421/96e27d17/attachment.html>

From alex.gaynor at gmail.com  Wed Apr 21 12:33:21 2021
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Wed, 21 Apr 2021 12:33:21 -0400
Subject: [Cryptography-dev] Problems with importing OpenSSL on Windows
In-Reply-To: <CAAUp-CgXCXGOnHpKxXSz-45Hzg5b-DFuXDsdHeMSRDPP8yW4=w@mail.gmail.com>
References: <CAAUp-Cjm=RAce_3MjYiWhDeVyVzXd_J7=z2US0dkiwn3tJ6K8Q@mail.gmail.com>
 <CAFRnB2UN-6F7js+XgznWfyRtju5z7AH5Q0oMow4t0ZXVTUYD=A@mail.gmail.com>
 <CAAUp-CgXCXGOnHpKxXSz-45Hzg5b-DFuXDsdHeMSRDPP8yW4=w@mail.gmail.com>
Message-ID: <CAFRnB2WagmCO823HEFiS=FkUYTBVGLX6BHMYOwH2mJPoZfw18g@mail.gmail.com>

I'd guess it's related to upgraded version of build tools in our release
pipeline -- possibly related to abi3?

Alex

On Wed, Apr 21, 2021 at 12:31 PM ?ukasz Hanuszczak <hanuszczak at google.com>
wrote:

> Hello again,
>
> On Thu, Apr 15, 2021 at 2:30 PM Alex Gaynor <alex.gaynor at gmail.com> wrote:
>
>> I don't believe we made any fundamental changes to our DLLs -- all we've
>> done is update OpenSSL versions over this period.
>>
>
> There is one thing I noticed that changed between 2.9.2 and 3.3.2: all DLL
> files in `cryptography\hazmat\bindings` (`_constant_time`, `_openssl`,
> `_padding`) in 2.9.2 are suffixed with platform compatibility tags (so it
> is `_openssl.cp36-win_amd64.pyd`), whereas in 3.3.2 they are not (it is
> just `_openssl.pyd`). While this might not be the culprit here, it is
> definitely suspicious?do you have any ideas what caused this change and
> whether this can affect how the DLL is loaded?
>
> Thank you,
> ?ukasz
>
>
>>
> We have definitely not introduced any new system dependencies.
>>
>> Alex
>>
>> On Thu, Apr 15, 2021 at 8:10 AM ?ukasz Hanuszczak via Cryptography-dev <
>> cryptography-dev at python.org> wrote:
>>
>>> Hello,
>>>
>>> Our team uses two continuous integration systems that build and package
>>> the Windows distribution of our project. One of them is internal and runs
>>> on Windows 7, the other one is AppVeyor (running Windows 2012, which is
>>> based on Windows 8 best to my knowledge) that works with our open-source
>>> repository on GitHub.
>>>
>>> Both systems use the same script that initializes the virtual
>>> environment, upgrades pip, installs dependencies. The only significant
>>> difference is that the internal pipeline does not have internet access and
>>> uses vendored wheels (though downloaded from PyPI upfront).
>>>
>>> Both systems use exactly the same version of Python and pip, as
>>> witnessed by the output of `pip debug --verbose`:
>>>
>>> ```
>>> (...)
>>> pip version: pip 21.0.1 from c:\buildenv\python_64\lib\site-packages\pip
>>> (python 3.6)
>>> sys.version: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC
>>> v.1916 64 bit (AMD64)]
>>> (...)
>>> ```
>>>
>>> Unfortunately, after upgrading `cryptography` from 2.9.2 to 3.3.2, the
>>> AppVeyor task started to fail with the following error (during runtime, the
>>> package installation process itself works fine):
>>>
>>> ```
>>> (...)
>>>   File
>>> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\__init__.py",
>>> line 7, in <module>
>>>     from cryptography.hazmat.backends.openssl.backend import backend
>>>   File
>>> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py",
>>> line 117, in <module>
>>>     from cryptography.hazmat.bindings.openssl import binding
>>>   File
>>> "c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py",
>>> line 14, in <module>
>>>     from cryptography.hazmat.bindings._openssl import ffi, lib
>>> ImportError: DLL load failed: The specified module could not be found.
>>> ```
>>>
>>> I have verified that on both systems there is the `cryptography` OpenSSL
>>> DLL available
>>> (`c:\buildenv\python_64\lib\site-packages\cryptography\hazmat\bindings\_openssl.pyd`)
>>> and that it has exactly the same size (3,107,328 bytes). I have also
>>> checked and tried using a newer version of `cryptography` (3.4.7) on
>>> AppVeyor but the problem does not go away.
>>>
>>> Has there been any change in handling of the OpenSSL DLL between 2.9.2
>>> and 3.3.2? Or is there any other reason that might hint the source of
>>> issues? Is it possible that the DLL itself introduced some new system
>>> dependency that has to be installed first?
>>>
>>> Thank you,
>>> ?ukasz
>>> _______________________________________________
>>> Cryptography-dev mailing list
>>> Cryptography-dev at python.org
>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>
>>
>>
>> --
>> All that is necessary for evil to succeed is for good people to do
>> nothing.
>>
>

-- 
All that is necessary for evil to succeed is for good people to do nothing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cryptography-dev/attachments/20210421/0ba5d6dc/attachment-0001.html>