[Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Hi Steve!


> This should be trivial to do with the Python email package, too.  I
> don't really see that a separate module would be useful, since we'll
> want to extract a fixed set of headers (ARC- and DKIM-specified).  Of
> course it should be factored into a separate function (or perhaps a
> generic "extract_fields" function and a couple of derivatives with the
> DKIM list (just DKIM-Signature?) and the ARC list (ARC-Seal,
> ARC-Authentication-Results, and ARC-Message-Signature).
>
>
Depending on the message, if it has previous Authentication Results added
as a header, that can be extracted too.  The entire message can be parsed,
and then all the possible headers involved in the authentication process,
ie. DKIM signature, Authentication results, ARC headers,  can be
 extracted. If not found, a suitable flag can be set for them. ie. example
- if no previous arc headers were found, a flag can be set. This can later
be used in deciding the flow of the mail such as the "i" tag value, whether
we need to perform ARC authentication for the previous ARC headers, and
other fields that depend on the occurrences of any previous ARC set.

    Another factoring issue: should you "import dkimpy" and call

> dkimpy.foo, or should you "from dkimpy import foo, bar, baz"?
> (Doesn't need to be settled for a while, and you can even try both at
> a small cost in time and effort.)
>
> From what I've read, I should use the `from ... import ...` when the no of
methods required from the package is less, around 3-4, the entire package
is not required and when naming conflicts are to be minded.
But yes, I can settle it later during the coding part :)

 >Isn't this inaccurate?  This "i" must *match* the Seal "i", no?

Yes, I meant the same thing. The "i" tag for all the three AAR, AMS and the
AS will be the same , and will be one higher than the previous instance of
the "i" tag. If no previous instance is there, it will be 1.

 > a,t,s,d: These can again be obtained from the dkimpy package.
>  >
>  > bh: The body hash. This can be obtained from the package. Here, we set
> the
>  > canonicalization to 'relaxed' and get the body hash.
>
> "The package" == dkimpy, right?
>
> Yes.


>  > 3. ARC Authentication Results.
>  > The "i" tag simply takes on the value same as the above "i" tags.
>  > Now from our previous conversation, as you suggested, the
>  > authenticity of the previous MTA who sent us the mail is not sure
>  > to be trusted. So in the case where we don't trust the previous
>  > MTA, we will have to perform our own DMARC, SPF, DKIM testing of
>  > the recieved mail. If previous ARC chain exists i.e. cv=V, then we
>  > perform the ARC test too.
>
> I'm not sure you understood me.  We *always* verify the preceding
> MTA's claims, even if we trust them, because of spoofing and
> man-in-the-middle attacks on the Internet itself.  (More precisely,
> verification is required by the I-D, and the rationale is spoofing and
> MITM.)  The MTA we "may or may not trust" is our *own* MTA.
>
>
Yes. So for every mail we receive, we always perform the authentication
tests for spf,dkim, dmarc (and arc if present).


> No.  The mail cannot be proved authentic by ARC, but that may be due
> to changes to the message at intervening hops.  There may be more
> sophisticated tests (eg, a PGP signature on a MIME body) that can
> prove it authentic.
>
>
Okay.! So we need to only include the ARC Headers and forward the message
to the subscribers, and
leave it upto their MTAs to do the needful.

 > Now coming to the testing part. There can be a number of tests like
>  > verifying the generated ARC signature, changing the body of the
>  > message, failing when the implicitly signed AMS headers are changed
>  > and other such tests.
>
> This is a little vague, but testing is hard.  You'll learn it as you
> go along.
>

I will come up with at least a few concrete tests that need to be performed
for each of the modules given below (in the milestones), and run them by
you once before the proposal.

>
> I count about 9 well-defined tasks in your message that could be made
> into good milestones.  You should try to come up with your own list,
> but if you're really not happy with your list, send it to me and ask
> for help and I'll give you my ideas.
>
>
I have come up with the following probable milestones for the project. -
(The project has been divided into milestones on the basis of the separate
modules that will be created. Each module is a milestone).
1. ARC Authentication Result - spf verification code completed. tests
passed. merge request created.
2. ARC Authentication Result - dkim verification code completed. tests
passed. merge request created.
3. ARC Authentication Result - dmarc verification code completed. tests
passed. merge request created.
4. ARC Authentication Result - arc verification code completed. tests
passed. merge request created.
5. ARC Authentication Result - generate AAR from the previous milestones
code. tests passed. merge request created.
6. ARC Message Signature code completed. tests passed. merge request
created.
7. ARC Seal code completed. tests passed. merge request created.
8. Generate the ARC set of headers from the previous milestones code, and
prepend them to the message.
tests passed. merge request created.

*As you mentioned, branch coverage will be the aim behind all the tests for
each module (Branch coverage would mean considering all possible scenarios
of the workflow).

Notes -

1. I've broken down the AAR set into different milestones, since each
method will require the use of different functions and packages. (ie. spf,
dkim, dmarc, arc).

2. Regarding milestone 8, separate modules will be responsible for
generating the components of the ARC header set, and these can be combined
at the end for getting the complete ARC set. This will be useful for
testing purposes.

3. We need to perform the dmarc testing manually since the gs.dmarc package
only provides the dmarc policy query functionality. The gs.dmarc package
can be used to query the dmarc record of the RFC5322.From domain. Then we
can verify using the aspf and adkim relaxed/strict tag values and the
spf/dkim results whether the dmarc authentication is a pass or fail, as
given in the RFC7489 draft (dmarc draft). Is there a better alternative to
this in your knowledge?

I would like your opinion on these milestones, and if possible your ideas
can be merged with these to come up with a better list :)

Thanks.

Aditya.