Herding Cats or DMARC and all that
In an effort to improve email security and reduce spam, Apple, Gmail, and Yahoo have introduced, or are introducing, new sender requirements for at least some of DKIM, SPF and DMARC. Requirements apply to sending domains and are stricter for bulk senders, though the precise number of emails per day that might attract a sender that designation appears to be undisclosed in some cases.
Alphabet soup explained
DKIM allows an organisation involved in the transit of an email message to digitally sign a digest of the message and selected headers using a private key. This verifiably associates the message with a domain under the organisation's control. The DKIM specification RFC 6376 includes details of signature and digest mechanisms and construction.
Multiple actors may sign a message, such as the sender's organisation, an email service provider, or a relay. Receiving systems can check the digests match the corresponding message parts by recalculating them. The signature, which also authenticates the digests, can be verified using the corresponding public key in the signing domain's DNS-based DKIM records. This provides assurance that the message has not been altered in transit, and of the signer's identity. Where the sender's organisation signs outgoing messages with DKIM in accordance with the sender requirements below, the effect is to guarantee that the message originated from the domain in the sender's "from" address, and that certain message content has not been changed in transit, but not that the from address exists, is correct, or was the source of the message.
SPF provides assurance that the sending server is authorised to send mail for the domain in the "envelope from" (or "return path" address, by encoding acceptance criteria in DNS-based SPF records. "Envelope from" is not usually displayed by mail clients, but is retained in email headers for technical purposes, such as returning failed deliveries, and may legitimately differ from the sender's "from" address. This is commonly seen in messages re-sent by mailing lists to subscribers, where the mailing list, rather than the original sender, should be notified of any delivery problems.
DMARC allows domains to publish policy in DNS records for how receiving systems should treat messages failing SPF and/or DKIM checks. Receiving systems query the relevant "from" domain's DNS records to see if DMARC policy exists. Policy options for failing messages include quarantine (send to spam), reject (return to sender) or none (the discrepancy should be ignored).
While the failure of SPF or DKIM checks is unlikely to alert recipients (as such) to suspicious messages, it is theoretically an indicator for those curious about email headers, and some mail clients ("email apps") allow filtering on the basis of header content. Receiving systems may execute DMARC policy actions if the relevant "from" domain specifies any, but these are not always honoured.
Sender Requirements
At the time of writing:
Gmail's sender guidelines require DMARC, SPF and DKIM passes for "bulk senders", which gmail defines as those sending (ie. from which it receives) over 5,000 messages a day. For non-bulk senders, SPF OR DKIM is sufficient, at least for the time being.
Yahoo's sender best practices appear to be similar in this respect, without quantifying a bulk sender threshold.
Some media reports suggest Yahoo and Apple require both SPF and DKIM from all senders, but that does not appear to be supported by Yahoo's published sender guidelines if they remain accurate. Apple's requirements seem to be difficult to establish.
Useful and good practice though it may be, the above sender guidelines suggest DMARC is not required for non-bulk sender domains, though spoofed "from" addresses may fall foul of the DMARC policies of large email providers. Or even smaller ones.
The addition of ARC headers is recommended for those regularly forwarding or re-sending email automatically, such as mailing list providers. ARC provides an authentication scheme for such intermediate servers in chains where SPF or DKIM may be broken.
Mailing lists and other services modifying email content post-hoc may break DKIM by modifying fields used in digest and signature generation, which gives the appearance of tampering. Such fields commonly include the body, subject and "sender from" address, displayed by mail clients as "From". If the message is modified by the intervening system before forwarding or re-sending it, the original signature and digest may no longer match the content from which they were generated, so comparison fails. As a workaround to this issue, there is an "l" (L) DKIM header which can be used to indicate the length of the body used in the digest, which allows any additional content appended by eg. mailing lists or gateways to be ignored in recalculation of digests and signature verification.
SPF checks fail if no mechanism exists in the SPF record of the "envelope from" domain which qualifies the sending system as authorised to send mail for the domain. Before re-sending a message, a mailing list system should therefore change the "envelope from" address to one with a domain for which it has an SPF mechanism defined.
Analysis
The triad of DKIM, SPF and DMARC does not provide complete verification for common email use cases. This allows some abuse of systems and identity to appear indistinguishable from legitimate mail, as far as verifiable properties are concerned, due to what might be described as verification gaps in coverage offered by the combination of techniques.
The large-scale change required in improving email security is something of a barrier to wider reliance on DKIM in particular. Some hosting services aimed at SMEs and consumers (if not indeed enterprises too) do not allow DKIM selectors (or "key names") other than "default", if they provide per-domain signing, or DKIM signing at all. The popular open source mail gateway Proxmox as yet only supports outgoing DKIM-signing using a single signing key per deployment. Such a lack of flexibility may result in DKIM failure for a domain making use of multiple email service providers, if multiple selectors of the same name are associated with multiple key pairs for a single domain, or if in some cases there are none. Selectors should be unique within domains, being associated with both a sending system's private key, and what should be the corresponding public key in the domain's DNS record. Key sharing between providers is an alternative, but is undesirable from a security perspective.
A greater degree of flexibility by hosting services seems to be in order to allow for DKIM signing as standard, and multiple selectors too, if the importance of DKIM is to be increased. At least where non-bulk senders are concerned, slow progress and broadly permissive policies might be to be expected for some time yet.
Meow for now.
References
The Authenticated Received Chain (ARC) Protocol
RFC 8617
https://www.rfc-editor.org/rfc/rfc8617
Domain Keys Identified Mail (DKIM) Signatures
RFC 6376
https://www.rfc-editor.org/rfc/rfc6376.html
Domain-based Message Authentication, Reporting and Conformance (DMARC)
RFC 7489
https://www.rfc-editor.org/rfc/rfc7489.html
Internet Draft: DKIM Relay Problem
Chuang et al., 2023
https://www.ietf.org/archive/id/draft-ietf-dkim-replay-problem-00.html
Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1
RFC 7208
https://www.rfc-editor.org/rfc/rfc7208.html
Alphabet soup explained
DKIM allows an organisation involved in the transit of an email message to digitally sign a digest of the message and selected headers using a private key. This verifiably associates the message with a domain under the organisation's control. The DKIM specification RFC 6376 includes details of signature and digest mechanisms and construction.
Multiple actors may sign a message, such as the sender's organisation, an email service provider, or a relay. Receiving systems can check the digests match the corresponding message parts by recalculating them. The signature, which also authenticates the digests, can be verified using the corresponding public key in the signing domain's DNS-based DKIM records. This provides assurance that the message has not been altered in transit, and of the signer's identity. Where the sender's organisation signs outgoing messages with DKIM in accordance with the sender requirements below, the effect is to guarantee that the message originated from the domain in the sender's "from" address, and that certain message content has not been changed in transit, but not that the from address exists, is correct, or was the source of the message.
SPF provides assurance that the sending server is authorised to send mail for the domain in the "envelope from" (or "return path" address, by encoding acceptance criteria in DNS-based SPF records. "Envelope from" is not usually displayed by mail clients, but is retained in email headers for technical purposes, such as returning failed deliveries, and may legitimately differ from the sender's "from" address. This is commonly seen in messages re-sent by mailing lists to subscribers, where the mailing list, rather than the original sender, should be notified of any delivery problems.
DMARC allows domains to publish policy in DNS records for how receiving systems should treat messages failing SPF and/or DKIM checks. Receiving systems query the relevant "from" domain's DNS records to see if DMARC policy exists. Policy options for failing messages include quarantine (send to spam), reject (return to sender) or none (the discrepancy should be ignored).
While the failure of SPF or DKIM checks is unlikely to alert recipients (as such) to suspicious messages, it is theoretically an indicator for those curious about email headers, and some mail clients ("email apps") allow filtering on the basis of header content. Receiving systems may execute DMARC policy actions if the relevant "from" domain specifies any, but these are not always honoured.
Sender Requirements
At the time of writing:
Gmail's sender guidelines require DMARC, SPF and DKIM passes for "bulk senders", which gmail defines as those sending (ie. from which it receives) over 5,000 messages a day. For non-bulk senders, SPF OR DKIM is sufficient, at least for the time being.
Yahoo's sender best practices appear to be similar in this respect, without quantifying a bulk sender threshold.
Some media reports suggest Yahoo and Apple require both SPF and DKIM from all senders, but that does not appear to be supported by Yahoo's published sender guidelines if they remain accurate. Apple's requirements seem to be difficult to establish.
Useful and good practice though it may be, the above sender guidelines suggest DMARC is not required for non-bulk sender domains, though spoofed "from" addresses may fall foul of the DMARC policies of large email providers. Or even smaller ones.
The addition of ARC headers is recommended for those regularly forwarding or re-sending email automatically, such as mailing list providers. ARC provides an authentication scheme for such intermediate servers in chains where SPF or DKIM may be broken.
Mailing lists and other services modifying email content post-hoc may break DKIM by modifying fields used in digest and signature generation, which gives the appearance of tampering. Such fields commonly include the body, subject and "sender from" address, displayed by mail clients as "From". If the message is modified by the intervening system before forwarding or re-sending it, the original signature and digest may no longer match the content from which they were generated, so comparison fails. As a workaround to this issue, there is an "l" (L) DKIM header which can be used to indicate the length of the body used in the digest, which allows any additional content appended by eg. mailing lists or gateways to be ignored in recalculation of digests and signature verification.
SPF checks fail if no mechanism exists in the SPF record of the "envelope from" domain which qualifies the sending system as authorised to send mail for the domain. Before re-sending a message, a mailing list system should therefore change the "envelope from" address to one with a domain for which it has an SPF mechanism defined.
Analysis
The triad of DKIM, SPF and DMARC does not provide complete verification for common email use cases. This allows some abuse of systems and identity to appear indistinguishable from legitimate mail, as far as verifiable properties are concerned, due to what might be described as verification gaps in coverage offered by the combination of techniques.
The large-scale change required in improving email security is something of a barrier to wider reliance on DKIM in particular. Some hosting services aimed at SMEs and consumers (if not indeed enterprises too) do not allow DKIM selectors (or "key names") other than "default", if they provide per-domain signing, or DKIM signing at all. The popular open source mail gateway Proxmox as yet only supports outgoing DKIM-signing using a single signing key per deployment. Such a lack of flexibility may result in DKIM failure for a domain making use of multiple email service providers, if multiple selectors of the same name are associated with multiple key pairs for a single domain, or if in some cases there are none. Selectors should be unique within domains, being associated with both a sending system's private key, and what should be the corresponding public key in the domain's DNS record. Key sharing between providers is an alternative, but is undesirable from a security perspective.
A greater degree of flexibility by hosting services seems to be in order to allow for DKIM signing as standard, and multiple selectors too, if the importance of DKIM is to be increased. At least where non-bulk senders are concerned, slow progress and broadly permissive policies might be to be expected for some time yet.
Meow for now.
References
The Authenticated Received Chain (ARC) Protocol
RFC 8617
https://www.rfc-editor.org/rfc/rfc8617
Domain Keys Identified Mail (DKIM) Signatures
RFC 6376
https://www.rfc-editor.org/rfc/rfc6376.html
Domain-based Message Authentication, Reporting and Conformance (DMARC)
RFC 7489
https://www.rfc-editor.org/rfc/rfc7489.html
Internet Draft: DKIM Relay Problem
Chuang et al., 2023
https://www.ietf.org/archive/id/draft-ietf-dkim-replay-problem-00.html
Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1
RFC 7208
https://www.rfc-editor.org/rfc/rfc7208.html
Follow This Blog