Subject: Re: [ietf-dkim] SSP + SPF records in DNS (was: New Issue: Do we need SSP record for DKIM=unknown?)

From: Douglas Otis <[email-masked]>
Date: Tue, 1 Jan 2008 15:15:28 -0800

On Dec 30, 2007, at 7:15 PM, Frank Ellermann wrote:

> Douglas Otis wrote:
>
>> DKIM requires acceptance of the message body.
>
> Obviously.  Receivers will still reject messages before DATA where  
> that makes sense from their POV, e.g. because they use services  
> explained in the ASRG SIQ and DNSBL drafts, or in the ASRG LMAP  
> draft, i.e. what is now RFC 4405 and RFC 4408.

Schemes based upon IP address path registrations create email delivery  
problems.  Even heavily phished domains publish open-ended IP address  
record sets.

> Hector's SMTP HEAD draft offered a kind of compromise between before  
> vs. after DATA, but it won't help with DKIM, and it's not clear how  
> to combine it with the "selective reject" I-D.

This header first mechanism could permit validation of DKIM signatures  
prior to the acceptance of the message body.  The signature's  
parameter bh= captures the canonicalized body part.  : )

>> DKIM's ability to offer reliable indications of misbehaviour will  
>> bolster that of content filtering results.
>
> DKIM without SSP (or similar) doesn't have such ability.

Without SSP, DKIM can prevent false positive detection of spoofing for  
heavily phished domains.  While SSP could help automate anti-phishing  
efforts, the message body still requires analysis.  Often deception of  
the recipient is not dependent upon a spoofed email-address.

>> DKIM can help overcome problems created by the unreliable  
>> application IP address path registration strategies.
>
> DKIM doesn't come near to the envelope sender address, we made sure  
> that it doesn't depend on SMTP.  Backscatter is mainly an SMTP issue  
> (+ RFC 3798 + 3834 + sieve vacation).

Without depending upon _any_ IP address path registration, a single  
TPA-SSP query can indicate whether a signature is authorized, and  
whether a signature is required for MailFrom.

>> DKIM can prove helpful at restoring integrity to SMTP, even after  
>> the dot. : )
>
> DKIM operates on messages (SDU, 2822), not on envelopes (PDU,  
> 2821).  You can use DKIM on the other side of a UUCP link if you  
> have access on DNS, or whereever you find a "fresh" message/rfc822  
> (e.g. as MIME part), it's not based on SMTP.  For various reasons  
> DKIM does *not* try to solve the SMTP issue backscatter.

TPA-SSP, in one small DNS transaction, mitigates a DSN problem by  
leveraging DKIM's ability to avoid path registration problems.

>> The "everything at once" approach represents a potential overhead  
>> that can well exceed that of accepting messages.
>
> Receivers are free to say "enough already" when it pleases them, in  
> fact they MUST and SHOULD do this (RFC 4408 10.1).

Evaluation libraries for SPF do not return the amount of DNS traffic  
generated, and many even fail to impose recommended transactional  
limits.  Danger is increased by opening up SPF evaluation to more  
names per message.

> IMO "a", "all", "include", "ip4", "ip6", and "redirect" are  
> essential for a simplified SPF.  The "ptr" is no problem, the  
> Authentication-Results I-D offers a similar test with similar  
> processing limits.

Allowing bad actors to link a succession of resource record  
transactions for each evaluation from any domain represents an unusual  
risk.  Resolving IP addresses for each host discovered, or expanding  
macros based upon various components of an email-address local-part  
significantly increases these risks.  When IPv6 comes into greater  
use, a minimum number of transactions may need to increase. : (

> The "exists" is rather exotic at the moment (in theory SES is a nice  
> idea), and tossing %{l} out, or maybe limit "mx" to at most one per  
> record, because the original RMX idea makes sense, might be possible  
> in a 4408bis.

DNS is designed to return short lists of host names, or IP addresses  
of a host.  Returning all IP addresses employed on behalf of a domain  
breaks this architecture, as illustrated by premature time-outs of SPF  
transactional sequences.  The premature time-outs disable critical DNS  
congestion avoidance.  Path registration by-name could have safely  
leveraged DNS name hierarchy.  A list of authorized parent domains,  
combined with something like CSV, could avoid the dozens or even  
hundreds of sizeable DNS transactions now imposed by the SPF  
evaluation process.

Nevertheless, DKIM is _not_ about IP address path registration.  DKIM  
even avoids IP address path registration issues!

> Nobody is going to learn new TPA-SSP tricks from scratch, unless you  
> go to draconian simplifications like "nobody must send from an MTA  
> to an MX of a 3rd party, when that MTA isn't listed as MX of the  
> sender".  This just won't fly, no matter how long we could argue  
> that it's the idea of an STD 10 "MX" - I'm not even sure that it's  
> the case.

With TPA-SSP, spoofing of MailFrom is mitigated before a DSN is  
considered.  At any point in time, there are more than 30 million  
sources of DSNs issued to MailFroms spoofed at legitimate MTAs.  TPA- 
SSP allows MailFrom domains to either validate or invalid an  
association with DKIM signature, in the same manner used for From  
headers.  Instead of the From header domain, the MailFrom domain is  
queried.

>> reliance upon unreliable IP address path registration can and  
>> should be avoided.
>
> Without time travel and removing RFC 1123 5.3.6(a) before it was  
> published all you can get is "BTNRP", better than no reverse path  
> (i.e. SPF).  If some senders get their "permitted IP enumeration"  
> wrong that is no problem for receivers, it's a problem for these  
> senders, they fix it.
>
> [MS issue with new RR types]

DKIM, by using TPA-SPP, can mitigate DSN problems and remain compliant  
with RFC1123 5.3.6(a).  SPF/Sender-ID can not.

>> MS might even view demise of DNS an advantage, as their alternative  
>> is heavily patented.
>
> Sometimes they're forced to adopt standards if they don't want to  
> lose market share.  Does this issue still exist in Vista, or does it  
> only affect XP and older MS platforms ?

An alternative MS name space was introduced with Vista, and upgraded  
XP.  RPC/DNS translations are at network boundaries and involve both  
reading and publishing.  The latest MS DNS 2003 warns against  
publishing unsupported RRs, (MS DNS represents less than 3% of  
Internet Name servers).  About 1 in 5 domains with a name server  
publish SPF TXT records, and of these, about 1 in 50,000 also publish  
newer 99 RRs.

>> Concerns remained regarding enterprise support of new RRs.
>
> We all know that the SPF RR has an "MS-compatible" sibling.
>
> At the moment "sender policy framework" stands for "we can do  
> 'sender permitted from' or even 'purported responsible address', and  
> HELO", not much for a wannabe-framework.  It is kind of obvious to  
> add SSP *IFF* it fits.

SPF/Sender-ID assertions pertain to IP address path registrations and  
not to DKIM signatures.  DKIM specifically avoids relating a IP  
address path to that of a DKIM signature.

> RFC 5016 tries to make sure that it won't fit (5.3 point 5), but IMO  
> nobody cares about cryptic abbreviations in a text-like DNS  
> record.   Of course it is good to use readable and/or intuitive  
> terms while talking about such abbreviations... :-)

A rather fundamental reality of the original 576 byte assured MTU and  
the 512 byte DNS message size is being ignored. : (

>> SPF now demands at least 10 linked records be permitted for each  
>> name validated.
>
> You can have *at most* 10 "redirect".  As you well know, I deleted  
> various synonyms for "intentionally telling what is known to be not  
> true".  If you eat up the overall limit 10 for "redirect" you have  
> no "a", no "mx", no "include", etc.  The "redirect" is a v=spf1 and  
> spf2.0/* feature, adding SSP to the SPF RR introduced by say v=ssp1  
> would define its own syntax, for 25 bytes it doesn't need a  
> "redirect" feature - it would need the 25 bytes in the RR set  
> (shared with SPF).

A chain of SPF records supports bulky manual IP address lists.  Each  
chain may also include a reporting record and subsequent IPv4/IPv6  
address resolution transactions for the included hosts.

>> The impact of IPv6 may cause this sequence to be further extended
>
> Nonsense, with ten "redirect" you have at least 4500 bytes to  
> enumerate permitted IPs.  You can and should aggregate them to  
> CIDRs, it is no problem if you permit many IPs which are never  
> sending mail as long as they are your IPs or known to behave.

This over-states the available payload and the 10x chaining scheme's  
capacity to scale.

> Besides this "redirect" business has nothing to do with SSP:
>
> Assuming SSP does share the SPF RR, and that for a given SPF RR set  
> the v=spf1 and spf2.0/* records already use the UDP limit (~450),  
> then it would be necessary to split one of these v=spf1 or spf2.0/*  
> records with "redirect" to make room for the 25 bytes in a v=ssp1  
> record.  They could also decide that the PRA experiment is now  
> finished and pull the spf2.0/pra record to make room for v=ssp1.   
> After all SSP is supposed to be "better" than spf2.0/pra, isn't  
> it ?  Of course I'm not convinced that "1st author" is a sound idea.

Adding yet another mechanism to an already bloated approach takes  
unwise to a new extreme, especially as address diversity increases.

>> Using SPF to validate a DKIM domain in some manner might then  
>> impose this 10xplus sequence for each of several possible signatures.
>
> Nobody proposed to use SPF to "validate a DKIM domain in some  
> manner".  SPF validates envelope sender addresses, it allows "accept  
> and bounce" after a PASS.  SPF is for SMTP, not for DKIM.  If you  
> reject a SSP FAIL at your border you don't need SPF, neither the  
> protocol(s), nor the record(s).
>
> I proposed to put the SSP info into a separate DNS record in the  
> existing SPF RR set.  You don't need to look at any v=spf1 or spf2.0/ 
> * record after a query=spf, you'd look for the v=ssp1 record.

Original proponents failed to limit per message name evaluations.   
What prevents DKIM misuse when SPF includes DKIM policy?  Why wouldn't  
a DKIM specific domain be evaluated using SPF, and seen as analogous  
to that of the PRA?

>> While not imposing change when consensus is not clear is  
>> understandable, a lack of a discovery record for assured acceptance  
>> can lead to serious DNS related risks.
>
> Let's say that there are still more pressing problems with DNS  
> attacks than your idea to abuse SPF to query addresses in ten MX sets.

This concern extends to macro expansions using the local-part of the  
email-address.  It is extremely difficult to determine whether a DDoS  
had been sourced by SPF.  Adding more names to be evaluated further  
enables exploits to threaten major Internet interconnects.  In  
addition, these exploits enable DNS poisoning.  There would be no  
remedy or obvious symptom for this exploit.  Email logs would be  
unlikely to offer any clue without extensive correlation with that of  
DNS resolvers.

> RFC 4408 is still the only RFC I'm aware of mentioning that MX  
> queries can be an issue, with MUSTard and a SHOULD to mitigate it.   
> If that turns out to be not good enough 4408bis will have to fix it,  
> e.g. as noted above.

If path registration is still desired, a scheme using a by-name  
approach would require less effort, be easier to manage, and would  
ensure minimal transactional overhead that is not easily exploited.

Normal use of MX records will not require a comprehensive IP address  
list be generated.  To satisfy SPF, every IP address of every host  
name must be resolved.  SPF allows for 10 MX records in different  
domains.  Each MX record may contain as many as 10 host names.  The  
evaluation process must be fully repeated for each different email- 
address being evaluated within the message.  An evaluation process may  
need to be repeated two or three times per message.  The need to fully  
repeat an evaluation process is necessitated by SPF's macro expansion  
feature.  RFC4408 abuses MX records well beyond any other experimental  
protocol.

>> The entire email policy concept is affected, not just SSP or TPA- 
>> SSP.  The alternative for asserting policy without an assured  
>> discovery record is to walk the DNS tree for A or AAAA records.   
>> The lack of a specific discovery record represents a serious  
>> architectural extensibility defect for SMTP!
>
> Yes, that's what SPF is for.  Trying to "reinvent" it again is a  
> dubious plan, simplifying it is more attractive.

SPF is strictly about IP address path registration.

TPA-SSP makes _no_ assertion about IP address path registrations.    
TPA-SSP, unlike SPF, provides a means to safely associate MailFrom  
with DKIM policies.

SPF does _not_ offer this ability.  SPF already suffers from extremely  
dangerous feature bloat.  To preserve security, it is crucial that SPF  
not be extended with yet more mechanisms, or in anyway assert DKIM  
policies.

-Doug

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Julian Mehnle <[email-masked]>
Date: Wed, 2 Jan 2008 00:04:49 +0000

> a DKIM specific domain be evaluated using SPF, and seen as analogous

First, you seem to be completely missing Frank's point.  In the above he 
is talking about sharing the "SPF" (code 99) RR typespace among v=spf1/ 
spf2.0 and v=ssp1 records, not about applying any v=spf1/spf2.0/v=spf3 
semantics to SSP or DKIM.

Second, your analogy (which as I said misses Frank's point) to the re-use 
of v=spf1 records for PRA evaluation (as propagated by Microsoft) is 
broken.  Even if some v=spf3 came along and defined a new "dkim:" 
mechanism, there would be no general risk of confusion whatsoever with a 
separate SSP record scheme, because any additional and potentially 
conflicting semantics would be entirely up for domain owners to declare!

IOW, if a domain owner published some "v=spf3 dkim:..." record according 
to whatever specification of v=spf3 or "dkim:", it would be that domain 
owner's decision.  The problem with the v=spf1/PRA fiasco was that the 
Sender ID specification (and any implementations thereof) did NOT leave 
it up to domain owners what their existing v=spf1 records were supposed 
to mean.

I am not going to discuss the details of your FUD about SPF being a threat 
to the livelihood of DNS and the internet as a whole, but let me summa- 
rize the SPF project's conclusion as follows for the benefit of others:

> had been sourced by SPF.

And that determination is of academic interest at best, because there are 
indeed other ways for DNS-traffic-based DoS attacks that are just as 
suitable as, if not more than, SPF:

http://www.openspf.org/auth/draft-otis-spf-dos-exploit_Analysis#rebuttal
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Julian Mehnle <[email-masked]>
Date: Wed, 2 Jan 2008 00:11:20 +0000

Julian Mehnle wrote:
> [...] there are indeed other ways for DNS-traffic-based DoS attacks that
> are just as suitable as, if not more than, SPF:
>

Er, that was supposed to read

  http://www.openspf.org/draft-otis-spf-dos-exploit_Analysis#rebuttal

Sorry.

---

From: Douglas Otis <[email-masked]>
Date: Wed, 2 Jan 2008 15:53:57 -0800

On Jan 1, 2008, at 4:04 PM, Julian Mehnle wrote:

>>> existing SPF RR set.  You don't need to look at any v=spf1 or  
>>> spf2.0/* record after a query=spf, you'd look for the v=ssp1 record.
>>
>> What prevents DKIM misuse when SPF includes DKIM policy?  Why  
>> wouldn't a DKIM specific domain be evaluated using SPF, and seen as  
>> analogous to that of the PRA?
>
> First, you seem to be completely missing Frank's point.  In the  
> above he is talking about sharing the "SPF" (code 99) RR typespace  
> among v=spf1/spf2.0 and v=ssp1 records, not about applying any  
> v=spf1/spf2.0/v=spf3 semantics to SSP or DKIM.

Of the small minority of domains publishing SPF records, 99.98% of  
these domains publish _only_ TXT RRs.  These records typically contain  
CIDR listings of IP addresses encompassing the SMTP clients.  These  
SMTP clients are thereby authorized to issue messages containing the  
email-addresses passed to the SPF processing routines.

However, SPF lacks a means to select a subset of IPv4 or IPv6  
addresses.  A realistic estimate of the text based CDIR payload might  
be about 140.  These CIDRs must span both IPv6 and IPv4 ranges.   
Cascading MX/hostname RRs is limited to 100, depending on limits  
imposed upon the hostname resolution process.  Some libraries limit MX/ 
hostname cascades to 50 CIDRs.  Any additional RRs placed at the email- 
address domain reduces overall CIDR capacity due to the RR property  
overhead, in addition to the RR content.  In addition, (as if this  
would ever happen) any versioning of SPF/PRA records _MUST_ also  
occupy the _same_ initial location.

Why should DKIM only use type 99 RRs, when the DKIM WG would not  
consider a new RR acceptable?  Essentially no one is using RR type 99  
now.  Any overhead reduction, albeit with larger payloads, is to be  
found by leveraging TXT RRs.  It it really wise to invite unrelated  
TXT resource records into this congested location, already (ab)used by  
a PRA evaluation process?

> Second, your analogy (which as I said misses Frank's point) to the  
> re-use of v=spf1 records for PRA evaluation (as propagated by  
> Microsoft) is broken.  Even if some v=spf3 came along and defined a  
> new "dkim:" mechanism, there would be no general risk of confusion  
> whatsoever with a separate SSP record scheme, because any additional  
> and potentially conflicting semantics would be entirely up for  
> domain owners to declare!

Each email-address evaluation may require processing an entire SPF  
record set.  This set may span as many as 111 resource record  
transactions, ignoring checks for TXT or type 99 and reporting  
records.  Subsequent processing of any different email-address, even  
within the same domain and message, may again span another 111  
transactions.  Complete reprocessing is needed to support expansion of  
macros contained within SPF records.  Any suggestion SPF should  
include DKIM policy greatly increases the risks related to SPF  
exploits that might then leverage the additional evaluations of DKIM  
specific email-address.  The risk caused by the number of DNS  
transactions employed, and a bad actor's ability to cache their  
attack.  More than 20% of domains override negative caching to  
intervals less than an hour.  Rampant SPF evaluations can result in a  
devastating attack completely free for spamer/attacker.  MTAs  
employing SPF are unlikely to detect when an SPF exploit is progress,  
nor will forensics of MTA logs be of much use.  An attack, free in  
almost every sense.

> IOW, if a domain owner published some "v=spf3 dkim:..." record  
> according to whatever specification of v=spf3 or "dkim:", it would  
> be that domain owner's decision.  The problem with the v=spf1/PRA  
> fiasco was that the Sender ID specification (and any implementations  
> thereof) did NOT leave it up to domain owners what their existing  
> v=spf1 records were supposed to mean.
>
> I am not going to discuss the details of your FUD about SPF being a  
> threat to the livelihood of DNS and the internet as a whole, but let  
> me summarize the SPF project's conclusion as follows for the benefit  
> of others:

The FUD is an attempt to increase awareness of risks remaining within  
the SPF protocol.  After all, DDoS are expensive for the victims. : (

OpenSPF's dismissed SPF transactional exploitation concerns based upon  
a bad actor's ability to also exploit DNS NS records?  Their example  
exploit was to return a bogus NS list misdirecting DNS transactions to  
other servers.  Their example somehow dismissed concerns related to  
the traffic generated by SPF's per message/MTA/MUA/recipient  
evaluation of a PRA and MailFrom email-address.  SPF still allows bad  
actors to exploit both SPF _and_ other resource records in tandem.   
OpenSPF expressed a rather callus lack of concern regarding their  
disabling of DNS congestion avoidance, allowing attacking records to  
be cached and then generate random queries, or to the very distributed  
nature of their mechanism.  SPF provides bad actors a tool that can be  
used for staging DDoS attacks, and even poisoning DNS itself. This  
exploit is beyond being controlled by firewalls, ACLs, and even  
universal implementation of BCP38. : (

>> This concern extends to macro expansions using the local-part of  
>> the email-address.  It is extremely difficult to determine whether  
>> a DDoS had been sourced by SPF.
>
> And that determination is of academic interest at best, because  
> there are indeed other ways for DNS-traffic-based DoS attacks that  

SPF leverages MX records, where MX records themselves offer a moderate  
amplification of about 10.  SPF limits the number of different MX  
records employed to 10, and the resolution of subsequent hostnames to  
10 each (10 x 10 = 100 transactions).  However, SPF permits references  
to MX record to be based upon labels generated by macro expanding  
email-address local-parts. : (

Normal use of MX records will seldom cause all hostnames to be  
resolved at once when in different domains.  Hostname resolution is  
normally interspersed with attempts to contact the host, after all.   
SPF MX records represents a different situation.  Addresses resolved  
might have a CIDR mask applied and then immediately compared against  
the IP address of the SMTP client.  In other words, the SPF MX record  
is not used to locate a receiving host, and lacks the typical  
processing impediments.

SPF/Sender-ID also does not impose reasonable transactional limits  
when considering how these active records might be abused.  Once  
attacking records have been cached, a spammer/attacker can recycle at  
no additional resource cost to them, once their campaign extends  
beyond a negative caching interval.  An interval often inappropriately  
limited by receiving domains.  An attack with a gain of 10 using MX  
records then transitions to a gain approaching infinity, since spam  
enabling the attack is still accepted by appending a "+all" or its  
equivalent as the final SPF mechanism.

SPF records might be processed by routines taking shortcuts that fail  
to impose even recommended transactional limits.  However, SPF also  
recommends starting evaluation of a different record set while still  
within a DNS back-off interval.  Not waiting protects an MTA receiving  
process, very much at the expense of the DNS protocol.

Several proponents of SPF remarked they think DNS is broken.  DNS is  
clearly not well suited for returning massive address lists, at  
least.  Path registration, if desired, should have been done by-name,  
in a manner far more compatible with DNS.

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 3 Jan 2008 10:36:07 +0100

Douglas Otis wrote:

  [Julian wrote:]
>> you seem to be completely missing Frank's point.

Indeed, the v=ssp1 (or similar) idea might help with
_some_ (by far not all) of Dave's objections against
the current SSP draft, and needs precisely the same
number of query=spf to get a v=ssp1 record as the 
current draft.  Not 10, not 100, only _two_ queries.

Eric said that _ssp._domainkey.dom.example is good to
keep the administration of _domainkey.dom.example and
the corresponding _ssp._domainkey.dom.example in one
hand, with that the v=ssp1 idea is at best "plan B"
IFF the current _ssp._domainkey plan is rejected.

Two objections against _ssp._domainkey I have in mind:
1 - It's not trivial in conjunction with wildcards
2 - Queries waste bandwidth and are pointless without
    significant deployment (i.e. _ssp publishers).

> Of the small minority of domains publishing SPF records,
> 99.98% of these domains publish _only_ TXT RRs.

Everybody knows that query=spf in practice still means
"query=txt and don't panic if query=spf causes errors",
and that won't change for some years.  It would be the
same if SSP is forced to share "type 99".  

> These records typically contain CIDR listings of IP 
> addresses encompassing the SMTP clients.

That's irrelevant for SSP.  Sharing "type 99" might help
with the two objections against _ssp._domainkeys listed
above assuming that that turns out to be necessary.

I don't  say that I *support* these objections - Eric's
argument pro _ssp._domainkey is also very plausible.
   
> A realistic estimate of the text based CDIR payload 
> might be about 140.  These CIDRs must span both IPv6
> and IPv4 ranges.

I guess you're now again talking about SPF records, but
I've no clue what "140" might be.  ip4:127.0.0.1/24 for
a /24 is 16 bytes, for IPv6 it would be somewhat longer.

For a/24 (IPv4), a//32 (IPv6), or a/24//32 (dual) it's
shorter.  Did you count 20 bytes in a:example.org/24//32
and then replace 11 for example.org by 130 to get 140 ?

> Some libraries limit MX/ hostname cascades to 50 CIDRs.

There are no "cascades" in SPF, let alone in SSP, and if
a pre-MARID SPF library doesn't implement the post-MARID
RFC 4408 file a bug report, use a fresher implementation,
fix it, or hire somebody to fix it.   

> Why should DKIM only use type 99 RRs

See above, SSP might if forced, not DKIM.  

 [Is it]
> really wise to invite unrelated TXT resource records
> into this congested location, already (ab)used by a PRA 
> evaluation process?

PRA doesn't look at v=ssp1, nobody with an IQ above zero
uses PRA looking at v=spf1, and pro-PRA arguments for its
spf2.0/pra are limited to "plausible after an erroneous
decision to ignore the envelope sender address" (and then
IMO more plausible than SSP's "first author" approach).  

I can't tell how many "type 99" (or rather corresponding
TXT) record users don't have 25 bytes for SSP directly, I
can't tell how many would be willing to pull spf2.0/pra
as "nice try, but now it's over" in favour of SSP, and I
can't tell if 25 is realistic:  RFC 5016 4.7 and 5.4 (2)
claim that 25 is not good enough.  OTOH a future SSP RFC
is free to obsolete parts of the informational RFC 5016.
 
> Each email-address evaluation may require processing an
> entire SPF record set.  This set may span as many as 111
> resource record transactions, ignoring checks for TXT or
> type 99 and reporting records.

If you don't ignore it you get 112 instead of 111.  I've
no idea what reporting records are.  The 111/112 affects
receivers wishing to evaluate an SPF policy with 10 MX
mechanisms, each with 10 names.  For receivers wishing to
evaluate only v=ssp1 that's irrelevant, admittedly they'd
need both query=txt and query=spf for some years.  

Again, this is a "plan B" if the two objections against SSP
at _ssp._domainkey listed above turn out to be bad enough.
I can't tell if that's the case.

> Rampant SPF evaluations can result in a devastating attack
> completely free for spamer/attacker.

In your scenario the attacker (not a spammer) maintains the
MX records, each with 10 bogus addresses of the victim.  It
is 1 query=mx to the attacker for 10 query=a to the victim,
that's cheap but certainly not "completely free".  Your idea
to use various MX records selected by different local parts
would stabilize this 1:10 amplification, for less traffic on
the attacking NS they'd stick to their evil MX records.  

Now that we have discussed it again also on this list let's
drop it *here*.  A future 4408bis will have to fix the MX
issue if needed, it's simple enough.  E.g. limit the mx per 
record to one, deprecate mx in favour of include, or limit
the number of NXDOMAIN - in fact a SHOULD in RFC 4408 can be
implemented by limiting NXDOMAIN as noted on the page cited
by Julian (see the last paragraph of the rebuttal, "a viable
void-lookups limit for SPF might lie between 2 and 5.")

> The FUD is an attempt to increase awareness of risks remaining
> within the SPF protocol.

We are aware of it, and "interoperability and implementation
reports" should IMHO state *how* they implement the relevant
RFC 4408 SHOULD.    
 
> SPF leverages MX records, where MX records themselves offer
> a moderate amplification of about 10.

No, the limit of 10 names per mx-mechanism is a hard RFC 4408
limit, it is no limit of "MX records themselves".

> SPF permits references to MX record to be based upon labels
> generated by macro expanding email-address local-parts. : (

One fresh malicious MX triggered per mail is enough to get an
MX amplification, that you insist on using local part macros
for this attack is IMO irrational.  I'm no fan of SPF's local
part macros, they are odd for UTF8SMTP, and messy for quoted
strings with leading / adjacent / trailing dots, but they are
not essential for an attack scenario.

> Normal use of MX records will seldom cause all hostnames to
> be resolved at once when in different domains.

"Call back verification" is an example of no "normal use" -
spammers have a reason why they forge "plausible" MAIL FROMs.

> spam enabling the attack is still accepted by appending a
> "+all" or its equivalent as the final SPF mechanism.

The attacker isn't interested in delivering spam, that could
help to locate the NS of the attacker.  Your idea to run a
DoS attack and a spam campaign simultaneously is irrational.

> Several proponents of SPF remarked they think DNS is broken.

RFC 1123 broke responsible SMTP forwarding, and it's kind of
odd that determining a DNS zone cut doesn't work, if that's
what your're talking about

This DNS oddity resulted in CSV's tree walk, it might affect
the current SSP proposal, but SPF can do without it:  It's
*unnecessary* to protect domains when they have neither an
MX nor an address with an SMTP server.  It's *possible* but
not *necessary* - forging such addresses makes no sense for
spammers, such addresses are "implausible", they fail in a
"call back verification".

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Thu, 3 Jan 2008 18:56:32 -0800

On Jan 3, 2008, at 1:36 AM, Frank Ellermann wrote:

> Douglas Otis wrote:
>
>> A realistic estimate of the text based CDIR payload might be about  
>> 140.  These CIDRs must span both IPv6 and IPv4 ranges.
>
>  Did you count 20 bytes in a:example.org/24//32 and then replace 11  
> for example.org by 130 to get 140 ?

Estimates based upon RR records approaching a system limits would be  
problematic.  For review, the estimate of 140 assumed an average for  
either IPv4 or IPv6 CIDRs to take about 22 characters, within payloads  
able to accommodate typical ancillary information, without which a  
truncation might otherwise result.

A high percentage of abusive email emanates from compromised systems.   
While large CIDRs may reduce the number required, this tactic reduces  
protection.  Unexpected sources of email are always popping up.

> There are no "cascades" in SPF, let alone in SSP, and if a pre-MARID  
> SPF library doesn't implement the post-MARID RFC 4408 file a bug  
> report, use a fresher implementation, fix it, or hire somebody to  
> fix it.

It remains clear the SPF exploit concern is still not understood.  In  
addition, victims of SPF parsing exploits might not be publishing or  
checking SPF records, or even using email.

Initial SPF records causing a "cascade of transactions" can be cached  
and then execute entirely different sequences.  It does not matter  
whether a cascade includes A, AAAA, PTR, MX, or TYPE 99/TXT records.   
For a spammer/attacker, the attack can recycle targets and become free  
once the duration of a spam campaign exceeds a receiver's negative  
caching.

Receivers parsing RFC4408 SPF records to evaluate email-addresses  
allows bad actors to generate a cascade of DNS transactions from  
cached records, such as:

(SPF) -> MX -> Hostname-A or Hostname-AAAA
            -> Hostname-A or Hostname-AAAA
            ...
         MX -> Hostname-A or Hostname-AAAA
            -> Hostname-A or Hostname-AAAA
            ...
       ...
or

(SPF) -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> SPF
      -> TXT

>> Is it really wise to invite unrelated TXT resource records into  
>> this congested location, already (ab)used by a PRA evaluation  
>> process?
>
> PRA doesn't look at v=ssp1, nobody with an IQ above zero uses PRA  
> looking at v=spf1, and pro-PRA arguments for its spf2.0/pra are  
> limited to "plausible after an erroneous decision to ignore the  
> envelope sender address" (and then IMO more plausible than SSP's  
> "first author" approach).

If DKIM SSP were to use type 99 RRs to avoid repeated transactions,  
expect use of TXT records for the same reason.  In essence, type 99 is  
unlikely to ever become adopted, as it will be seen as a wasted  
transaction to be avoided as well.

Of course, the TXT RR at the base domain does not scale and should be  
avoided. : )

> I can't tell how many "type 99" (or rather corresponding TXT) record  
> users don't have 25 bytes for SSP directly, I can't tell how many  
> would be willing to pull spf2.0/pra as "nice try, but now it's over"  
> in favour of SSP, and I can't tell if 25 is realistic:  RFC 5016 4.7  
> and 5.4 (2) claim that 25 is not good enough.  OTOH a future SSP RFC  

RR-sets consume 12 bytes for RR properties, where strings consume an  
additional byte or so for length.  Per the current SSP 41 byte string  
of "v=ssp1 dkim=strict handling=process t=n:s" a total of 54 bytes is  
needed for its separate RR that could comprise about 18% of available  
resource space.  When more than two different protocols utilize the  
same TXT RR and location, newer versions are unlikely to remain viable.

>> records.
> If you don't ignore it you get 112 instead of 111.  I've no idea  
> what reporting records are.

SPF has an exp mechanism to fetch a macro expanded messages that could  
be added to a DSN.  While an unlikely method to spam, this adds yet  
another possible TXT transaction in the sequence.

> The 111/112 affects receivers wishing to evaluate an SPF policy with  
> 10 MX mechanisms, each with 10 names.  For receivers wishing to  
> evaluate only v=ssp1 that's irrelevant, admittedly they'd need both  
> query=txt and query=spf for some years.
>
> Again, this is a "plan B" if the two objections against SSP at  
> _ssp._domainkey listed above turn out to be bad enough. I can't tell  
> if that's the case.

The macro expansion of SPF records to evaluate DKIM related email- 
addresses must be avoided.  To ensure this, plan B should be avoided.

> In your scenario the attacker (not a spammer) maintains the MX  
> records, each with 10 bogus addresses of the victim.  It is 1  
> query=mx to the attacker for 10 query=a to the victim, that's cheap  
> but certainly not "completely free".  Your idea to use various MX  
> records selected by different local parts would stabilize this 1:10  
> amplification, for less traffic on the attacking NS they'd stick to  
> their evil MX records.

This conclusion overlooks cached records induce the sequence of  
transactions by leveraging SPF macros.  The resulting transactions no  
longer require _any_ of an attacker's/spammer's resources, and becomes  
a zero cost attack.

> Now that we have discussed it again also on this list let's drop it  
> *here*.  A future 4408bis will have to fix the MX issue if needed,  
> it's simple enough.

Further limiting the SPF MX mechanism will not solve the caching  
exploit created by SPF's macro expansion feature.  Preventing  
expansion of local-part components will require exploits to expend  
more resources initially, but then any label component is able to  
recycle an attack from cached records, again at zero cost. : (

> E.g. limit the mx per record to one, deprecate mx in favour of  
> include, or limit the number of NXDOMAIN

Such limitations will affect those who depend upon MX/CIDRs for  
delegating to domains that are not publishing SPF records.

> - in fact a SHOULD in RFC 4408 can be implemented by limiting  
> NXDOMAIN as noted on the page cited by Julian (see the last  
> paragraph of the rebuttal, "a viable void-lookups limit for SPF  
> might lie between 2 and 5.")

Unless a different version of the SPF record is published, and where  
older versions are removed, there is no way to prevent the use of  
unsafe routines.  IMHO, converting to different version is highly  
unlikely.  There is no viable means of escape.

> reports" should IMHO state *how* they implement the relevant RFC  
> 4408 SHOULD.

Unfortunately, even you are still unaware of the exploit concern, nor  
are there any simple fixes.

A focus on security should move email away from IP address path  
registration.  Converting to DKIM in conjunction with TPA-SSP would  
provide safe alternatives for controlling DSNs in a manner compatible  
with RFC1123.

>> SPF leverages MX records, where MX records themselves offer a  
>> moderate amplification of about 10.
>
> No, the limit of 10 names per mx-mechanism is a hard RFC 4408 limit,  
> it is no limit of "MX records themselves".

However, size constraints of MX responses provides roughly comparable  
limits.

> One fresh malicious MX triggered per mail is enough to get an MX  
> amplification, that you insist on using local part macros for this  
> attack is IMO irrational.

Staging an attack at _zero_ resource cost to the attacker is  
irrational?  Is it rational to use a method that both reflects and  
greatly amplifies an attack compared to the spam itself?  Of course  
the spam would be sent regardless.  The draft that I wrote should not  
have estimate gain in this fashion.  It seems to have been too  
distracting.  I apologize for not being clearer.

> I'm no fan of SPF's local part macros, they are odd for UTF8SMTP,  
> and messy for quoted strings with leading / adjacent / trailing  
> dots, but they are not essential for an attack scenario.

Fortunately, resources consumed in staging an attack normally provides  
revenue when not attacking.  SPF's macro feature enables a free  
attack, which removes an important factor constraining an otherwise  
greater botnet problem.

>> resolved at once when in different domains.
>
> "Call back verification" is an example of no "normal use" - spammers  
> have a reason why they forge "plausible" MAIL FROMs.

It is also unlikely that call back verification will rapidly attempt  
to resolve all MX targets that are within different domains.  However,  
this will happen with SPF.

>> spam enabling the attack is still accepted by appending a "+all" or  
>> its equivalent as the final SPF mechanism.
>
> The attacker isn't interested in delivering spam, that could help to  
> locate the NS of the attacker.  Your idea to run a DoS attack and a  
> spam campaign simultaneously is irrational.

The NS of the attacker is likely to be a compromised system that is  
moved every few minutes.  A spammer/attacker is interested in  
capitalizing upon their botnet.  When their attack emanates from DNS  
resolvers used by recipients, the botnet remains hidden.  An attack  
that does not impair their spamming activities would be a generous gift.

> RFC 1123 broke responsible SMTP forwarding, and it's kind of odd  
> that determining a DNS zone cut doesn't work, if that's what your're  
> talking about

It would appear DKIM helps solve your concern about forwarding.  TPA- 
SSP can impose the desired constraints on the MailFrom without  
reliance upon IP address path registration. : )

> This DNS oddity resulted in CSV's tree walk

IMHO, Dave was right back then about validating SMTP client first.   
Publishing domain wide policy was considered by others to be a means  
to encourage CSV use.  However, it seems providers would rather have  
recipients depend solely upon email domain authorizations.  At least  
providers should not be troubled by a TPA-SSP authorization scheme.  : )

> it might affect the current SSP proposal, but SPF can do without  
> it:  It's *unnecessary* to protect domains when they have neither an  
> MX nor an address with an SMTP server.  It's *possible* but not  
> *necessary* - forging such addresses makes no sense for spammers,  
> such addresses are "implausible", they fail in a "call back  
> verification".

I am in even in favour of eliminating acceptance when there are no MX  
records.  Permitting just A records for discovery would causing the  
hunt for email policy to punish second level domain providers, or  
require policy be published adjacent to every A record.  Perhaps hosts  
could be publishing within a "_host." subdomain to avoid this problem?

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 15 Jan 2008 06:57:05 +0100

Douglas Otis wrote:

> Of course, the TXT RR at the base domain does not scale and should
> be avoided. : )

Yes, that's how we arrived at type 99, or here at _whatever labels.

> Per the current SSP 41 byte string of "v=ssp1 dkim=strict
> handling=process t=n:s" a total of 54 bytes is needed

IIRC I counted only "dkim=strict handling=deny" when I wrote "25",
but the syntax could be trimmed if necessary to squeeze this info
into another RR elsewhere with its own size limitations.  So far
nobody thinks it's necessary, but we're not yet ready with Dave's
two related issues.

> When more than two different protocols utilize the same TXT RR
> and location, newer versions are unlikely to remain viable.

Yes.  On the "SPF devel" list folks discussed that the UDP limit
will be soon anyway obsolete (= not good enough for IPv6).  

>> If you don't ignore it you get 112 instead of 111.  I've no
>> idea what reporting records are.

> SPF has an exp mechanism to fetch a macro expanded messages
> that could be added to a DSN.

ACK, with that you can get 113 as worst case.  JFTR what that
means:  A receiver tried q=SPF (TXT), and after getting nothing
q=TXT (SPF), or some variation counting as 2 queries.  After
that the sender policy contained 10 mx mechanisms (or 9 mx + 
1 ptr), more would hit a MUST NOT, so we are at a total of 12 
queries.  Each mx (or the ptr) stays at the limit of 10 names,
more hits another MUST NOT, for at total of 112.  

After that the final outcome is FAIL, therefore the receiver
could fetch an exp= modifuer record arriving at 113 queries.
An exp= does not hit the limit 10, because reporting a given
FAIL is more important than whining about the 11th exp= query.

I certainly agree that a policy with 9 or 10 mx can't be sound.

The intention in RFC 4408 is to make these limits very simple
to understand (count to ten, no matter what), and to arrive at
reliable limits (for sound non-attacking policies), i.e. if a
policy violates these hard limits all SPF implementations are
supposed to treat it as broken (PermError).  

It wasn't the intention to sanction "9 or 10 mx" as remotely
plausible:  For starters that would be at odds with a general
SHOULD following the three MUST NOTs for "ten".  I said this
before, it's perfectly okay if 4408bis introduces additional
limits.  As long as implementors understand what the purposes
are (not only anti-DoS, also reliable PermError) it's fine.

> The macro expansion of SPF records to evaluate DKIM related
> email-addresses must be avoided.

SPF could offer an SSP-shortcut anyway *IFF* that makes sense
for enough receivers supporting both. 

>> the limit of 10 names per mx-mechanism is a hard RFC 4408
 
> However, size constraints of MX responses provides roughly
> comparable limits.

Yes, that is how Wayne (the co-author) arrived at this "ten",
and we were aware that some domains have as much as eight MXs.

> It is also unlikely that call back verification will rapidly
> attempt to resolve all MX targets that are within different
> domains.

CBV is used for the decision to accept a given MAIL FROM, it
has to be fast (within the relevant timeout of the SMTP client).

> It would appear DKIM helps solve your concern about forwarding.

My concerns are somewhat limited to "I got one pseudo-551 caused
by an SPF FAIL after traditional forwarding since May 2004", and
I'm not aware of any case where MAIL FROM me vanished in a black
hole for the same scenario but accepting FAIL as "suspicious".

Of course others consider this as serious problem.  Likewise I'm
not much concerned about DKIM's potential mailing list issues,
and for SSP the caveat is clear enough.

> I am in even in favour of eliminating acceptance when there are
> no MX records.

If that flies (as a future common practice) describing it in an
RFC would be simple.  For the other way around, an attempt to 
introduce it in an RFC, I doubt that "they" will let you try it.
But I like the idea (outside of 2821bis).

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Tue, 15 Jan 2008 12:10:42 -0800

> nobody thinks it's necessary, but we're not yet ready with Dave's  

The v=ssp1 and 13 bytes were added since you suggested this  
information was within a separate RR.  The t=n:s was added for  
completeness.

>> When more than two different protocols utilize the same TXT RR and  
>> location, newer versions are unlikely to remain viable.
>

What is your estimate for "soon"?  Keep in mind, truncated responses  
are not problematic for all portions of DNS infrastructure.

>> SPF has an exp mechanism to fetch a macro expanded messages that  
>> could be added to a DSN.
>
> q=TXT (SPF), or some variation counting as 2 queries.  After that  
> the sender policy contained 10 mx mechanisms (or 9 mx + 1 ptr), more  
> would hit a MUST NOT, so we are at a total of 12 queries.  Each mx  
> (or the ptr) stays at the limit of 10 names, more hits another MUST  
> NOT, for at total of 112.

I never used this as an example because such an attack seemed  
unlikely.  The critical aspect is 10 to 100 new and different DNS  
transactions may occur from cached SPF records when receivers expand  
SPF macros for each email-address evaluated.  More than one evaluation  
may occur per message.  When evil SPF records are cached, an  
attacker's resources are not being expended.

> After that the final outcome is FAIL, therefore the receiver could  
> fetch an exp= modifuer record arriving at 113 queries. An exp= does  
> not hit the limit 10, because reporting a given FAIL is more  
> important than whining about the 11th exp= query.
>

The expansion of macros is where SPF becomes extremely dangerous.   
Macro expansion permits a resource free attack to be trigged by spam's  
email-addresses.  The most dangerous of these macros are those  
expanding components of the email-address local-part.  By leveraging  
the local-part, attacks do not need to wait for negative caching to  
expire before recycling targets.  Botnets are dangerous even at a gain  
of one.  SPF provides botnets a reflective attack mechanism that is  
not dependent upon the attacker's resources, and that can not be  
mitigated with BCP 38 or ACLs.  : (

> The intention in RFC 4408 is to make these limits very simple to  
> understand (count to ten, no matter what), and to arrive at reliable  
> limits (for sound non-attacking policies), i.e. if a policy violates  
> these hard limits all SPF implementations are supposed to treat it  
> as broken (PermError).

The original recursion depth of 20 records containing any number of  
mechanisms was so absurd, it made a 10 mechanism limit for some seem  
better than it was.  In addition, SPF time-outs were not well  
specified and do not ensure outstanding DNS operations conclude before  
continuing.

> plausible:  For starters that would be at odds with a general SHOULD  
> following the three MUST NOTs for "ten".  I said this before, it's  
> perfectly okay if 4408bis introduces additional limits.  As long as  
> implementors understand what the purposes are (not only anti-DoS,  
> also reliable PermError) it's fine.

Keep in mind, IP address path registration strategies are applied to  
more than just the MailFrom.  Some check IP address path registration  
for the PRA, in addition to the MailFrom.   Macros provided by SPF  
necessitates an entire process be restarted for every email-address  
evaluation, even for those within the same domain.  Whatever limit  
that is placed upon an IP path registration process, this must be  
multiplied by all possible evaluations.  MailFrom, PRA, and two or  
three DKIM signing domains might then increase SPF transactions by  
factors of 2 to 5.

>> addresses must be avoided.
>
> SPF could offer an SSP-shortcut anyway *IFF* that makes sense for  
> enough receivers supporting both.

SPF could be avoided entirely by adopting TPA-SSP.  TPA-SSP can scale  
using a single transaction and is easier to manage than:
  1) SPF
  2) sharing of private keys
  3) delegation of domains

>>> the limit of 10 names per mx-mechanism is a hard RFC 4408 limit,  
> Yes, that is how Wayne (the co-author) arrived at this "ten", and we  
> were aware that some domains have as much as eight MXs.

However, Wayne imposed a default limit of 5 host-names per MX to limit  
DDoS in his library.  This strategy causes erratic problems for some  
domains, especially when subjected to default SPF records. : (

>> attempt to resolve all MX targets that are within different domains.
>
> CBV is used for the decision to accept a given MAIL FROM, it has to  
> be fast (within the relevant timeout of the SMTP client).

CBV mechanisms often employ both low and high level result caching.   
CBV use of MX records is also be limited to the number of email- 
address evaluations.  CBV evaluations consume an attacker's resources  
and allow an attack of about a 10x amplification where the MX domain  
is contained within email logs.  Alternatively, SPF allows an attack  
that does not consume an attacker's resources, and where email logs do  
not indicate the MX domain.  Unfortunately, SPF can also provide a  
magnitude of attack orders of magnitude greater than the spam traffic  
being used as a trigger.  Spam traffic occurs whether or not it is  
used to trigger an attack.

> My concerns are somewhat limited to "I got one pseudo-551 caused by  
> an SPF FAIL after traditional forwarding since May 2004", and I'm  
> not aware of any case where MAIL FROM me vanished in a black hole  
> for the same scenario but accepting FAIL as "suspicious".

The concept was to ensure valid DSN are received, and that invalid DSN  
are dropped without reliance upon IP address path registration.

> Of course others consider this as serious problem.  Likewise I'm not  
> much concerned about DKIM's potential mailing list issues, and for  
> SSP the caveat is clear enough.

The goal of TPA-SSP was to ensure DKIM addresses are not also checked  
using an IP address path registration scheme.  TPA-SSP would be one  
transaction, whereas SPF could be order of magnitude more.

>> I am in even in favour of eliminating acceptance when there are no  
>> MX records.
>
> If that flies (as a future common practice) describing it in an RFC  
> would be simple.  For the other way around, an attempt to  introduce  
> it in an RFC, I doubt that "they" will let you try it. But I like  
> the idea (outside of 2821bis).

Agreed.  As additional email related policies are added, an MX record  
requirement offers a safe solution.  IMHO, DKIM should be able to make  
this assumption for any signed message, at least.

-Doug