Subject: Re: [ietf-dkim] list vs contributor signatures, was Wrong Discussion

From: Ian Eiloart <[email-masked]>
Date: Wed, 02 Jun 2010 12:04:07 +0100

--On 26 May 2010 11:48:53 -0700 Michael Thomas <[email-masked]> wrote:

>
>> Perhaps I'm missing something. I'm working with the mental model
>> that the underlying problem ADSP advocates would like to address
>> is phishing or brand protection, as they're the only concrete problems
>> I've seen mentioned.
>
> So you're on a tangent about something that's not even vaguely in our
> charter. ADSP doesn't claim to solve "phishing", it just allows
> bit-for-bit domain names to be safely tossed if they don't have a
> signature. How that integrates into the larger anti-phishing modules is
> frankly where the secret sauce lies for vendors.

Addressing an issue and solving it aren't the same things. Heck, neither is 
even a prerequisite for the other, given that some problems are 
self-resolving or solved accidentally.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 02 Jun 2010 12:08:14 +0100

--On 26 May 2010 14:00:54 -0700 Steve Atkins <[email-masked]> 
wrote:

>
> Given that, it's not something that will provide any benefit once ADSP is
> deployed - maybe just the opposite, as it will effectively neuter the
> approach you're currently using. You may win the battle of preventing use
> of the string "paypal.com" in the non-displayed part of the From: field,
> yet lose the war of protecting your users from phishers.
>

There's nothing "undisplayed" about the From header in my mail client. Mail 
clients that don't display the From header address probably should not be 
used. In future, I'll bet they'll display the DKIM/ADSP status, too.

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 02 Jun 2010 05:09:42 -0700

On 6/2/2010 4:08 AM, Ian Eiloart wrote:
> --On 26 May 2010 14:00:54 -0700 Steve Atkins<[email-masked]>
> wrote:
>>        You may win the battle of preventing use
> There's nothing "undisplayed" about the From header in my mail client.

That's nice, but it is no longer typical.  Far From: it....

>           Mail
> used.

As soon as you convince all of the major MUAs to change and as soon as those 
changes propagate into the user world, it will be reasonable to worry about 
whether displaying the information matters.

That is, it will be reasonable to then ask whether users will assess the 
validity of that information and make intelligent decisions based on it.  (Hint: 
  user interface works makes pretty clear that they won't...)

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 02 Jun 2010 12:50:46 +0100

--On 27 May 2010 14:57:06 -0700 Steve Atkins <[email-masked]> 
wrote:

>
> On May 27, 2010, at 2:22 PM, Steve Atkins thinkoed:
>>
>> Legitimate email from paypal:
>>
>>    72% rejected by ADSP
>>    28% not rejected
>>
>> Phishing emails using "paypal" in the From line:
>>
>>    39% rejected by ADSP
>>    61% rejected.
>
> That should be
>
>> Legitimate email from paypal:
>>
>>    72% rejected by ADSP
>>    28% not rejected
>>
>> Phishing emails using "paypal" in the From line:
>>
>>    39% rejected by ADSP
>>    61% not rejected.

How many used "<…@paypal.com>"?

> Cheers,
>  Steve

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 02 Jun 2010 12:58:08 +0100

--On 26 May 2010 15:51:33 -0400 Brett McDowell <[email-masked]> 
wrote:

> On May 26, 2010, at 1:42 PM, Steve Atkins wrote:
>
>>>>> I'm big on concrete examples. So how does your logical conclusion
>>>>> deal with these two situations?
>>>>>
>>>>>  $ host -t txt _adsp._domainkey.paypaI.me
>>>>>  _adsp._domainkey.paypaI.me descriptive text "dkim=discardable"
>>>>>
>>>>>  $ host -t txt _adsp._domainkey.paypal.com
>>>>>  _adsp._domainkey.paypal.com descriptive text "dkim=discardable"
>>>>>
>>>
>
> I would like to answer your question... but I can't grok it from these
> two ADSP look-ups.  Are you asking about what we do if someone other than
> ourselves registers a cousin domain?

He seems to be. There are several approaches that could be taken:

1. A publication mechansism that allows you to say which cousin  domains 
you have registered.

2. A filter checking domains with Hamming distance measurements. Like a 
magnifying glass casts a shadow round the focal point, I'd imagine that 
domains close to those with high reputation could be assigned negative 
reputation - unless they acquired reputation from (1) above.

3. It simply would not be advisable to use cousin domains. Though 
registering them to prevent use might be a good option.

---

From: Steve Atkins <[email-masked]>
Date: Wed, 2 Jun 2010 09:09:06 -0700

On Jun 2, 2010, at 4:50 AM, Ian Eiloart wrote:

> 
> 
> --On 27 May 2010 14:57:06 -0700 Steve Atkins <[email-masked]> wrote:
> 
>> 

100% of the legitimate email from paypal. 39% of the phishing email that used "paypal" in the From line.

(A large fraction of the 61% of the phishing email that ADSP considered legitimate did use "@paypal.com", though, IIRC. More use of friendly from and less use of associated domains than I've noticed in recent phishing mail in general.)

Cheers,
  Steve

---

From: Steve Atkins <[email-masked]>
Date: Wed, 2 Jun 2010 11:41:12 -0700

On Jun 2, 2010, at 10:59 AM, Brett McDowell wrote:

> On May 28, 2010, at 1:08 AM, Steve Atkins wrote:
> 
>> Paypal is rather a special case, as they actively register
>> many, many domains in a lot of TLDs that contain the word
>> paypal or some misspelling of it, both proactively and in
>> response to enforcement. I didn't consider those domains
>> as triggering an ADSP rejection for a number of reasons.
>> One is that many (most?) of them would have been acquired
>> by paypal though enforcement action after the phishes were
>> sent, and the other is that it's a behaviour (registering a
>> huge number of domains purely to deny them to others)
>> that's atypical and that doesn't scale.
>> 
>> Havning said that, I did spot check quite a lot of the phishes that
>> I'd tagged as "not rejected" and the vast majority weren't
>> using domains I'd expect paypal to have proactively reserved
>> (paypal.net, for instance) - they were mostly using the word
>> "paypal" in the friendly from, the local part or a subdomain of
>> the domain part. Of those that weren't of that form many were
>> things like "@paypal-access.com" and suchlike. So I think those
>> two numbers are likely accurate to within a few percent or better.
> 
> Your numbers were so far off from what we see that I was perplexed, but now it's clear why.  
> 
> In reality we do register many of the domains you assumed we don't (like paypal.net) and we are not unique in that practice.  We have over a thousand of these domains parked.  
> 
> The result of this simple error in assumption has skewed your data to the point where it is no longer representative.

There's no error in assumption. The data is fairly accurate as measured (and also reasonably representative of the behaviour of a mixed-use domain, I think). 

First, as I said above

>> they were mostly using the word
>> "paypal" in the friendly from, the local part or a subdomain of
>> the domain part

... in other words your argument doesn't apply at all to most of the phishing email that ADSP failed to deal with.

Second...

    steve$ host -t txt _adsp._domainkey.paypal.net    
    _adsp._domainkey.paypal.net has no TXT record
    steve$ host -t txt paypal.net
    paypal.net has no TXT record
    
... I wasn't going to mention it, but you brought it up. The MX for paypal.net will also give a 2xx response to any RCPT TO in the paypal.net domain.

Third, as I mentioned above, even if you were publishing ADSP records for lookalike domains, most of those lookalike domains appear to have been acquired after enforcement of a phishing issue - meaning that there would have been no ADSP records when the phishing mails were sent, and your ownership of them now is irrelevant.

Fourth, as I mentioned above, even if all you said was valid, registering thousands of domains in order to make ADSP sort-of work against phishing isn't something that scales, either in terms of domain name system nor the expense. If ADSP requires users to spend tend of thousands of dollars a year to maintain domain registrations in order for it to have a significant effect on phishing then it's not something that's really going to scale to more than a handful of senders.

>  I feel compelled to point out this error since several people on the list have been quoting your data since you circulated it and are likely to draw erroneous conclusions from it.

I understand you want to discredit the statistics - they show that as measured at my inbox, based on paypal related email, ADSP dkim=discardable as currently used by PayPal to combat phishing is significantly less useful than flipping a coin. 72% false positive, 61% false negative.

These statistics are quite valid, and reasonably representative. The only non-representative thing about them is that they're measured at my mailbox, rather than at a consumer mailbox of a user who has no interaction with PayPal other than transactional and bulk email (which is a big difference, sure, but it seems unlikely paypal would want to drop all usage of paypal.com other than for their bulk mail)..

So rather than attacking the statistics that demonstrate that your current usage of ADSP simply doesn't work for it's stated purpose it would be better to concentrate on fixing the serious issues that those stats demonstrate. I'd start with the hideously high false positive rate, as that's an easier thing to fix than the hideously high false negative rate.

We've already discussed some ways to improve the false positive rate (dedicating entire domains to B2C bulk mail rather than mixed use, rewriting every mailing list manager in the world and so on). Constraining ADSP usage to domains that are solely used to send bulk email to consumer ISPs strikes me as something that would be acceptable to many potential users, and would avoid most of the false positive issues.

Cheers,
  Steve

---

From: Brett McDowell <[email-masked]>
Date: Wed, 02 Jun 2010 13:59:48 -0400

The result of this simple error in assumption has skewed your data to the point where it is no longer representative.  I feel compelled to point out this error since several people on the list have been quoting your data since you circulated it and are likely to draw erroneous conclusions from it.

-- Brett

---

From: Michael Thomas <[email-masked]>
Date: Wed, 02 Jun 2010 12:06:37 -0700

> Fourth, as I mentioned above, even if all you said was valid, registering thousands of domains in order to make ADSP sort-of work against phishing isn't something that scales, either in terms of do…

I'd be willing to bet a good chunk of money that Paypay -- and lots of other domains --
have this practice regardless, and preceding ADSP. ADSP use supplements that current
best practice for phishing target domains. Which is sort of the point.

Mike

---

From: Brett McDowell <[email-masked]>
Date: Wed, 02 Jun 2010 15:15:25 -0400

> best practice for phishing target domains. Which is sort of the point.
+1

---

From: Brett McDowell <[email-masked]>
Date: Wed, 02 Jun 2010 15:28:48 -0400

> ... in other words your argument doesn't apply at all to most of the phishing email that ADSP failed to deal with.

I don't think you are following me... the receivers who are performing ADSP look-ups and enforcing "discardable" would have blocked a ton of that email you are reporting would have made it through.

> ... I wasn't going to mention it, but you brought it up. The MX for paypal.net will also give a 2xx response to any RCPT TO in the paypal.net domain.

...and I wasn't going to mention that I tried to work with you off-list to get more information about your phish from paypal.net but you didn't respond.   If you get a chance, please do send that along.

> Third, as I mentioned above, even if you were publishing ADSP records for lookalike domains, most of those lookalike domains appear to have been acquired after enforcement of a phishing issue - mea…

As I've reported before, we have over 100 millions reasons (blocked attacks from our registered domains) why you are wrong about this assertion.

> Fourth, as I mentioned above, even if all you said was valid, registering thousands of domains in order to make ADSP sort-of work against phishing isn't something that scales, either in terms of do…

You have this backwards.  Major brands register cousin domains for more reasons than phishing.  This practice pre-dated ADSP and it will continue with or without ADSP.  It's simply an operational fact that is relevant to any debate about how ADSP can/should be used.

> I understand you want to discredit the statistics

and I understand you wanted to report statistics to discredit ADSP... actually I don't understand that but it is certainly consistent with your arguments

> - they show that as measured at my inbox, based on paypal related email, ADSP dkim=discardable as currently used by PayPal to combat phishing is significantly less useful than flipping a coin. 72% false positive, 61% false negative.

but your processing assumptions about what would get through vs. what would bounce was wrong.

> 
> These statistics are quite valid, and reasonably representative.

now I'm just repeating myself...

> The only non-representative thing about them is that they're measured at my mailbox, rather than at a consumer mailbox of a user who has no interaction with PayPal other than transactional and bulk email (which is a big difference, sure, but it seems unlikely paypal would want to drop all usage of paypal.com other than for their bulk mail)..

yep, that's rather significant.

> 
> So rather than attacking the statistics that demonstrate that your current usage of ADSP simply doesn't work for it's stated purpose

it works quite well actually, as a component to a multi-part strategy.

> it would be better to concentrate on fixing the serious issues that those stats demonstrate. I'd start with the hideously high false positive rate,
> as that's an easier thing to fix than the hideously high false negative rate.
> 

because MLM's break DKIM signatures, it does help the immediate pain to move employee mail to non-ADSP=discardable domains... like I have.

> We've already discussed some ways to improve the false positive rate (dedicating entire domains to B2C bulk mail rather than mixed use, rewriting every mailing list manager in the world and so on).…

somewhere in there is a nugget of a good idea for a Senders BCP.

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Wed, 2 Jun 2010 15:30:37 -0400

> -----Original Message-----
> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of Michael Thomas
> Sent: Wednesday, June 02, 2010 3:07 PM
> To: Steve Atkins
> Cc: DKIM List
> Subject: Re: [ietf-dkim] list vs contributor signatures, was Wrong
> Discussion
> 
> > Fourth, as I mentioned above, even if all you said was valid,
> registering thousands of domains in order to make ADSP sort-of work
> against phishing isn't something that scales, either in terms of
domain
> name system nor the expense. If ADSP requires users to spend tend of
> thousands of dollars a year to maintain domain registrations in order
for
> it to have a significant effect on phishing then it's not something
that's
> really going to scale to more than a handful of senders.
> 
> have this practice regardless, and preceding ADSP. ADSP use
supplements
> that current

Thousands of domains registered and counting. Been doing it well before
ADSP and view it as only one of the things that owners of abused domains
do.

Mike

---

From: Steve Atkins <[email-masked]>
Date: Wed, 2 Jun 2010 14:09:02 -0700

> ...and I wasn't going to mention that I tried to work with you off-list to get more information about your phish from paypal.net but you didn't respond.   If you get a chance, please do send that a…

I did[1].

It looks like your mailsystem is discarding email it shouldn't. There's a copy at http://tupid.org/paypal1.txt if you can't find it.

It seems that paypal is not currently monitoring phishing, nor doing anything to deter it, on 99.9% of the domains they own, so have no real idea of what phishing is going on. 

Pointing those thousand domains at a catch-all mailserver with a wildcard MX and looking for bounces and spamfilter rejections might be a good way of getting metrics about how phishers respond to domains being owned by paypal over time. Those same metrics after adding SPF and ADSP records for those domains over time would be interesting. http://blog.wordtothewise.com/2010/05/how-to-disable-a-domain/ has some examples of how to set those up.

That's the sort of data gathering I was suggesting you do, rather than just a bald count of DNS queries, when I looked at the numbers for my mailbox. (There's a copy of my raw data at http://tupid.org/paypal1.sql.txt if anyone is interested in running their own model against it.)

(I'm not going to respond to the other misunderstandings unless someone really wants me to. I'm guessing most people are long past tl;dr at this point.)

Cheers,
  Steve

[1] May 28 13:54:47 fruitbat postfix/smtp[31990]: DA551814E6: to=<[email-masked]>, relay=gort.ebay.com[216.113.167.215]:25, delay=0.74, delays=0.17/0/0.45/0.11, dsn=2.0.0, status=sent (250 ok:  Message 769193797 accepted)
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html