Subject: [Asrg] Re: Usefulness of wholesale blocking of attachments for SMTP? (various)

From: [email-masked]
Date: Mon, 19 Apr 2004 13:29:30 -0500

Again, I'll respond to several comments together (and with a lot of snipping).

>> Once you identify the "real" machine where the transmission occurred, more 
>likely than not it will turn out to be a zombie-infected spambot.  So you get 
to throw bricks at a fellow victim. 

> No, not a fellow victim; a negligent abuse-enabler

Perhaps, but they're still victims too.  The fact of the matter is that there 
are a lot of technically VERY clueless folks out there, and unless you've come 
up with some kind of magic pill they can take which suddenly makes them 
technically savvy, there are going to CONTINUE to be clueless types out there.

We need to simply make it very much harder for viruses and worms to infect those 
people's machines.  Closing the attachment transmission window (especially for 
executable-type attachments) to just a sliver of what it is today will certainly 
help a great deal.  SPF and other such schemes is unlikely to help very much at 
all, because those machines which DO get infected can send viruses, worms, and 
spam using their authenticated SMTP.  It's outrageous and irresponsible that 
nearly everyone's mail clients freely accept and allow people to receive and 
click on things like PIF files, which are virtually NEVER used for anything 
good.  .EXE files are likewise almost never legitimately needed in E-mail, and 
those who DO need it usually know why, and from whom they need to be able to 
receive such stuff.  

We will NEVER solve the spam problem if we don't get a major handle on the virus 
and worm problem, and A-V software is NOT the ultimate solution because it 
basically all works on the "blacklist" principle (i.e. "if we don't know about 
it (yet) then it's probably okay."  While A-V software is a valuable component, 
and certainly helpful in disinfecting a machine once it's been infected, 
hopefully we can come up with a solution that's better than just go around 
forever swatting flies.  Let's close the window!!

SPF, challenge-response, and other such infrastructure-disrupting schemes 
basically don't achieve that because they are based on an unrealistic base 
assumption:  that spam necessarily uses bogus return addresses, and/or is sent 
by "known bad" machines belonging to spamhauses.  The first of those can be 
changed literally overnight, and the second hasn't been true for quite a long 
time already.

>snip

>>Perhaps it is, although it's hard to define "a bulk email event".  

> This discussion on-point

>>Rather than making mail filters poll a central site (these have in the past 
been 
>victimized by DDOS attacks), maybe one approach here might be to set up a 
>Yahoogroup that could be used to rapidly distribute these "disreputable domain 
>names" and IP addresses along with a utility which would add them to the 
>recipients' HOSTS files (and from there, to their incoming mail filters).

I will point out that this is again just a component, since it is another "20/20 
hindsight" approach much like A-V software solutions.  At most it inconveniences 
spammers in that they must burn through a neverending stream of disposable 
domain names and IP addresses;  unfortunately, they HAVE figured that out and 
long-term this only will serve (if we can get the propagation delay short enough 
for the "bad list" updates) to dramatically reduce the useful lifetime (perhaps 
to just a few hours) of these disposable domain names and IP addresses.

>>> Just today, I was at a friends office and my outgoing email was blocked 
>because cliff.concentric.net is now on a spam list.

>>Yes, and that's one of the problems with SPF, "authenticated users", 
>certificates and other such schemes.  Ultimately it's not very helpful to throw 
>rocks at fellow victims.

>[snip]

>>>>   I mail-server I use regularly (Indiana University) has taken, in 
>>>response to worms and other malware useing .pif. zip, exe, etc 
>>>attachments to spread their damage, has taken the (IMO) rather drastic 
>>>step of blocking almost *all* attachments ...

>>That's sort of the approach of Microsoft's new version of Outlook, where they 
>allow blocking by attachment extension.  That's better than nothing, but it 
>needs to be SOMEWHAT finer:  it needs to allow the recipient to enable specific 
>attachment types (and certain classes of HTML markup) to be received from 
>specific approved-and-trusted senders.

>>>>  Coincidentally, another list I'm on had a post from the admin of a
>>local university, talking about spam.  Here's what he said.
>>
>>(quote)
>>   I don't support any Windows systems, yet I seem to spend a huge amount
>>of time dealing with problems relating to Windows non-security.  During the
>>fall/winter term we had to deal with
>>
>>   1) When students returned to Residence in September, at least half of
>>      their machines were infected.

>> One of my colleagues recently dealt with a client whose system was "having 
>assorted problems".  Upon installing Spybot-Search-and-Destroy over TWO HUNDRED 
>instances of spyware were found on the unwitting user's computer.  There were 
>additional Spyware programs that SPYBOT S&D did NOT find, besides.

> You have to use Bazooka and Adaware besides.

I wasn't aware of Bazooka, thanks for the tip.

>>>One of the things that has been 
SINGULARLY unhelpful toward addressing the 
>problem of overloaded mail servers is this plague of HTML-burdened 
"alternative" 
>copies of E-mail messages.  It is rare indeed that these provide genuinely 
>valuable additional content;  instead they usually are loaded with gratuitous 
>graphic gizmos, Web bugs, possibly malicious scripting, misrepresented 
clickable 
>links, and text-as-image designed to evade content filters.  While I'll accept 
>that some folks can argue that their needs for HTML-burdened E-mail is 
>legitimate, certainly a lot of it is not.  Mail with HTML-burdened attachments 
>is typically 3x-5x larger than it would be as plain ASCII text.
>
>If HTML-burdened attachments were removed from non-whitelisted senders' E-mail 
>(and this would catch at least most of today's spam) then such mail would be 
>70-85% smaller in volume than it is today.  

> This is all documented.  It needs to be a new standard.  Feel free to
point offenders to <http://www.camblab.com/nugget/htmlmail.pdf>
(or send it to them)

I've typically in the past sent a similar reply to those sending me 
HTML-burdened E-mail messages, and I think I have a similar discussion of the 
topic at my personal Web site too... ah yes, here it is:

 http://personal.terabites.com/htmlmail.html

I've stopped sending out the rant to HTML-burdened mail senders, for the most 
part, since I've configured my own incoming mail filter to simply strip out the 
HTML-burdened alternative attachments, and the great majority of the HTML tags 
(if any) in the main body of the E-mail messages.  Some senders are simply 
belligerant, some just don't care, and many are just ignorant.  It's just easier 
and more pragmatic to handle things at the recipient end, which gives the 
recipient the control at the level of their own systems.

Regarding the camblab document, I think it's quite outrageous that you have to 
individually configure specific folks to send them non-HTML-burdened E-mail;  it 
seems that by default, all these mail clients seem to want to deceive folks into 
sending HTML-burdened mail, even when they don't have any idea that they are, or 
what the implications are.  The DEFAULT should be plain ASCII text, and anything 
else should require special handling.  (And likewise, plain ASCII text should be 
the only kind of E-mail that comes in unless the specific sender is whitelisted 
to be able to send anything else).  

AOL, of course, is one of the big offenders.  And their users are almost by 
definition clueless enough that one despairs of educating them to be more aware 
and responsible.  Perhaps pressure can be put on AOL itself to change its 
defaults;  if enough people refused non-whitelisted HTML-burdened E-mail then 
maybe AOL would get the message.

>> I don't want to see any solutions that result in some "authority" deciding 
what one can and cannot send. 

> Right, only community consensus should be applied.   It is doable.

I hope so.

> The whole discussion needs to be recast from "make the victims pay by
coming up with solutions (filtering etc)" to "make the offenders pay".

The problem is that then you have to define "who are the offenders" and 
ultimately the best judge of that is the individual recipients.  Ultimately the 
best weapon that recipients have is to en masse simply refuse to accept or read 
mail which doesn't meet their conditions for acceptance.  As the savvy types 
that we in this group supposedly are, we should be able to help provide guidance 
for creating tools (and/or create them) which will prove workable, and which 
gives recipients a feeling of control again.  The worst thing is for them to 
continue the feeling of victimized helplessness and frustration that so many of 
them feel today.

Just about unique among the approaches I've noticed being discussed here, my 
solution:

  1)  puts the control in the hands of the recipient

  2)  almost totally closes the window for recipients to receive attachments 
containing viruses and worms

  3)  sets responsible default behavior so clueless types are likely to be much 
safer than they are today

  4)  is single-ended and could be implemented TODAY, it doesn't require any 
worldwide consensus

  5)  doesn't require any major changes in sender infrastructure

  6)  doesn't increase E-mail overhead costs associated with individual messages

  7)  fully supports mailing lists, discussion groups, and other important 
E-mail mechanisms

  8)  provides major incentives to return to plain ASCII text E-mail, especially 
for unsolicited messages, which would cut the bulk of such mail 
by something like 3/4!

> The only way to do that is to refuse mail from spam-enablers.  It fixes
the problem immediately.  Again: <http://www.camblab.com/misc/univ_std.txt).

So your dear sweet Aunt Margaret's system gets infected and becomes a spambot 
zombie.  Now the questions are:

  1)  Does she REALLY deserve an "Internet death certificate"?

  2)  How long should that punishment, once imposed, last?  

  3)  Do you really think that (if everything else stays as it is at present) 
she being forced to change ISPs and set up a new user account will actually make 
her LESS likely to be victimized again?  
  
  4)  Don't you think it REALLY makes more sense (cost/hassle/overhead/etc) to 
simply detect and trash the bogus spam (/virus/worm) E-mails?  

> Spam would stop worldwide within DAYS if most of us, instead of whining
and coming up with impractical technical or legal 'solutions', would 
agree to do on the Internet what society does in every other field 
of human activity: ensure that actions have consequences.  It is THAT
SIMPLE.  (Any parent knows.)

The problem is that there are way too many users who are clueless, and who are 
simply never going to learn to behave more defensively online.  Coming down too 
hard will drive them offline (they'll just give up) and it's not clear that 
that's ultimately either the way to educate them (hopefully, some of them at 
least) nor that it's better from the standpoint of the future of our society to 
have them stay offline.

>> Rather than making mail filters poll a central site (these have in the past 
been 
> >victimized by DDOS attacks), maybe one approach here might be to set up a 
> >Yahoogroup that could be used to rapidly distribute these "disreputable 
domain 
> >names" and IP addresses along with a utility which would add them to the 
> >recipients' HOSTS files (and from there, to their incoming mail filters).

> This approach has been proposed previously on net.admin.net-abuse.email.
You will need a way to verify that the data is correct. 

That's true, and clearly one doesn't want to have malicious types adding 
legitimate domains like google.com or yahoo.com to the "bogus" blacklist.

More serious is the problem of people with a tendency to add legitimate domains 
which have been victimized as the counterfeit "From" address on spams.  :-(

> Suggestions were PGP/GPG signing the message.

The problems with all such approaches I've seen is that a spambot zombie can 
send mail using all the authentication/certificates/etc that had been 
established by the legitimate owner of the machine.  I think it's ultimately far 
better to be able to identify spam mail by what is in THAT mail message, and 
limiting the amount of "damage" and "deceptions" that such mail can employ... 
rather than trying to reach some kind of "blanket" thing saying that certain 
people "of course" are (globally!) good.  That just makes those "trusted" people 
a more attractive target for turning their machine into a zombie.

> Quite a few DNSBLs already allow access to the raw zonefiles via rsync.
It should be fairly easy to convert that data into a format usable by
the local DNS server.

I'd like to see a solution which does not require additional network overhead 
for every E-mail sent/received.  

Although I'm not convinced of its great value as a longer-term thing, since some 
spammers are finding their way around it, one strategy that's still fairly 
effective today is to block E-mails as spam if they contain links or URLs that 
point at spam-promoted Web sites.  I don't know how long that will last as a 
useful discriminator.

><snip>
> [reformatted line wrap]

>>> You have to use Bazooka and Adaware besides.

> > One of the things that has been SINGULARLY unhelpful toward addressing the 
> > problem of overloaded mail servers is this plague of HTML-burdened 
> > "alternative" copies of E-mail messages.  It is rare indeed that these 
> > provide genuinely valuable additional content;  instead they usually are
> > loaded with gratuitous graphic gizmos, Web bugs, possibly malicious 
> > scripting, misrepresented clickable links, and text-as-image designed 
> > to evade content filters.  While I'll accept that some folks can argue
> > that their needs for HTML-burdened E-mail is legitimate, certainly a
> > lot of it is not.  Mail with HTML-burdened attachments 
> > is typically 3x-5x larger than it would be as plain ASCII text.
> >
> > If HTML-burdened attachments were removed from non-whitelisted senders' 
> > E-mail (and this would catch at least most of today's spam) then such
> > mail would be 70-85% smaller in volume than it is today.  

> Isn't modifying mail in transit bad?

If it were done without the consent of the recipient, yes.  

In my case, my policy is that I simply don't want the wasteful bulk of 
HTML-burdened alternative text attachments in incoming E-mails.  Lots of folks 
send those, and many don't seem willing to stop doing so.  It's nearly 
impossible (and certainly unrealistic, at best) to expect AOL users (as a prime 
example) to go to the substantial hassle of turning off HTML individually on 
each E-mail message they send.

It's simply more realistic for me to t-can all that unwanted junk as soon as my 
systems get their hands on the incoming E-mail messages.

If the senders don't want my systems to futz with the mail they're sending me, 
then it's real simple.... send it to me in a reasonable format to start with!

Now, I will agree that my mail filters go a LOT further in processing incoming 
E-mails than most others I've seen... for example, I actually strip out things 
like "free E-mail ads" and too-familiar gratuitous junk taglines too.  
Admittedly this changes the content of the E-mail message, and this would 
confuse things like encryption-based content integrity schemes and such.  That's 
a price that I'm personally willing to pay...

>>> This is all documented.  It needs to be a new standard.  Feel free to
> point offenders to <http://www.camblab.com/nugget/htmlmail.pdf>
> (or send it to them)
> 
> 
> >
> > I don't want to see any solutions that result in some "authority" deciding
> > what one can and cannot send. 
> 
> Right, only community consensus should be applied.   It is doable.
> 
> The whole discussion needs to be recast from "make the victims pay by
> coming up with solutions (filtering etc)" to "make the offenders pay".
> The only way to do that is to refuse mail from spam-enablers.  It fixes
> the problem immediately.  Again: <http://www.camblab.com/misc/univ_std.txt).

> I do agree with this point. However, the business community does not
want to agree to that standard. They would rather pay for a lot of spam
rather than possibly lose a single email. And until businesses
understand that blocking the spam enablers makes sense, this is not
going to happen.

I don't think you can truly control mail getting INTO the Internet... there's 
simply far too many ways to put mail into the system.  What you CAN achieve, I 
believe, is to simply make sending unwanted mail futile, by ensuring that little 
or none of it reaches its intended destination.  

Yes, it's true that there will probably always be "Internet vandals" who get 
their rocks off by loading the system just to show that they can do that, but I 
think that's really a separate issue and probably requires a different solution 
that we can talk about another day.

>> Spam would stop worldwide within DAYS if most of us, instead of whining
> and coming up with impractical technical or legal 'solutions', would 
> agree to do on the Internet what society does in every other field 
> of human activity: ensure that actions have consequences.  It is THAT
> SIMPLE.  (Any parent knows.)

> Consequences to whom? Sender identification methods attempt to identify
the sender so that appropriate measures can be taken against the sender.
Blocking based on DNSBLs stops supposedly spam supporting senders.

Again, I think that ultimately it achieves little to punish your fellow victims.

The great majority of the systems today sending spam are NOT doing so as the 
result of a conscious approval and awareness on the part of those responsible 
for the system(s) involved.

I agree totally that it would be a better state of affairs if everyone attached 
to the Internet were competent and savvy, but that's simply not realistic, nor 
will it be anytime soon.

> For most users, the need is to stop the spam without blocking legitimate
mail from the same host. This is what drives the concept of filtering
over using a DNSBL.

...and filtering is confounded by things like HTML, which supports things like 
text-as-image, obscured URLs, scripts that perform encryption/decryption of 
message text, possibly malicious ActiveX, and things of that genre.

This is a problem that will never be solved, IMHO, until we get a handle on HTML 
and attachments.  And, surprisingly I think, once we adopt an enlightened and 
resonsible attitude regarding those two features of E-mail messages, (and 
combined with a good content filter on what's left), a surprisingly large amount 
of the problem simply vanishes!

[snip]

>> The best way ultimately to put a serious dent in spam (and worms/viruses)
is for it to be perceived as largely futile.  And if 99.99% of the junk (or 
more) never even gets delivered to a human being, this has got to help achieve 
that.

> I suspect that many spammers are the people who buy what appears to be
spam-in-a-box software packages offered in late-night infomercials.

Those are the relatively clueless spammers, I think.  I'm far more concerned 
about the really devious and belligerant ones.  Some of those (like Spamford 
Wallace) have been driven out of the business (hopefully) by legal measures, but 
it seems there are plenty more that have risen up to take his place.  :-(

> If the legal system can't go after the people who are selling the
spam-in-a-box products, perhaps under the category of Truth in Advertising
the TV stations who air the infomercials can be required to add warning
announcements to the effect that, "The following product is used for sending
unsolicited commercial email, commonly known as spam."

Sure, although I believe those who send spam are generally quite cognizant of 
what they're doing.  And they just don't care.

Gordon Peterson                  http://personal.terabites.com/
1977-2002  Twenty-fifth anniversary year of Local Area Networking!
Support free and fair US elections!  http://stickers.defend-democracy.org
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 19 Apr 2004 15:40:19 -0400

> Perhaps, but they're still victims too.

True.  But victim or not, I _cannot_ afford to let a provably insecure 
source continue to spew malware or spam at our users.

If there are real users on that machine, our inline rejection notices 
will tell them how to get their problem fixed.

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 19 Apr 2004 19:16:11 -0400

> forever swatting flies.  Let's close the window!!

I am not sure if it can be closed. The latest viruses are being sent 
around in ZIP files encrypted with a password which people still manage 
to open and encryption bypasses virus scanning on gateway level. Unless 
we throw out the baby with the bathwater - forbid all attachments, virus 
writers will always find a way around whatever semi-blocking technique 
we come up with. Same for HTML - unless all HTML is forbidden, I do not 
see how any benefit can come from it. But forbidding HTML will close 
many existing use cases which many companies and people do every day.

The biggest problem with this approach is that while it might work in 
theory, in practice I do not think we can convince any sizable majority 
of Internet users to accept it. This is the main problem with any 
proposal - aside from the idea itself, it must also be marketed and be 
convincing enough to be acceptable.

However, if you want to take a shot at it and write up a BCP document as 
a short Internet draft, go ahead. There is definatly a good use of some 
attachment blocking such as .EXE files.

>>Right, only community consensus should be applied.   It is doable.

We must account for a possibility of a community system being subverted. 
Any system that does not account for that will be no better than an 
authority. There was a couple research papers floating around that try 
to address the same problem in P2P networks - the problem is solvable if 
it is taken into account.

Yakov