Entries by Matthias Leisi

Ein faszinierendes Experiment: Wer es schafft, IPv6 aufzusetzen, kann gemäss ipv6experiment.com gratis auf Pornographie zugreifen, für die ansonsten 30 USD/Monat bezahlt werden müssten.

Die Initianten des Projekts (Your.org) möchten mit dem Experiment zweierlei erreichen: Erstens soll damit das Bewusstsein für und die Verbreitung von IPv6 verbessert werden. Zweitens hatten frühere Versuche der Initianten mit IPv6 gezeigt, dass die meisten bestehenden IPv6-Installationen, -Tools und -Programme noch nicht “ready for prime time” sind (gut erklärt in diesem Beitrag auf der NANOG Mailingliste von einem der Initiatoren des IPv6 Experiments).

Das revidierte Fernmeldegesetz (FMG) tritt heute, 1. April 2007, in Kraft. Mit dieser Revision hält auch ein neuer Artikel in das Gesetz gegen den unlauteren Wettbewerb (UWG) Einzug (Art. 3, Bst. o):

“Unlauter handelt insbesondere, wer:
o. Massenwerbung ohne direkten Zusammenhang mit einem angeforderten Inhalt fernmeldetechnisch sendet oder solche Sendungen veranlasst und es dabei unterlässt, vorher die Einwilligung der Kunden einzuholen, den korrekten Absender anzugeben oder auf eine problemlose und kostenlose Ablehnungsmöglichkeit hinzuweisen;
wer beim Verkauf von Waren, Werken oder Leistungen Kontaktinformationen von Kunden erhält und dabei auf die Ablehnungsmöglichkeit hinweist, handelt nicht unlauter, wenn er diesen Kunden ohne deren Einwilligung Massenwerbung für eigene ähnliche Waren, Werke oder Leistungen sendet.”

Der Absatz ist etwas länglich und zumindest einige Vertreter der Spam-Branche wollen oder können ihn nicht verstehen (mehr dazu weiter unten). Hier eine schrittweise Erklärung von UWG Art. 3 Bst. o:

Ever wondered where your spam comes from? Some partial answer may be found within. We will also explain why over 98% of the Internet are a good place to be.

I let the SpamAssassin plugin which I wrote in response to SpamAssassin Bug 4770 run for a couple of days on my spamtrap. Then I wrote a small script to group the received spams by ASN (Autonomous System Number) — see the full report data.

...is not very different from what the rest of the world does.

While researching a couple of requests and additions to the dnswl.org whitelisting data I stumbled upon the “Most popular sites visited by Saudi users” (http://www.isu.net.sa/surveys-&-statistics/pupuler.htm), thanks to the Great Firewall of Saudi Arabia.

Now I do have profound issues against such country-wide web content filtering (and I highly doubt the justification that this is in order to protect local traditional value but suspect more blatant motives), but it is interesting to note that the Saudi users are interested in the same things as the rest of the world: Windows and virus-pattern updates, Google, Yahoo and MSN, stock quotes and (local) news, websites of TV stations (Al Jazeera and Al Arabiya), and travel websites. Additionally, there’s the occasional edification website.

The proxy statistics of most companies around the world will be highly similar (replacing the obvious local-language counterparts). That’s a pretty good thing, actually.

I was reading up some interesting slides (PDF, 446KByte) by Joe Sauver humbly titled “Route Injection and Spam”. Despite it’s title, the slides are great at connecting all the dots one might have in his head regarding routing, ASNs, routing views and registries and all that stuff.

While I was reading the paper I wondered whether the ASN where a particular message (purportedly) came from would make a good token for spam filtering, either in a Bayes context or in a white-, grey- and blacklisting context.

So in order to dig a little bit deeper, I wrote an experimental SpamAssassin plugin which will add a header. It looks like X-Spam-ASN: 24940 213.239.192.0/18 where “24940” is the ASN the mail came from and “213.239.192.0/18” is the announced network.

You can download the ASN plugin at http://matthias.leisi.net/uploads/ASN-Plugin.tar — feedback is welcome.

Btw., you can also take a look at Init 7’s ASN extension for Webalizer.

Update Dec. 15, 2006, 20:45 CET: A slightly revised version of the plugin (including now being part of the Mail::SpamAssassin::Plugin package) can be found at the Apache/SpamAssassin issue tracker, bug 4770

Update Dec. 22, 2006, 20:50 CET: The version on bug 4770 has progressed substantially over the one available here; I have thus removed the local version.

When writing the tools and website for the DNS Whitelisting project, http://www.dnswl.org/, I had to deal with a lot of IP calculation: data is stored in CIDR format, eg “127.0.0.1/32” to indicate a single IP address, “213.144.132.248/29” for a range of 213.144.132.248 to 213.144.132.254 and so on. RFC 1878 explains this notation in some detail.

Storing the dotted quad notation of an IP address would be pretty inefficient – the string is 15 characters long (four times up to three digits plus three dots), while it is technically just a “nice” display of a 32bit number. Additionally, comparing IP addresses (eg “is this address within a given range”) is a lot easier when done on the numeric representation since we can just use simple numeric comparison. In any case we need a way to convert between these two representations.

The CIDR notation itself is somewhat difficult to deal with – most humans will not be easily able to always know that the range “213.144.132.248/29” has “213.144.132.254” as it’s last IP address. We therefore also need a way to transform between CIDR and range notation. Unfortunately we need to do this in multiple environments, eg directly within the database (for certain batch jobs), in PHP (for regular backend operations) and in JavaScript (to avoid a server roundtrip on each transformation for the human reader).

Secureworks has published the analysis of a (the?) central control server of the SpamThru trojan. Besides the fact that I would dearly love to learn more on how Spamhaus managed to identify the control server, the analysis has some proof about things we previously only could speculate about:

• Spammers managed to increase the quality of their lists. While I have observed this by eg the decreasing number of invalid recipients, Secureworks’ analysis proofs that some address lists have been stolen from unsecure webservers (by simply copying complete database dumps).
• The botnet had (has?) a size of about 73’000 computers. Even when basing on cautious assumptions, such a botnet can send out messages in the order of magnitude of 10^9 per day (“billions” for americans, “milliards” for the rest of the world).
• Most infected machines suffer from multiple diseases—from poor patching and updating cycles (only half are on a more-or-less up to date Windows release) to other malware being present.

Are you sure that your Windows machine is not part of the botnet?

The dnswl.org project now has a somewhat functioning infrastructure—some backend tools and database to create various output formats (eg for the DNS servers, for rsync, for Postfix etc) and is ready to actually manage the data.

The goal of dnswl.org is to initiate a collaborative effort of whitelisting. Instead of each and every admin and site managing and maintaining their own whitelist, dnswl.org allows to join forces by combining data from different sources (plus manually added data) and redistributing it through several channels.

In order to widen the use and reach of dnswl.org, I’m asking for your help:

• Currently, the main DNS server for dnswl.org answers about 2’000 queries per minute; since the traffic is growing, I’m looking for someone willing to host a DNS mirror.
• If you have whitelisting data from a trustworthy source, we can arrange a one-time or regular import of this data. You can contact me by mail (admins@dnswl.org) or use the feedback/request form.
• If use the data and you see a) errors or b) “false negatives” (ie spam coming from a whitelisted IP address) please let me know.
• If you would like to join the administration of dnswl.org, I should know you already or you should know somebody I know (you may want to check my openBC/XING profile and contacts).

Thanks!

Most spam filtering is usually accompanied by some degree of whitelisting in order to avoid too many false positives. Given the recent increase in spam levels, filtering must become ever more aggressive, and the effort both to maintain the rules and the whitelists to mitigate the increased risk of false positives is increasing as well.

For many sites, this will mean a slow but steady shift away from the “Default Allow” paradigm of Internet e-mail towards a “Default Deny”. As long as you are not whitelisted on the receiving end, the chances of your mail being lost will steadily increase. And worst of all: most often you will not notice, since the mail will end up in some central or user-specific Junk folder, never to be seen again. Given the widespread heavy filtering, we must realize that the “Default Allow” policy of Internet e-mail does no longer exist.

And now it is time to get honest. Internet e-mail should switch to a “Default Deny” policy. Everything not whitelisted is rejected with a 5xx error and a hint on how to get yourself whitelisted (eg: make a phone call).

Spam filtering is an example of Enumerating Badness—we try to identify the millions of bad senders in 2^31 IP addresses, instead of finding the couple of hundred or thousand good senders and giving the yet unknown good senders an efficient method of making themselves known. Even 100’000 good senders are roughly 0.005% of the complete (theoretical) IP address range.

Compare that to the Spamhaus XBL which contains some 4.5 million records, ie somewhere around 0.2% of the address range. Just a single (albeit large) DNS blocklist contains 45 times more entries than a comparable whitelist of good senders.

Of course there will be edge cases, especially the mailservers of large providers. But then some light spam filtering on these couple of hundred servers would still be possible.

And of course, such a transition is not without pain. But then even the not yet very comprehensive dnswl.org data already whitelists more than 30% of all incoming messages at a particular e-mail system with 10’000 users. Given enough priority, a significant improvement seems plausible.

Complex cryptographic and semantic work à la DKIM is not necessary. A healthy competition amongst reputable providers of whitelists should be enough to make the paradigm shift from “Default Allow” to “Default Deny”.

Vor längerer Zeit hatte ich mir die Domain “dnswl.org” für den Aufbau einer DNS-basierten Whitelist reserviert und einen kurzen Text auf die Webseite gestellt. Es gab noch keine Daten und kein gar nix. Erstaunlicherweise haben aber einige Mailadmins die Webseite über Google gefunden und nutzen die nicht vorhandenen(!) Daten bereits seit einigen Wochen.

Das hat mich dazu motiviert, das Projekt weiter zu treiben. Es besteht nun eine (immer noch kleine) Webseite: www.dnswl.org, es gibt einen(!) DNS-Server und es gibt—tada!—jetzt auch Daten.

Ich freue mich auf Feedback und Kommentare; bei entsprechendem Interesse werde ich das Projekt weiter verfolgen (DNS-Infrastruktur, Webinterface, Request-Handling usw). Freiwillige sind herzlich willkommen!