So here’s some more crap on data retention by ISPs. In this particular case, the attorney general of the U.S.A. is telling ISPs they should voluntarily record the last two years of data from all their customers. Ahuh. Let’s just see how the maths work out on that…
Let’s be super conservative first up. Let’s say this logging merely requires the logging of a time and both IPs of a connection. We’ll presume we can fit that into 16 bytes. If the average person visits, say, 25 pages per day, each will require around, say, 20 individual connections. That’s 500 log entries per user per day; 8kB. So, 2.92MB per user per year. Easy, right? Well, if we have 10 million users in Australia, that’s only 29.2 TB per year. Pfft, no worries. We’ll just get a whole stack of Xserve RAIDs, the big 7 TB ones. That’s only $138,420 worth.
Wow, that’s nothing.
But that’s the conservative case. Here’s a few problems with that – it doesn’t take into account the need to mark the start & end of connections, so you’ve just doubled your log data right there. It also doesn’t account for the need to map local IPs to user accounts, which is another 20 bytes or so per session. And oh dear, it doesn’t scale very well for UDP… in that case you need to note the time of every packet… better hope some careless user doesn’t accidentally use UDP transport for some streaming video. Yikes.
So, let’s consider my own usage. I probably perform about 2000 HTTP requests a day, at least. Add in my iDisk syncing, checking of email, etc, and that’s probably another couple of thousand connections. Add in my BitTorrent traffic, that’s another ten thousand connections a day, thereabouts. I don’t even know how to consider my UDP usage… that could be on the order of tens of thousands of packets per day, some days.
So let’s just be conservative again and say I’m averaging 15,000 logs per day. At 32 bytes per log entry, plus let’s say ten 20 byte map records, that’s only 480.2kB per day. A mere 175 MB per year. Bah, that’s nothing, surely. Now let’s see what happens if I turn out to be the average case – 1752.73 TB per year. That’s quite a few more Xserve RAIDs… but only $6,948,684 worth. Hah, that’s nothing. When you consider the techs needed to run those 251 Xserve RAIDs, the actual purchase cost pales in comparison.
And of course, throughout this I’ve completely ignored the need to actually use RAID on these Xserve RAIDs. Even if you were living dangerously with the bare minimum single mirror, that only doubles the cost. And don’t forget we need two years of records, too.
So, in the context of Australia, this meagre little mandate only requires ISPs to spend a mere few tens of millions of dollars, to satisfy some ignorant troglodyte’s desire for 1984. Increase numbers by a factor of ten for the U.S.A. Increase by a factor of ten again for a more global cost. Who doesn’t want to spend a billion dollars or so tracking down that damn Nigerian prince?
It’s really hard to convey just how stupid this sort of legislation is. Most of it is completely impractical, even before you consider the ethical, morale and legal issues. And on the topic of legal issues… just associating a particular connection to a particular account is not going to cut it for many cases. There’s the fact that your average household has at least two or three people who share the connection. There’s also the problem of guilt by association – what, so because the logs say I connected to port 80 on some host which was running BitTorrent means I of course must have been downloading that latest Britney Spears classic, right?
Don’t even get me started on encryption. Gah!
Stupid, stupid governments. Bad government, bad! No more Schmacko’s for you, no!
