Friday, January 06, 2006

Finding subversives with Amazon wish lists

Applefritter, Tom Owad, has scooped up the wishlists of hundreds of thousands of Amazon customers and collected them into a searchable 5GB database. Now he can do his own searches for subversives who are be interested in books that might give cause for concern to the authorities.

"I downloaded all the files to an external 120 GB Firewire drive in UFS format. The raw data occupied little more than 5 GB. I initially wanted to move all the files into a single directory to facilitate searching, but as the directory contents exceeded 100,000 items, the speed became glacially slow, so I kept the data divided into chunks of 25,000 wishlists.

Next comes the fun part – what books are most dangerous? So many to choose from. Here's a sample of the list I made. Feel free to make up your own list if you decide to try some data mining. Send it to the FBI. I'm sure they'll appreciate your help in fighting terrorism.
On Liberty by Stuart Mill. First sentence: "The subject of this essay is not the so-called 'liberty of the will', so unfortunately opposed to the misnamed doctrine of philosophical necessity; but civil, or social liberty: the nature and limits of the power which can be legitimately exercised by society over the individual." What more do you need?
Slaughterhouse-Five by Kurt Vonnegut. The classic anti-war novel.
Fahrenheit 451 by Ray Bradbury. Dystopian.
Brave New World by Aldous Huxley. More dystopian.
1984 by George Orwell. Most dystopian.
Critical Thinking by Alec Fisher. Can't have any of that.
Build Your Own Laser, Phaser, Ion Ray Gun and Other Working Space Age Projects by Robert Iannini. Obviously.
Apple I Replica Creation by Tom Owad. Building your own computer should be illegal. (ok, it's also here because I wrote it.)
The Catholic Worker Movement: Intellectual And Spiritual Origins by Mark & Louise Zwick."

A reader at Boing Boing who picked up the story reckons he has a much easier way of collecting the same data.

No comments: