How Bad Does Your Credit Card Snitch on You?
What I saw for sale in 2004, what's on the shelf today, and why Maryland just changed the rules.
In 2004 I got a call offering me a job as a database administrator for America Coming Together — the big 527, the precursor to the super PAC, the left's massive voter-data project. They'd raised hundreds of millions of dollars. They had left-leaning companies and nonprofits and charities lined up willing to loan them data. Their problem was they couldn't get it together in any meaningful way. Too much data. They couldn't handle it.
They called me twice. The second time they offered me more money. I said no twice. I didn't want to work twelve-hour days, seven days a week, and that's what they were asking for.
Then one Sunday — I was at Deutsche Bank on a project going downhill because nobody would listen to me, one of those projects where you can see the disaster coming — a recruiter called and asked me to join the team helping John Kerry. And they named a sum of money I couldn't say no to. That was Sunday. Monday morning I went to work for them.
I scoped the project, found the difficulties, and came up with a plan. It involved spending about three million dollars on new hardware and a data center — which in 2004 meant a room with electricity and internet and everything caged in, kind of like a dog kennel. That was state of the art. If you'd said “hyperscale” back then, people would have thought you were talking about descaling agents.
We got up and running. And here's the part I want to tell you about, because I wasn't the analyst — I ran the database team. I couldn't hire or fire, but I assigned the tasks and kept everybody working. The schedule worked out to come in by nine, leave after nine, book twelve hours, seven days a week. No days off. No sick days. You worked. That was the job. And it worked — we were extremely successful.
But the data. That's what stayed with me.
One of the data points the analysts were using was women's dress sizes. Not as trivia — as a segmentation variable. Three-size bands. And you have to understand: the things that move a size-two vegan woman with no pets, no alcohol, no cigarettes are very different from the things that move a size-18 woman who drinks and smokes and eats meat. Body type was a proxy for lifestyle, and lifestyle was a proxy for votes. As a database guy I admired the elegance of it. As a human being I found it a little chilling.
Then there were the credit card data points, and these genuinely shocked me. I'm not sure whether it was the Kerry project or a similar project I did for a company that worked with Marriott, but I had to review everything we could buy, and two boolean fields stopped me cold: has bought adult novelty toys through the mail — yes or no. Has purchased adult movies in a hotel — yes or no. Your credit card company was selling that. Visa was a major player in this game. So was one of the Donnelleys — R.R. or R.H., I always got them confused, the directory people turned data merchants.
I used to torment myself with the false-positive problem. Say Joe Blow's cousin comes to visit. Joe's never watched that kind of thing in his life and would be shocked by it. But Joe puts the cousin up at the hotel on his credit card for the week, and the cousin watches the movies. There's no corresponding field that says “his credit card was used to buy it but he didn't do it himself, it was the cousin.” Joe Blow is now, in some database somewhere, a man who watches adult movies in hotels. And there's no appeal.
A terabyte was a massive database in 2004. Within a few years I was building multi-terabyte systems — half a dozen databases in the 8-terabyte range for one company, and we weren't considered large by industry standards.
[PAYWALL — BREAK HERE]
