I officially started work at the Internet Archive over a year ago.

Let’s remove any tension – it has been a fantastic year, where I have gotten more done in the way of preservation and computer history work than my entire previous 40 years combined. Internet Archive seems to like me, I really like them, and I’m staying.

Here I am with the boss:

Brewster and I are two rather different people, and although the Venn diagram of our interests does not intersect everywhere or even close to it, what we do share in terms of goals and passions is very similar. There’s no hidden agenda with this guy – the headquarters in SF isn’t secretly a meth lab, we’re not actually some lobbying group or anti-whatever think tank trying to destroy or anything. There is only the Mission, the goal to bring as much human knowledge as universally as possible, and to preserve and keep all matter of knowledge as reliably as possible.

Oh, there’s occasional office flare-ups and disagreements and I’m sure some clenched fists and not every day is an endless buffet of awesome, but every single person in this organization understands the Mission and pretty much 100% of the disagreement is how best to achieve that mission with what resources there are (or to gain new resources). That’s rather refreshing from, oh, let’s say, every other goddamn place I’ve worked at, where the goals of some people are “get to retirement age” combined with others who mostly have signed up for “do absolutely nothing until you either get bored and leave or get fired”. That’s not going on here. It was a shocking office culture to run into, everyone just kind of pressing in towards the overarching mission without being waylaid by one group trying to undermine the others for some other bonzo reason unrelated to what the place was going for. Again: People leave, people join this place, but they all understand that dream, that plan, that hope, that dream. Maybe this happens elsewhere, but not in my previous lines of work. So that’s somewhat mind-blowing on a daily basis.

I will feel really stupid if I start listing out names of co-workers and then miss some, so I will tell you that I have someone who is a “handler” for me, and she is far and away one of the best bosses I’ve ever had, and I’ve had a few really, really great bosses in my time. I have people I sit with when I’m in San Francisco who are brilliant, hardworking people (again, all aimed at this goal) who do stunning work. We have communication channels where various groups talk, and it’s like shoving your face into a Brilliance Fountain 24/7. I’m not making this stuff up to butter anyone.

Remember, this isn’t people all sitting around figuring out how to monetize farting or who are blowing up paradigms with slide-scale infradoobles using Ruby on Crack combined with Hibbledoo Middleware. This is a non-profit online library providing petabytes (petabytes!) of data to millions in the most efficient way possible. Speaking of which…

One of the job descriptions/goals for me was “bring in data”.

I just checked the internal tracker to see how I’ve been doing on the upload front. Very well, apparently – I have uploaded 120 terabytes of data. That’s into 82,438 individual items, which could be anything from texts or songs up through to .tar files of web captures. When I started, I said my goal was to upload a terabyte of data a month. As I am apparently doing ten times that amount, I’ll consider that goal met.

I’ve brought in so much “stuff”, in fact, that it would nearly impossible for me to tell you all of it. Let’s throw out some highlights.

I was asked to look into bringing in software. So, I started out with CD-ROM shareware discs, not dissimilar to what I have with cd.textfiles.com. Well, that has been a wild success. I am ready to declare The Internet Archive as the largest collection of shareware on the Internet. Seriously. First, there’s over 1,100 CD-ROMs and DVD-ROMs contained in the CD-ROM collection. But oh, it gets better. You see, functionality was added this year to allow you to browse inside the ISO images. Feast your eyes inside this CD-ROM, for example. You just add a slash at the end of the ISO image reference and there you are. But let’s go even further than that: Let’s take a GIF file of Winter from 1991: http://archive.org/download/SoMuchSharewareV1_918/SoMuchSharewareV1_1991.iso/GIFS/WINTER2H.GIF

You see how you can reference a file inside a CD-ROM image in a permanent URL that can be pulled from anywhere? That’s why, as far as I’m concerned, The Internet Archive now has well over four million shareware programs, artworks and documents online. At least. That’s a game changer. And this year? We’re going to double it.

Computer magazines. Lots and lots and lots of computer magazines. Out of print, fondly remembered and otherwise obscure magazines on a range of technical subjects, currently the province of attics and basements and long-unopened warehouses and a smattering of living spaces – now up and readable.

This collection of computer magazines as well as a smaller spanish-language set constitute  30 years of technical publication, and well over a thousand individual issues, many of those in the hundreds of pages rage, which means there’s a lot of history squished into all this data. I’ve already been informed of university and high school classes out there using these issues to bring up discussions of history or to point out aspects of computer technology that have shifted or changed. Some of the issues have indexes already (the Compute! Magazine collection is a shining example) and I hope more will get them over time. I’ve got lots more issues to add, too.

Manuals! Damn, do I love getting manuals up where people don’t have to search like crazy to find them. It actually saves the environment to some small amount, since people will happily buy older equipment knowing they can get the manual easily and make the use of the item. So manuals are a big deal:

Arcade manuals. DEC manuals. Synthesizer manuals. Commodore manuals. Whenever I track down a cache of these or get sent them, they go up. I want to be able to have someone grab any piece of equipment new or old and understand what exactly everything does on it, and maybe even the why.

Audio! Video! 59,000 open-licensed albums. 2,100 nights of live and club music. Hours of GET LAMP raw interviews. A complete port of 10 years of Jesse Thorn’s The Sound of Young America. Bit by Bit. There are many other such audio and video projects where I use scripts to get them into the archive as collections – part of my work has been writing stuff to inject massive amounts of data into archive.org’s servers to make it that the uploading is the least of the issues. Which brings us to:

FUCK YEAH, ARCHIVE TEAM. I can’t begin to really describe how much data Archive Team has brought in – so many people working together to take snapshots of important things that are being shut down with poor or no notice, as well as proactive “panic downloads” where we recognize things are on the outs and we grab as best a copy as we can.

Like the Internet Archive itself, Archive Team’s collections are not always meant to be short-term beneficial and in fact are pretty clunky – 50gb .tar files and the like. What they are meant to be is raw material for later efforts and rescue of lost data – the panic downloads are basically someone stepping in at the present time and running the duper just before a whole range of data disappears forever. Some of it will be absorbed into the Wayback machine. Some will be filleted for their GIFs or mp3s or who knows what else. And still others will result in data, meaningful first-generation data about how people used computers or how solutions were found to old problems. Or maybe we’ll just laugh at the hair.

I’d go off more on Archive Team but I’m scheduled for something like a half-dozen speaking engagements around the world this year related to it, so I’ll probably just link to those talks when they come out. Actually, here’s a talk I gave about it a couple months ago, which is hosted at, and took place, at the Internet archive.

As we speak, Archive Team is uploading something like 25 gigabytes an hour into the Internet Archive. Chew on that for a bit. So many good people, so much good work, on both sides of the wire.

This is getting a bit long, and I’ll split more out into entries this year to give context and meaning, but the upshot is that this has been a very successful year, a lot of amazing things are happening and continue to happen, and every single waking moment I spend related to this “job” is what I’ve always wanted to do.

And that’s pretty nice. Thanks for taking the gamble, Brewster!

Over two and a half years ago, I wrote a statement on Sockington selling out, where I basically said the following:

“I am not going to sell Socks out.  Period.  Drag your “proposal” or ‘touching base” or “big idea” or “possibility” to your trash icon, or I’ll kindly take the time to do it for you.  The store is closed. It was never open.”

I also felt I had to clarify what “selling out” meant, and what I came up with was:

For me, “Selling Out” for something like Socks comes when the cat or myself are doing things we would never do on our own, and people give us money to convince us to do this. Oh, they may couch it as “paying for your time and effort” or “to help with your maintenance costs”, but it’s taking cash to do something otherwise never happening.

So in that spirit and knowing that, I will inform you I was contacted a month or two ago by a company that wanted me to sell Socks out. Do a promoted tweet, as they call it. As usual, I toy with these guys, so I needled them along and asked how much. $5000/tweet, they said.

Hmm.

Sockington (the actual grey cat) is now (we think) about seven. Penny (the actual orange cat) is now easily ten. Penny and Socks have both seen years of very nice health – no major issues, no overnight visits, you name it. They’re kept on a good diet and kept indoors and exercised and they’re in a rather beautiful home outside of Boston, a place of stairs to climb and rooms to run and general Cat Heaven.

But cats don’t live forever and let me tell you, being involved in the whole Sockington Twitter thing has told me an awful lot about how quickly things go with them. One day they’re purring in your lap and wondering when the next mealtime is, and the next day they’re very, very sadly meowing and you go to the Vet and the Vet suddenly drops in your lap some sort of terrible decision. And that decision is often one that, bluntly, translates to “For three thousand dollars, your cat will live for at least months and probably years, or we can kill it.”

So, call it the years piling on, but I had this rough idea in my head that it might be kind of sweet of Socks and Penny (and Tweetie, the third cat, who needed actual $1500 surgery a year or two ago) to have a couple tweets that ensured that the only decision in a future medical malady was how fast they could get back on their feet. Call it a wavering, a moment of weakness, maybe for some of you a “cold dose of reality”. But what the heck, it wouldn’t hurt to talk to these people.

It took a while, weeks really, but they eventually came back with actual things to sign. And, of course, because everyone in that sort of business is a bait-and-switch scumbag, it suddenly went from “$5000/tweet” to “$2500/two tweets”. And if that sounds awesome, it would be scripted by someone else, specifically to sell a product, a product that had nothing to do with cats.

Subsequently, I could see how this was going to go. With an Non-disclosure agreement attached, and more requirements than I could shake a stick at, it was going to be months of e-mails to get the actual money, money which, again, falls down into the “I sort of have access to that kind of money” level and which, like any such thing done, was a loss of principle and meaning in the name of some hocked-together justification. It’ll heal my cats!!!!!!!!

So while there was this tiny, tiny percentage of me going “hmm”, most of me got sickened by the entire enterprise within a day or two, and as the actual documents piled in, that pretty much settled it. So no, Socks continues to not push house cleaning products on his twitter feed. He continues to be a very strange, very odd little cat.

And as for my little bait-and-switch scumbag marketing guys, who wanted to make a cat tweet about some household product for a few bucks, and were willing to jerk me around for eyeballs? It turns out I don’t do that.

But what I will do is drop all the documents and contracts related to the deal.

So here we go, here’s everything they sent me.

• A W9 (Tax ID Number) request form, which is pretty standard, but still, it’s for a cat tweet.
• A Vendor Diversity form, in case there was a bonus diversity to the Sockington enterprise. like being a historically black college or if Socks lived in a generally downtrodden part of the countryside, just waiting to be lifted out of poverty.
• Finally, here’s what you probably want to actually read: Sponsored Social Media Agreement 04-27-12 - a word-compatible .doc file of the terms of two Sockington tweets to promote the product in question.

Oh, it’s all in there – the kind of stuff you sign your life away with as “talent” in a promotion, the willingness to get involved in secrecy, in acting like oh, one day Socks decided to start tweeting about some sort of product, in a way unlike himself, and right off to those delicious million eyeballs.

Somewhere down there is this line:

Talent agrees that if Talent commits a material breach of any provision of this Agreement or at any time fails or refuses to fulfill Talent’s obligations hereunder, then Marketer or Agency may terminate this Agreement and Talent will not be entitled to any compensation. Talent further agrees that if Talent should die, or fail to fulfill Talent’s obligations hereunder due to illness, injury or accident so that, in Marketer’s or Agency’s judgment, Talent’s disability will preclude Talent from rendering the Services described above, then Marketer or Agency may terminate this Agreement and Talent will not be entitled to any compensation.

Since I didn’t sign, and I have told you about it, I guess that constitutes a “material pre-breach”, where I’ve already taken a big ol’ dump on the whole prospect of the cat being turned into a mouthpiece for said products. Oh well. I’ll get over it.

Enjoy the glimpse into how bad it can get.

And I’ll pet Sockington for you.