Life, Video

How I manage my digital home videos

My family shoots a lot of video. We’re on our second Flip camera, we used a mini-DV camcorder until it would no longer record cleanly, my wife and I both have iPhones, and we have two young sons who we find very photogenic.  Add that all up, and you get what can seem like an unmanageable amount of digital video: 560GB divided between 1,500 video clips since 2004, 80% of that since our first was born in 2009.  At last count, those clips totaled 80+ hours, and that number is increasing every day.

A couple of years ago, I digitized all of my wife’s family’s home videos from the 80s and 90s, and it was heartbreaking to see how much of the videotape had degraded and was no longer watchable.  I didn’t want anything like that to happen to our family memories, whether it be due to hard drive crash, accidental deletion, or just disorganization, so over the last few months, I’ve formalized the system that I’ll use to ensure that all of our videos stay organized and backed up.


On the first of every month, I ensure that all videos from our Flip camera (which we use rarely, and usually only for long recordings) and our iPhones have been transferred to a computer. On that computer, there’s a daily cron job that looks for new video files in the Flip and iPhoto libraries and copies them to the “Home Videos” folder on our Drobo, which is network-accessible via our Time Capsule.

Using exiftool, all of these new files are automatically renamed using the format “2012-01-02 03 04 05.MOV”, where the filename is the date and time the video was taken:

exiftool "-FileName<CreateDate" -d "%Y-%m-%d %H %M %S.%%e" [files_to_rename]

I then add captions to the videos by adding a description of the video to the filename (after the timestamp). If multiple clips are from the same event, I can leave them without a description, and a cron job will automatically add the description from the previous clip, appending ” – 2.MOV”, ” – 3.MOV”, etc. (It only does this for clips taken within a 15 minute window of one another.) It takes me about 10 minutes to caption a month’s worth of video.

I chose to organize the video files directly, without using any video library management software, since I want to have as much control over the portability of the clips as possible.  I’ll want to be able to stream them to myriad devices over my home network or even the Internet, and I don’t want to have to deal with whatever folder and file structure was chosen by the management software.

If all I wanted to do was keep all of my clips organized by date in a folder, I’d be done right now, but I want to burn DVDs of this footage for easy physical portability, universal playability, and as a last-ditch backup solution.1

A cron job runs every night, checking if I have accumulated 2 hours of named footage in the folder on the Drobo that has not yet been burned to DVD. If I have two hours, but it’s not all named, I get an e-mail reminding me to name it. If I have two hours and it is all named, it will automatically create an iMovie project, populating it with the clips (using ffmpeg to convert all the non-.MOV videos into a format that iMovie won’t choke on). It also auto-creates chapters in the project by stripping the timestamps from the clips and adding a new chapter for each new event. For example, this set of clips:

  • 2001-01-01 00 00 00 – New Years Day.MOV
  • 2001-01-01 00 04 03 – New Years Day – 2.MOV
  • 2001-01-01 00 17 12 – New Years Day – 3.MOV
  • 2001-01-21 17 00 00 – Christina’s Birthday.MOV
  • 2001-01-24 14 00 00 – Hiking.MOV
  • 2001-01-24 14 11 00 – Hiking – 2.MOV
  • 2001-01-29 15 00 00 – Snowboarding.MOV

would result in this chapter list:

  1. New Years Day
  2. Christina’s Birthday
  3. Hiking
  4. Snowboarding

Once the project has been generated, the rest of the process is manual, since I want to hand-check the chapters and make sure that a multi-day event (like a family vacation) hasn’t been split across DVDs. I also like to customize the DVD with relevant background images and an appropriate theme. (I’ve even been known to pick out some complementary background music for the menus if I’m feeling particularly creative.)

Once the DVD project is ready, I export it as a disc image to a different folder on the Drobo and eventually burn it to a physical disc.


It’s a solid setup for my next project, which is to find a way to easily watch all of these videos on our Tivo, our iPhones, or at least via a Mac Mini connected to our TV, because having all of this video of family memories doesn’t matter if it’s too much of a hassle to actually watch it.

How does this compare to your own process for organizing and managing your personal media collection? Any tips or tricks that you’d like to share?

1. It’s not my only backup: in addition to the inherent drive failure protection provided by the Drobo, I have everything backed up to two external drives as well, one of which is kept at home, the other off-site. Every six months, I update the contents of the on-site drive, swap it with the off-site drive, and then update the contents of that one too.

Standard
Browser Add-ons, Feed Sidebar, For Sale, Mozilla, Mozilla Add-ons, Mozilla Firefox, RSS Ticker

Firefox Add-ons for Sale

The time has come for two add-ons, RSS Ticker and Feed Sidebar, to find new owners.

Since I started developing add-ons for Firefox, I’ve written at least forty different extensions: some for personal use, some as a freelancer, and some as the primary function of my full-time employment.

In order to free up more time to pursue new ideas and projects, I occasionally need to either retire or transfer ownership of add-ons that I wrote of my own volition. Some of these, like TwitterBar and FireFound, were retired by executive decision. Today marks the first time that I am attempting to actively sell the rights to some of my add-ons.

Allow me to brag about these add-ons

RSS Ticker and Feed Sidebar are my two most popular add-ons, and they’re fully functional in the most recent version of Firefox.

RSS Ticker Screenshot

Both of them have been featured as “featured” or “recommended” add-ons Mozilla Add-ons multiple times, and both maintain healthy usage by a dedicated base of users not interested in Web-based feed readers.

RSS Ticker averages 39,000 daily users and has been download 1.3 million times. Feed Sidebar averages 21,000 daily users and has been download over 900,000 times. The two add-ons are unlikely to have overlapping users.

RSS Ticker is the first Google result for firefox ticker. It is also the first result for rss ticker and feed ticker.

Feed Sidebar screenshot

Feed Sidebar is the fourth result for firefox feeds and firefox feed reader and number two for firefox feed. It’s number one for feed sidebar.

Both of these add-ons would be excellent footholds for a Web-based feed reader to attract users who prefer to consume news directly from their browsers; they would be equally beneficial for content-oriented sites to push recommended feeds to users via the already-integrated “Featured Feeds” feature in both add-ons. (Both extensions regularly check a list of “Featured Feeds” and suggest these feeds as subscriptions to their users.)

What do you get if you buy?

If you buy either of these add-ons, I’ll:

  1. Transfer ownership of the add-on(s) to you on Mozilla Add-ons.
  2. Redirect the add-on pages on my blog to the pages of your choosing.
  3. Forward any feedback e-mail I receive regarding the add-ons indefinitely.
  4. Add you to my Christmas card list.
  5. Be a good guy in general and offer free consulting and advice related to the add-on(s) for as long as I can.

If you’re interested in purchasing either or both of these add-ons and taking over their development, e-mail me at chris@efinke.com.

Standard
AutoAuth, Browser Add-ons, Feed Sidebar, Interpr.it, Links Like This, Mozilla, Mozilla Firefox, OPML, Programming, RSS Ticker

Interpr.it now speaks Mozillian

My browser extension translation platform, Interpr.it, is now able to parse locale files from extensions for Firefox, Thunderbird, SeaMonkey, or any other Mozilla-powered program, and it can likewise generate Mozilla-compatible locale files. The interface for translation is the same as the one for translating Chrome extensions, but when the locales are downloaded via the API, the files are returned in the format in which they were originally uploaded (either DTD files or Java-style .properties files).

This is most obviously introducing a competitor to Babelzilla, the only major site offering a translation platform for Mozilla extensions. Babelzilla is a functionally sufficient solution for translation (I’ve used it without much issue for almost six years), but I’m moving away from it for two reasons:

  1. Translation/localization is a problem that I’d like to understand better, and I find the best way to understand a problem is to try and solve it yourself.
  2. I think that the experience of localizing an extension (or developing a localizable extension) can be better, and I have the hubris to think that I can be the one to make it better.

In the spirit of putting my money1 where my mouth2 is, I’ve moved five of my Firefox extensions (AutoAuth, Feed Sidebar, OPML Support, RSS Ticker, and Links Like This) from Babelzilla to Interpr.it.

If you are interested in trying Interpr.it, upload your extension (either using the Web form or API), and let me know how it works for you.

  1. For extremely small values of “money.”
  2. For extremely large values of “mouth.”
Standard
anyMail, Mozilla, Programming, Thunderbird, Toby

anyMail

Unlike most of my posts, this one isn’t about some new software I’ve written; it’s about some old software I’ve written.

In 2002, like any other college student out on his own for the first time, I decided to write my own e-mail client.

What – you haven’t written your own e-mail client? You will. As a complement to both Letts’ Law (“All programs evolve until they can send email”) and Zawinski’s Law (“Every program attempts to expand until it can read mail.”), I propose Finke’s Law: “All programmers will write their own mail client (and later regret it).”

I named my webmail client Toby and used it as my primary client for three years; once it was stable enough for other people to use, I started publishing updates on SourceForge (back before GitHub became the new SourceForge). Toby was my first legitimate open-source project.


Toby Web Mail, in its heyday, Outlook Express was a big influence. Also, the Pokemon cards were my brother’s.

In 2004, I needed a few more credits to finish my degree in Computer Science, and I opted to do an independent study under Joseph Konstan. (Professor Konstan was one of the only profs at the University of Minnesota at that time who had some involvement in Web software; the other one that I knew of was John Riedl, who was already the head of the fledgling Software Project class that I was a part of, which is a topic for another post.) For my project, I decided to refine my mail client, using IBM’s Remail prototype as a roadmap.

Over that semester, I rewrote Toby and branded it as anyMail. I don’t remember why I called it anyMail, but it was probably because it could connect to any IMAP or POP3 mail account. Clever.

Some of the features I planned on integrating into anyMail were inspired (read: stolen) from the very-new (at that time) Gmail; specifically, labels (instead of folders), the concept of “archiving”, and an entirely AJAX-based system for loading messages (although at that time, the term “AJAX” hadn’t been coined yet). However, one feature that I beat Google to was my own version of Priority Inbox. From my project proposal:

I intend to implement: [...] Automatic sorting of incoming mail into different categories, such as read, unread, un-replied to, messages from people the user has replied to in the past, messages likely to be spam, messages from people in the user’s address book, etc. I feel that one of the largest areas for improvement in current Web mail clients is that of automatic information sorting and presentation.

(Of course, I wasn’t the first to ever think of this concept. Microsoft has prior art dating to the late ’90s.)

A concept I implemented from IBM’s Remail was thread arcs. Thread arcs are a way to visually represent a conversation using circles and arcs:

The hollow circles are messages sent by the user; the solid circles are replies. Clicking on a circle loads the corresponding message. Neat, but not terribly useful.

As far as I know, I was the only person to have implemented thread arcs outside of the Remail prototype at that time. They’ve gotten around by now; there is even a Thunderbird add-on that generates them.

So, fast-forward to a couple of days ago, where I had blissfully forgotten about the hundreds of hours I spent writing code to parse multipart e-mail. I was searching through my e-mail for something unrelated, and I happened across a conversation between myself and Rob Malda; I was applying for a job at Slashdot, and I had sent him a link to anyMail as a part of my application. The URL was dead, so I did the reasonable thing: hunted down a copy of the source code, upload it to that server, and get it to work.


Here goes nothing.

And it did work! After ten minutes of creating the necessary database tables and an email address to use for testing (I didn’t want to inadvertently wipe the contents of an important account – I trust my code, but I don’t trust trust it), I logged in and had a working instance of anyMail before me:

I played around, sent and received a few e-mails to confirm functionality, and was mostly pleased with how well anyMail held up.


The only flaw I found was that attachments didn’t get sent. That might be a misconfiguration my Web host though.

This was amazing to me. The last time I used anyMail, it was in Firefox 1.0, and it still works in Firefox 11 and Chrome Umpteen, without the benefit of a JavaScript library like jQuery doing the heavy lifting of cross-browser compatibility. It was also written in an environment with PHP < 4.3, and ran mostly without issue under PHP 5.2.

The other thing that surprised me was how consistent my coding style has been over the last eight years. Even though I was shocked and appalled by the gaping security holes (XSS, CSRF, and SQL injection issues were all present) and disgusted by the number of synchronous XHR calls, I was pleased to recognize the code as mine. I don’t specifically remember writing it, but I can tell by the formatting, variable names, and structure that it was formed by my hand.

I spent a few minutes cleaning up the code — some basic protection against XSS and SQL injection — and uploaded it to GitHub. I don’t expect anyone to use it (and I hope nobody does, at least not without modification), but I’m keeping it around so I can look back on it and learn from my mistakes. But hey, if you need a class to generate thread arcs in PHP or an email client that works in Firefox 1.0, you know where to look.

Update: Toby is no longer supported. I recommend using RoundCube Webmail instead.

Standard
Christmas, Life

Our 2011 Christmas Letter

Dear close personal friend or requisite family member,

It’s been a banner year for the Finke household. (By which I mean: Christina took up banner making as a hobby and graced each of our family milestones with an ornate commemorative pennant.) Here’s a rundown of those banner-worthy occasions:

In January, Christina and I took a trip to Palo Alto, CA and then drove up the West Coast to see our friend Leah and her husband Jacob in Oregon. Highlights of the trip included eating at our first Vietnamese restaurant, seeing the Stanford campus, and not being in Minnesota in January.

In February, Gabriel had his first bloody nose. It looked bad, but you should have seen the other guy! (The other guy was a stepstool.)

Alternate version of the previous paragraph: In February, Gabriel got his first bloody nose by hitting his face on a stool. Better than getting your first bloody stool by hitting your face on a nose, am I right??

In March, we bought a piano. You might recall that we already had a piano a few years ago; the difference between this one and that one is that this one was not free, and that one was free for a reason.

In April, our second son Henry Gideon was born. He weighed 8 lbs. 8 oz. and was 19½ inches long. He has since gotten heavier and taller, and he plans to continue on that path.

In May, Gabriel drank some cologne, and we had to call poison control. Fortunately, he didn’t drink enough to need medical attention, but his burps still smell like a summer breeze.

In June, we went camping with Christina’s family and friends. Christina had fun. Gabriel had fun. Gideon had fun. I had hot dogs.

In July, Christina held a garage sale that probably broke records for gross revenue. Why she was selling gross things I’ll never know!

August, a.k.a. September: nothing happened.

In October, Christina and Gideon went up north with some friends of Christina’s, leaving Gabriel and me home for our first Men’s Weekend. This is where I put a cliche about the house falling apart when Christina isn’t home for a few days.

In November, I accepted a job offer from a company called Automattic, which does something with computers and Internet I think? Christina is still working part-time at GE, doing things with numbers maybe?

In December, Gabriel accomplished the incredible feat of finishing potty training. Our potty can now sit, stay, roll over, and speak.

BONUS MAD LIBS PORTION: Gabriel picked up a few new skills this year, mainly playing guitar, drumming, and   verb  ing. He also thinks that   nouns   are really   adjective  !

Merry Christmas,
Chris, Christina, Gabriel, and Gideon

Standard
Browser Add-ons, Flock, Mozilla, Mozilla Add-ons, Mozilla Firefox, Mozilla Firefox for Mobile, Netscape Navigator, Software, URL Fixer

URL Fixer Has Been Acquired

URL Fixer, one of the first add-ons I wrote for Firefox, has been acquired! It is now being managed by the team at URLFixer.org.

URL Fixer was inspired in 2006 by this Firefox bug report. Since then, it has been a featured add-on on the Mozilla Add-ons Gallery, it was one of the first add-ons to be compatible with Mobile Firefox, and it placed in the Extend Firefox 2 contest. It used to be compatible with both SeaMonkey and Flock (remember Flock?); its functionality was included in Netscape Navigator 9, and it was at one point under consideration to be included in Firefox 3.

URL Fixer has also been the subject of several experiments: it was the source of the statistics I used in my examination of what people type in the address bar, and it was the add-on I used to test the feasibility of selling a freemium browser add-on.

The new team in charge of URL Fixer recently released version 4, which you can install without needing to restart Firefox; I’m looking forward to seeing what other improvements they make and in what direction they take the add-on. Please note: support questions should no longer go to me; please send them to info@urlfixer.org.

Standard
AOL, Automattic, Blogging

Tturn, Tturn, Tturn

As Pete Seeger and King Solomon wrote, “To everything there is a season: A time to be born, a time to die; a time to plant, a time to start a new job at a new company.”

In that spirit, I’ve left AOL in order to join the team at Automattic. Not everyone has heard of Automattic, but almost everyone has heard of their flagship service, WordPress.com.

At Automattic, I’m a part of the Theme Team: a group of very attractive people with highly specialized skills in the realm of “Making your blog look good.” That’s a daunting task when you’re trying to cater to millions and millions of bloggers (and millions and millions and millions more readers), but I’m enjoying the challenge.

Standard
Browser, Google Chrome, Ideas, Internet Explorer, Life, Mozilla, Programming, Safari

An API for Browser Screenshots

What do the following screenshots all have in common?

From Amazon’s Cloud Reader Installation:

From the University of Virigina’s guide to setting proxy settings in Firefox:

From HootSuite’s TwitterBar acquisition announcement:

From Tecca’s Guide to Internet Explorer:

That’s right: they all include portions of browser chrome. (Chrome 13, Firefox 3, Firefox 4 for Windows, and Internet Explorer 9, I believe.)

What else do these screenshots have in common? They will all one day be out of date (if they aren’t already). As soon as Google modifies their extension installation dialog, or Mozilla changes their proxy settings tab, or the Firefox address bar gets a new background color, these screenshots will no longer accurately represent the interaction through which they’re meant to guide the user.

A Modest Proposal

I propose that this problem of stale browser screenshots could be alleviated by the creation of a Web service that exists solely to serve semi-dynamic screenshots of browser chrome. Allow me to explain with examples.

The Amazon screenshot above could be replaced with a call like this:

<img src="http://browsers.foo/addons/installation?highlight=confirm&w=460&h=60" />

Or the TwitterBar image could use this URL instead:

<img src="http://browsers.foo/toolbar/?include=url-bar,icon&icon=http://foo.com/hoot.png&highlight=icon" />

(Note the idea of being able to merge existing images into the screenshots.)

The IE add-ons dialog screenshot could just as easily call this URL:

<img src="http://browsers.foo/addons/tracking-protection?browser=ie&version=9&highlight=easy-list" />

The API would automatically use the user’s user-agent to determine what browser, version, and platform to show in the screenshot (although these could also be specified manually, as seen in the IE example). If images from the exact current version aren’t available, the most recent version could be used instead.

I think that with a couple dozen high-resolution, high-quality screenshots of the various windows and dialogs in each major browser version on each major platform combined with metadata defining the position of key elements in those screenshots (e.g., the home button, the address bar, the History menu), 90%+ of the browser-specific screenshots on the Web could be replaced by calls to this service.

What do you think?

Is this a solution in search of a problem, or is it a legitimately useful idea? I think it would be worth its development costs just for organizations like Mozilla or Google to use in order to populate their help documents with screenshots that would always be up to date. Tell me what you think in the comments below.

Standard