The massively popular social network Facebook uses images to display the e-mail addresses of your friends, making it impossible to copy the e-mail address or click on it to send e-mails to your friends, thus making Facebook’s own proprietary in-site messaging system more attractive to its users. Yesterday, Gervase Markham posited that it should be possible to determine the text displayed in the image programmatically by way of the canvas tag and some JavaScript. I’m writing this to confirm that it is indeed possible and has been achieved.
The extension I’ve written to do this is called Facebook Image-to-Email. On any Facebook page containing an e-mail address image, the extension converts the image to text using the following workflow:
- Copy the image to a canvas using drawImage()
- Scans through the canvas to find matches for a pre-determined set of character sprites using getImageData
- Replaces the image with a clickable text e-mail address
Here’s an example of a “before” view:
And “after”:
Of course, this appears to be pretty simple. Ironically, the hardest part of solving this problem was the only part that Gervase remarked would be trivial: matching the image against a set of known character patterns. Since the font is not monospaced, and certain letters bleed into each other when adjacent (such as “89” and “ef”), it wasn’t possible to just store the pixel values for each full letter. What I ended up doing was only matching against only the center of the letters (which are never affected by adjacent letters) and just ignoring character edges.
One other detail as to the implementation: there appears to be some sort of security restriction in Firefox on reading data from images that are not in the same domain as the script reading them. For example, trying to call getImageData() from the chrome on a canvas that contained an image loaded from facebook.com returned null every time; the same happened if the script was running locally but loading a remote image. For this reason, the actual scripting that converts the image to text has to be injected into each page that requires it so that it appears to be running in the same domain as the image.
I’m not claiming that this is the most efficient implementation, but it is definitely complete. In my testing so far, it has correctly identified 100% of the e-mail addresses displayed in the images.
Leave a Reply