Caveat: This whole thing was thrown together without any regard for coding or design conventions. Also, I can't guarantee that you won't be banned from Digg for using any of this information - I was, a few times. Proceed with caution.

First, the data structures. The Top Diggers lists are supported by two database tables: `digg` and `digg_users`. `digg` holds the set of front-page stories and their submitters, while `digg_users` stores aggregated information about each user in the system.

CREATE TABLE `digg` (
  `user` varchar(128) NOT NULL default '',
  `submission` varchar(255) NOT NULL default '',
           `date` datetime NOT NULL default '0000-00-00 00:00:00',
  PRIMARY KEY  (`submission`),
  KEY `user` (`user`)
);

CREATE TABLE `digg_users` (
  `user` varchar(128) NOT NULL default '',
  `frontpage` int(11) NOT NULL default '0',
  `dugg` int(11) NOT NULL default '0',
  `submitted` int(11) NOT NULL default '0',
  `profileViews` int(11) NOT NULL default '0',
  `frontpagestatic` int(11) NOT NULL default '0',
  `frontpagetotal` int(11) NOT NULL default '0',
  `submittedstatic` int(11) NOT NULL default '0',
  `image` varchar(128) NOT NULL default '',
  UNIQUE KEY `username` (`user`)
);

Next, some data to get you started. Here is a SQL dump of the two tables described above, including information on about 13000 Digg users (for `digg_users`), as well as the last 4 stories on the Digg homepage at the time of the last update (for `digg`). Import this data into your database in preparation for the next step.

The next step: data retrieval. The main work of updating the top 100 list is done by a Python script:

import digg

# Grab the last X pages of popular stories
digg.update_news(100)

# Update the top X profiles
digg.update_profiles(110)

Of course, this code makes no sense without the digg.* methods, downloadable here. This script also requires the excellent BeautifulSoup Python HTML parser. You will have to modify digg.py to change the database connection parameters.

For those who don't care to read through the code, it achieves two main objectives: Find out who submitted any frontpage stories since the last update (it stops when it hits a story already in the `digg` table), and using that information, determine the new top 100 users and update their profile information.

The last step is data presentation. The information in the database tables needs to be transformed into a readable HTML file. I'll leave this step as an exercise for the reader, but to get you started, this SQL query will get you the data you want in an easy-to-read format:

SELECT
   user `Username`,
   frontpagetotal `Frontpage Stories`,
   submitted `Stories Submitted`,
   dugg `Stories Dugg`,
   profileViews `Profile Views`
FROM digg_users
WHERE
   frontpagetotal <= submitted
ORDER BY
   `Frontpage Stories` DESC,
   `Stories Submitted` ASC,
   `Stories Dugg` DESC
LIMIT 100

So to sum up, if you want to manage your own "Top 100 Diggers" list, take the following steps:

1. Import the dump of digg data linked above.
2. Set up and run your scripts
3. Create a readable version of the data.

Have fun, and beware the Digg ban-hammer.

Oh, and where is that OpenID support that was promised to be added "sometime this year" in early 2007? Here is what I wrote in an e-mail to Tom Drapeau after hearing that:

I'm not normally a betting man, but I'd put a week's pay against Digg actually rolling that out before December 31, 2007. Their ratio of features promised to features delivered is, how you say, crappy.

I'd say that I won that bet quite handily.

Gaming the system like this will not be tolerated, and any Digg users caught breaking the laws of mathematics will immediately be banned from both Top Users lists.

Top 1000 Diggers as of 2007/09/26

If you'd like to be notified whenever I release a new dataset, you can subscribe to the Digg Statisical Data RSS feed, which will include only the dataset posts, or my main RSS feed, which is updated with all of my blog posts.

Top 1000 Diggers as of 2007/09/12

If you'd like to be notified whenever I release a new dataset, you can subscribe to the Digg Statisical Data RSS feed, which will include only the dataset posts, or my main RSS feed, which is updated with all of my blog posts.

(Here's the actual Digg story.)

Top 1000 Diggers as of 2007/09/05

If you'd like to be notified whenever I release a new dataset, you can subscribe to the Digg Statisical Data RSS feed, which will include only the dataset posts, or my main RSS feed, which is updated with all of my blog posts.