F@H Team Statistics Scraper

December 11, 2009

I created a team for Little Filament on Folding@home. Our team number is 172406 (in case you want to join), but I wanted to add our latest stats on the Little Filament site. As far as I can tell there is no API for the stats, so I worked up a scraper in bash.

Basically all it does is fetch the page, then grep and sed it’s way to the variables, finally dumping them into a json file (for easy JavaScript consumption).

The kicker is that the stats server is overloaded or down a lot, so we can’t rely on it and we don’t want to stress it out further. My decision was to poll it at a large interval, 12-24 hours. I don’t have enough clients on the team to exact significant change over 6-12 hours, but I don’t want to fall too far out of date either. So if the server is overloaded and drops it once or twice, not a big deal.

Without further ado, here is the script.

That cranks out fah_check.json, which looks like this:

To see it in action, check out the Little Filament Folding page.

Categories: Geek
Tags: , , , ,

Comments

  1. […] I’ve posted before, I’ve started running Folding@home on my machines. One issue I’ve found is that on a […]

Leave A Comment

Your email will not be published.