Category: Geek

Building the Chicken Cam

October 8, 2013 » Geek

Yesterday I published a Chicken Cam for Buttercup farm, basically just a webcam for the 21 chicks we got a few weeks ago.


Oh, hey.

I had decided to do this project on a whim. I had a Raspberry Pi lying around unused, and I figured it would be simple to get running. It was a simple project, but I hit a few constraints which made it take longer.


One of the first problems I came across was bandwidth. I live 10 minutes from Omaha, and 10 from Blair. We do not have blazing internet out here. I have ADSL which is spotty when the weather is bad. I can’t afford to dedicate too much of my upstream to a chicken cam.

My first thought was to take a photo at an interval, and push it out to S3. That would save bytes since it would push the cost of serving more than one user outside of my link. The problem with that, is I didn’t have a simple mechanism to tell my camera to start and stop sending images. It was always on, and always consuming bandwidth.

My second thought was a proxy type system, and that’s what I ended up using. I wrote a quick node app with a background function on an interval which requests a new image from the camera. It stores this JPEG into a buffer and sleeps. When it wakes up, it checks a timestamp to see if someone has recently requested a new frame. If they have, we loop again, otherwise we wait a while and check again.

Update Loop

This way we serve images at a decent rate, and don’t use bandwidth when no one is watching.

var http_callback = function(response) {
  var img = '';
  response.on('data', function (chunk) { img += chunk; });
  response.on('end', function () {
    latest_image = img;
    latest_image_timestamp = +(new Date());
    setTimeout(updateImage, REFRESH_INTERVAL);
function updateImage () {
  if(( +(new Date()) - last_request ) > PAUSE_TIMEOUT) {
    setTimeout(updateImage, PAUSE_INTERVAL);
  else {
    http.request(http_options, http_callback).on('error', function (e) { setTimeout(updateImage, BACKOFF_INTERVAL); }).end();

I put this proxy up on Heroku and it’s been humming along just fine.


The Raspberry Pi

The RPi has decent specs, it’s not a really wimpy machine in an embedded context, but I figured, why push it? I wanted to find a really lightweight way to serve up the images.

I initially looked at motion, but it was way too feature rich and heavy. Likewise I ruled out ffmpeg because I wanted stills on demand, not an MJPEG stream.

Luckily, I eventually found tinycamd, a little c program which worked well after a few tweaks.

I had to compile this myself on the RPi since it’s not in the Debian repos. Easy enough. I started with the CNXSoft minimal image and installed build-essential and libjpeg-dev.

root@raspberry-pi:~/# apt-get install build-essential libjpeg-dev

Let that run for a bit and then you can build the program. It’s a very simple program, with a very simple build process. No autotools, just type “make”.

One change I had to make to get it to compile was turn off -werror for the compiler, it was dying on a “variable set but not used” warning, which isn’t really a big deal to ignore.

root@raspberry-pi:~/tinycamd# make
cc -Wall -Werror -O2 -MMD     -c -o tinycamd.o tinycamd.c
tinycamd.c: In function 'main':
tinycamd.c:280:15: error: variable 'httpdThread' set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors
make: *** [tinycamd.o] Error 1

Removing -werror from the CFLAGS lets it build, which it does pretty quick.

Index: Makefile
--- Makefile	(revision 46)
+++ Makefile	(working copy)
@@ -1,5 +1,6 @@
-CFLAGS := -Wall -Werror -O2 -MMD $(CFLAGS) $(COPTS)
+CFLAGS := -Wall -O2 -MMD $(CFLAGS) $(COPTS)
+#CFLAGS := -Wall -Werror -O2 -MMD $(CFLAGS) $(COPTS)
 LDLIBS += -ljpeg -lpthread -lrt
 HOSTCC ?= cc

The Webcam

The next hurdle I encountered might be specific to my hardware. I’m running a Logitech UVC webcam I had laying around. It claims it can stream JPEG’s straight from the hardware, and it claims you can set the JPEG compression rate, but it was dying during the ioctl for setting that compression level, VIDIOC_G_JPEGCOMP error 25, Inappropriate ioctl for device.

Rather than fighting it further, I commented out that chunk of tinycamd and ran it with the YUYV format and pushed JPEG compression of the camera and into libjpeg. This makes it consume more resources, but it was the quickest workaround for me.

Index: device.c
--- device.c	(revision 46)
+++ device.c	(working copy)
@@ -439,6 +439,8 @@
 	  fatal_f("Unable to set requested pixelformat.\n");
+	comp.quality = quality;
+	/*
 	if (-1 == xioctl( videodev, VIDIOC_G_JPEGCOMP, &comp)) {
 	    if ( errno != EINVAL) errno_exit("VIDIOC_G_JPEGCOMP");
 	    log_f("driver does not support VIDIOC_G_JPEGCOMP\n");
@@ -449,6 +451,7 @@
 	    if (-1 == xioctl( videodev, VIDIOC_G_JPEGCOMP, &comp)) errno_exit("VIDIOC_G_JPEGCOMP");
 	    log_f("jpegcomp quality came out at %d\n", comp.quality);
+	*/
 	if (-1 == xioctl( videodev, VIDIOC_G_PARM, &strm)) errno_exit("VIDIOC_G_PARM");
 	strm.parm.capture.timeperframe.numerator = 1;

With all that done I installed it as a daemon (it ships with an init script) and I was good to go.

Running, it stays under 30% CPU and 3% of memory, and that is with YUYV conversion and 40% compression on the JPEG frames. Pretty good.

The Bunker

Got Concrete?

The final hurdle was our garage, where the chicks are kept. We have a very unusual garage, consisting of a concrete silo with very thick concrete walls. It’s also the furthest point from the router in the house. Wifi is slow and spotty out there, so I bought a bridge and ran 50′ of Ethernet cabling from the garage into the house where the bridge was set up. This decreased latency enough to make the camera viable.

The Ethernet Bridge

The End Result

It took more effort than I thought it would, and a few days of waiting for hardware to ship, but I think it was worth it. I intend to try and keep the camera running as they grow up and move outside, which will involve running even more cabling and probably a power line. We’ll see if I stick to it or not.

If you want run tinycamd on your own pi, I’ve included my build with an install script. This is the version with device JPEG compression disabled, so be aware of that if you decide to stream JPEG instead of YUYV. tinycamd-rpi-debian.tar.gz

You can also download my patch file if you want to build it yourself.

Update (2013-10-09): I published the proxy/cache code on github;

Update (2013-10-09)

In the comments Matt asked for clarification on the configuration, so I thought I would put that up here.

There are two services running, tinycamd on the RPi and my node app on Heroku. Both are HTTP servers. I’ve port forwarded 8080 through my router to the RPi (shown as on the diagram) which means that the tinycamd HTTP server on the RPi is available on the public internet if you know my WAN IP (direct chicken access!)

The Heroku app is configured with that WAN IP, so it knows where to make an HTTP request for a new JPEG. To simplify configuration, I actually have it pointing to a dynamic DNS host, which is kept up to date by software on my router.

The Network Configuration

So when you make a request to Heroku for an image, Heroku doesn’t forward that request to tinycamd, it just serves you whatever image is currently in memory. In this way, it’s not really a proxy, that’s a bad term. It’s more of a self-updating cache, because it goes out and updates the frame on it’s own schedule, not due to incoming requests.

Matt made a good point that web sockets would be a better control mechanism. I agree, but this system doesn’t have very hard real time constraints, and I’m fine with leaking a few kb for unused frames. Polling is gross, but sometimes it’s the simple option. S3 is no longer involved in serving frames, that was my first approach, which I abandoned.

Update (2013-10-10): Added a version for giggles.

Delayed Queues for RQ

April 15, 2013 » Geek

I really like RQ. It’s got a sensible API and it’s built for Python, something I can’t say about Resque, pyres is fine, but it’s not the same.

My one beef with RQ is that I can’t delay a job for an arbitrary amount of time. You queue it up and it runs. To get around that, I built a simple delayed queue system for RQ, using the same redis backend.

My delayed queues leverage sorted sets to store and select jobs. We put the job in with it’s earliest run timestamp as the score, then we have a cron job or daemon that pulls them out when they are ready and pushes them over to RQ. Simple enough!

Here’s the really relevant code, everything else is trimming.

Delaying Jobs

    def delay(self, queue, job, seconds, *args, **kwargs):
        '''Delay a queue job by a number of seconds.'''
        self.redis.zadd('queue:delayed', pickle.dumps({'job': job, 'queue': queue, 'args': args, 'kwargs': kwargs, 'id': uuid.uuid1().hex}), self._now() + seconds)

Waking Jobs

    def enqueue_delayed_jobs(self, now=None):
        '''Enqueue and clear out ready delayed jobs.'''
        if not now:
            now = self._now()
        jobs = self.redis.zrangebyscore('queue:delayed', 0, now)
        for pickled_job in jobs:
            job = pickle.loads(pickled_job)
            Queue(job['queue'], connection=self.redis).enqueue(job['job'], *job['args'], **job['kwargs'])
            self.redis.zrem('queue:delayed', pickled_job)
        return len(jobs)

You can run this at a granularity as low as one second the way it is written, and I’m sure you could go tighter if you wanted. We run it at a minute granularity, since our jobs are not highly time sensitive.

And because redis is atomic, you can run enqueue daemons/cron jobs on multiple machines, and everything should work fine, which is great for availability.

Grab the code and examples here if you are interested,

Goodbye Omaha (April Fools 2013)

April 1, 2013 » Geek, Life

Rick Astley

This year I decided to pull a tiny April fools joke. I decided this at four in the afternoon, on the day of, so I didn’t have time to prepare.

I cashed in on John Henry Müller’s recent departure from the Omaha area to pretend I was leaving too.

I quickly threw up a page on my blog that would redirect to a Rick Roll, then I realized that various Twitter clients follow HTTP redirects and unwrap links to get the “real” URL and page title. I’ve been burned by that before, so I changed my tactic slightly to move the redirect into JavaScript. That let me put a sliver of believable content onto the page so that Facebook shares would work too.

The redirect page.

Finally, I tweeted it out, and started trapping suckers!

The Tweet.

The reaction.

I didn’t get Google Analytics on immediately, so I might have missed some folks, but I got 30 uniques on it that day from Facebook and Twitter.

The Tweet.

Not bad at all. SPN even wrote about it!

Impromptu logging from a connection

October 27, 2012 » Geek

I recently participated in a live streamed event that provided a “watching now” counter usin Basically it was a super simple node.js script which incremented or decremented a variable when users joined and left the channel, and broadcasted the count to it’s subscribers. What I didn’t realize until right before the event that we might want to have a record for users on page at a given point in the broadcast. With so little time before the broadcast, I didn’t want to tinker with the server and break it, so I did the next best thing, I logged from the subscriber side.

I put up a quick PHP script on my laptop that allowed cross-domain access from the origin server and logged the incoming counter.

  header('Access-Control-Allow-Methods: GET, POST, OPTIONS');
  header('Access-Control-Allow-Credentials: true');
  header('Access-Control-Allow-Headers: Content-Type, *');
  file_put_contents('log.txt', time() . ', ' . $_REQUEST['count'] . "\n", FILE_APPEND);

Then, in Chrome’s JavaScript console, I just hooked updates from into an XHR to provide the values to my PHP.

socket.on('update', function ( data ) { $.get('http://localhost/logger.php', { count: data.count } ); } );

It worked like a charm, I didn’t have to mess with the server at a crucial point, and we got the data we needed.

Let the Facebook Object Debugger Into Staging

October 27, 2012 » Geek

One often important, and often overlooked aspect of modern web development is Open Graph tags. You know, those meta tags with weird attributes that break your page validation? That’s a whole other topic though.

Today, I want to talk about the Facebook Object Debugger, and giving it access to an HTTP Auth protected environment, such as a staging or pre-launch production site. This is Apache specific, so nginx fans will have to look elsewhere.

Assume you have this setup in your Apache config or htaccess;

AuthUserFile /var/www/staging/.htpasswd
AuthType Basic
AuthName "Secure Area"
Require valid-user

The easiest way that I’ve found to make this work is to accept based on user agent. I originally tried allowing it based on IP address, but the debugger uses many IP addresses, and after I had added a half dozen I gave up and switched to user agent.

Be aware, that because of this, it’s quite easy for someone to fake their UA and gain access, so I recommend only using this code while you actively use the debugger, and turning it off afterwards. This also prevents leaks if someone pastes the URL into an actual Facebook comment.

AuthUserFile /var/www/staging/.htpasswd
AuthType Basic
AuthName "Secure Area"
Require valid-user
# Allow from Facebook
SetEnvIfNoCase User-Agent facebookexternalhit.* facebook
Order allow,deny
Allow from env=facebook
Satisfy Any

Pretty easy!

Check out this page at AskApache for a nice guide to SetEnvIfNoCase.