Tag: Python

MAC Randomizer Alfred Script

December 19, 2016 » Geek

A recent conversation I had dealt with free wifi that limited the amount of time you could use it before it kicked you off. Now, while I support the right of wifi providers to do as they please, it’s an interesting question. AFAIK most tracking of that sort is done based on MAC addresses, which you can easily spoof if you want.

I wrote up a quick Alfred workflow that shells out from Python to do the real work. Note that if your wifi interface isn’t called en0 this won’t work for you.

Workflow Overview

The first script shells out to ifconfig to get the current address. Which gives output like the following. We are interested in that ether f4:5c:89:b3:37:e1 line. The first three octets are of a MAC are the Organizationally Unique Identifier (OUI) and we don’t need to change those, what we have is valid already.

Our script captures the OUI, then generates three more octets for the rest of the address, and prints it out.

Next we need to actually set this new random MAC. This is a privileged operation, so it we passed it directly to ifconfig it would error out. Long story short, if we want a nice authorization dialog we have to pass through applescript, russian nesting doll style.

I also added a way to reset it to the hardware value. The networksetup command handily has that for the taking. We just shell out, capture it and pass it through to ifconfig again.

You can download this workflow, comments and improvements appreciated.

ChatOps: Snack Preview Script

November 15, 2015 » Geek

One of the things we’ve built at Pack is Snack, an iOS app which shows users five great dog photos every day.

Ideally, each Snack would be thematic, and curated. However, curation doesn’t always get done. Either no theme is apparent, or we just don’t get to it in time.

When there is no curated snack, we draw from our Editors Picks, a collection of the best dog photos available. This gives us quality content, even when we don’t have the chance to pick all five by hand.

There is a downside to this though. Some of our editors picks are specific to an event or time period. Like Christmas or Halloween photos. These wouldn’t be very good in a Snack in the middle of May.

So how do we balance the light editorial touch while making sure the random snacks are cohesive?

My solution was a preview injected into our chat. Every day a script grabs tomorrows Snack, makes a composite, and sends it into Slack for us to review. If we see something is off, we jump into the admin and fix it.

It’s fairly brute force, but a good example of centralizing tools around chat, which is something we try to do. First we get the photo information, then we download each photo. We take those and use ImageMagick to create a captioned composite. Finally, we upload that to S3 and send the link to Slack.

This first listing is pretty simple. We just send a request for tomorrows Snack JSON from the public API. You might wonder why we don’t just set day_requested to tomorrow, but the API doesn’t support that, and neither does the logic of the app. This runs on a cron job which mails failed runs to us using cronic, so we call raise_for_status() to explicitly fail on bad HTTP requests.

This next section shells out to download each image into a temporary file. Order matters in a Snack, so we use a counter instead of taking random filenames.

ImageMagick is a powerful suite of tools, and in the next block we use the montage command to stitch our photos together.

-title, as you might imagine, lets us write some title text onto the composite. -geometry specifies the size and spacing of each image, 300px square, with 20px of vertical offset from the title. Lastly, -tile lays out the images in a 5 by 1 grid instead of stacking them in the default method.

tinys3 is an awesomely light API to S3 uploads, and then we use the Slack Incoming Webhook integration to send the message to chat.

All in all a simple, effective piece code that drops needed information into our chat every morning. Here’s the whole listing for a complete picture.

And here is an example of a random Snack that needed correction. Problem solved!

Many thanks to tinys3 and requests.

Updating S3 CORS With Boto

November 17, 2014 » Geek

Sometimes you don’t have access to the S3 console, but you do have keys for a bucket. If you need to change CORS for that bucket, it turns out you can. Boto has API methods for this.

Tags: , , , ,

Homoglyph Substitution for URL’s

August 29, 2014 » Geek

At Pack we use ascii-based unique identifiers in URL’s a lot. We call them slugs. Dogs have them, users have them, breeds have them, etc.

I made the decision early on to keep the slugs plain old ascii. No unicode. These are primarily for URL’s, and I wanted them easy to type. Most slugs in the system are automatically generated. These slugs are derived from names when a dog or user is created in the system. This is a problem, because there are a lot of people in the world who use characters outside of the ascii set.

Usually, the solution is just to drop non-ascii characters. This is the simplest option, and it works. For example, Designer News uses this technique. In the case of John Henry Müller, they simply drop the ü because of the umlaut, giving him the user URL of https://news.layervault.com/u/11655/john-henry-mller/. Müller becomes mller. I find this less than optimal.

A second technique is to use homoglyph substitution. A homoglyph is a character which is visually similar to another, to the point that they are difficult to quickly distinguish with just a glance. I’m familiar with them from the world of phishing, where people register domains that look very similar to other domains by using homoglyphs.

Once you build a list of homoglyphs, it’s easy to create slugs that are ascii only through substitution. We expanded the definition of homoglyph for our list to include anything you could squint at funny and think they were similar. The method is a bit brute force, but it only ever runs once per string, and I think the outcome is worth it.

This works well for us, we get reasonable URL’s for dogs like “Hólmfríður frá Ólafsfjordur”. holmfriour-fra-olafsfjordur is not the same, but it’s close enough for a URL that you don’t mind, and it’s better than using hlmfrur-fr-lafsfjordur.

Hólmfríður frá Ólafsfjordur

Unfortunately, this doesn’t work well for un-romanized languages, notably asian languages, such as “クッキー“. In this case, the system breaks down and we end up with no usable slug, so we build from a default. I’m still seeking a solution for that. Maybe I should use automatic translation on it.

Building Pandemonium

May 18, 2014 » Geek

Pandemonium

Every year, What Cheer creates something fun for Big Omaha.

Previous years have been very interactive, requiring direct participation. A seek and find game, a conference only chat tool, etc. These have been fun, but interaction with the project is sporadic and not ubiquitous. This year we decided to build something that everyone would participate in, simply by being in the audience. Alex had the excellent idea of tracking the loudness of the auditorium over time, and we decided to monitor Twitter as well.

Hardware

To measure sound levels in the auditorium (hangar? main stage?) we would obviously need some hardware on site. We chose a Raspberry Pi for simplicity, and because we already understood it. I initially experimented with using an electret microphone and GPIO, but as time ran out I went simpler and ordered a USB audio interface to plug in.

Before the event Paul and I went to KANEKO to set things up. The helpful guy from binary.net who was setting up the network gave us a hard line so we wouldn’t have to deal with wifi traffic, we ran the mic up the wall, plugged it in and watched the data flow. Pretty smooth install.

Raspberry Pi taped to the floor boards.
Raspberry Pi taped to the floorboards.

Our little mic on the wall.
Our little mic on the wall.

Software

The architecture of Pandemonium is perhaps a bit over complex, but I was having fun gluing things together and who’s gonna stop me?

Pandemonium Architecture

Audio

Audio starts at the input, which we read with PyAudio. We read 10ms of audio, then calculate the RMS Amplitude of that data to produce our “loudness” value.

This packet gets pushed into a queue with a timestamp that is shared with the UDP client process. This process picks it up, and collects 50 other samples finding the peak value. Once it has collected 50 packets (0.5 seconds) it takes the peak value, wraps it with a signature and sends it off. The signature is an abbreviated HMAC to verify the origin and quality of the data. Originally we were sending 100% of the samples collected, so 100 per second. We decided that was a bit extreme and added the summarization code to reduce it to twice per second.

The UDP server receives the packet, unpacks it, and checks the signature. If it’s valid, it stores it in MySQL (async) and also pushes it to a Redis pubsub channel.

From there a node.js server picks it off the Redis pubsub channel and sends it down through socket.io to waiting clients. Even with all these hops, the roundtrip is pretty snappy, and there is less than a second of obvious lag.

On the client side we had a digital VU-style meter which scaled the volume over it’s seven bars and lit up accordingly. We also pushed the data to a live graph powered by HighCharts.

Pandemonium VU-Style Meter

Twitter

Tweets were collected for the hashtag #bigomaha and stored directly into MySQL by a daemon using the Twython library.

A second process would aggregate and average the tweets per second, then push that data to a Redis pubsub channel to be distributed by the node.js bridge.

Since there isn’t a natural comparative value for Tweets, the aggregator keeps the peak value in memory and compares the current value against that for a percentage. Not perfect, but it’s works.

Mistakes Were Made

Everything performed better than I expected, honestly. We didn’t have the opportunity to test the audio sampling at a large, loud venue, so I was worried about that. Paul and I installed it in the back of the auditorium, just past a speaker, and put the mic as high up the wall as we could, which seemed to isolate it pretty well.

However, there were some problems. Due to a fat finger, none of the audio data from day one was saved until about 3pm. So that was a bummer. A quick fix gave us good data for day two through.

My second goof was that the MySQL library I used for storing tweets assumed that data was latin-1, even though I created my tables as utf-8. So, when people tweeted anything with odd characters, the database barfed and it dropped the tweets. That also got fixed in the afternoon on day one.

Success?

I think it was a neat project, I certainly had fun building it. And it worked, which is always what we are aiming for, and it didn’t require any direct interaction from attendee’s to succeed, it survived on it’s own. I wish I hadn’t made mistakes, but they weren’t too damaging to the real-time experience at Big Omaha.

Day one data.
Day one data.