Tag: JavaScript

ThinFire

November 12, 2013 » Consume, Geek

Update (2014-02-10): I forked an existing mini-campfire chrome extension and added some tweaks of my own. That is what I use now. You can get it here; github.com/jmhobbs/mini-campfire-chrome

I use Campfire in my browser, and I often run multiple rooms side by side. This gets crowded in each browser window, so I made a bookmarklet to easily trim out the extra UI. It drops the sidebar and header, and widens the remaining elements. You loose some functionality, but I find I don’t really need it. A refresh will restore it to normal.

BeforeAfter

You can add this to your bookmarks bar by dragging this link to the bar: ThinFire

Impromptu logging from a socket.io connection

October 27, 2012 » Geek

I recently participated in a live streamed event that provided a “watching now” counter usin socket.io. Basically it was a super simple node.js script which incremented or decremented a variable when users joined and left the channel, and broadcasted the count to it’s subscribers. What I didn’t realize until right before the event that we might want to have a record for users on page at a given point in the broadcast. With so little time before the broadcast, I didn’t want to tinker with the server and break it, so I did the next best thing, I logged from the subscriber side.

I put up a quick PHP script on my laptop that allowed cross-domain access from the origin server and logged the incoming counter.

Then, in Chrome’s JavaScript console, I just hooked updates from socket.io into an XHR to provide the values to my PHP.

It worked like a charm, I didn’t have to mess with the server at a crucial point, and we got the data we needed.

Reading Code: Drew Gourley’s Countdown Script

April 17, 2012 » Geek

A core step of growing as a programmer is reading code. I claim to be a fan of reading other peoples code, but I never seem to do it. Yesterday the excellent @DrewGourley tweeted about some code he deployed on the (gorgeous) new MahaMusicFestival.com.

Since I enjoy Drew’s work (the Oxide site has lots of really neat stuff), but I’ve never really read any of it, I thought I’d give some code reading a whirl. As a little experiment I’ve decided to transcribe my thoughts through the entire reading process; I’m typing it as I read.

This may turn out scattered, but it will show how I read code, and hopefully poke a little hole of insight into Drew’s code.

Note: The code on Maha has since changed, and probably will in the future. For reference I’ve saved off a copy of the code I read, so you can follow along if you’d like. Please recognize that this is not my work, it’s all Drew and the Oxide crews’ – maha.js.

Also, I went back and cleaned up some grammar and spelling errors, but for the most part this is verbatim what I wrote as I was reading.

The Code

So the product I’m going to poke around in is the countdown timer on MahaMusicFestival.com – going there to check it out I find this cool piece.

And, the slick part is this great number animation, which I have captured for posterity here.

So let’s find the source for it. Popping open view-source I quickly find a script tag for http://mahamusicfestival.com/wp-content/themes/maha2012/js/maha.js in the header. Since every other script tag there is for something generic (jQuery, Modernizr, etc) I’m guessing this is it. Also it looks like the site is based on WordPress – it’s amazing how flexible themes can be.

Clicking over to that source I see it’s a collection of JavaScript bits for the site, wrapped in a jQuery ready event.

That’s a nice, and very correct way of using jQuery for page load. I’m usually lazy and just do this:

I should probably up my game a bit there, huh? Just in case another library has taken over $ onload of my scripts.

Scrolling down I see some relevant functions that are all prefixed with Oxide, like OxideSlider.

My first thought here was “namespace pollution!”, but then I realized that this is all encapsulated in that ready closure, so it’s good to go.

Lot’s of stuff I could look at here, but I’m on a mission: coundown code. Browsing down I find my target. Lines 87-208 seem to be what I’m after. I’ll deal with this in chunks now.

Declarations

First thing up is the variable declarations. Nice and orderly.

I see a few things here. First, I like the notation of $dayspan – prefixing variables with $ is a good way to visually identify jQuery objects, sort of a Hungarian Notation for jQuery.

Second, I’m a bit skittish of d, h, m, s, go and di. I can instictively identify what the first four are (day, hour, minute, second) but the last two elude me. I’m sure this will become clear as I read on, but since it feels weird to me I guess that means I like my variable names a bit more descriptive. Learning about myself through others!

Core Function

Up next appears to be the core function of which this all revolves around, OxideCountdown. And it’s a doosie, so I’m going to try and chunk it out in pieces.

First up are some variable assingments. Interestingly d, h, m, s are not integers, that’s what I was expecting. This function might hold some surprises for me!

So, this next chunk seems to be pulling the current values out of the spans, one at a time.

Interesting use of parseFloat here, as I would have expected parseInt. Could this be related to the bugs in interpretation by parseInt?

So, it works it’s way through each time span, grabbing the value of the each digit in order and pushing it onto the corresponding array, so that each array would read out the same as the visual version, i.e. today d = [ 1, 1, 6 ];

Also, it’s interesting to see that go seems to be an accumulator, but I’ve not figured out what for yet. It’s just incrementing with all the values of all the digits, which doesn’t seem useful to me. i.e today go would equal 8 (1+1+6) after the $dayspan.each. My curiosity is piqued!

Okay, on to the next chunk!

Math!

So here’s the next block, I warn you it’s a bit hairy!

So I’m now seeing that di is how many digit’s will be displayed in the days area? I’m lost now, I was operating on the assumption that each section has a fixed number of elements in it. Let’s jump to the HTML real quick…

So yeah, I suppose it’s possible that days might not have all those spans in it, but that seems true of all of the blocks if we are going down that route. Hopefully reading more will shed some light. But first, let’s double back and look at the math a piece at a time.

Looks like this will be called once a second, and will decrement the right-most seconds column by one second (the reverse indexes here could get confusing).

It seems that the rest of this section is just balancing the carry for that subtraction. Here’s the seconds code, but I stuck in comments where I heard them in my head. The pattern repeats up until we hit tens of hours.

Now that we have an underrun on tens of hours we have to tinker with days. So there is what di is for! Since the number of spans for days is (apparently) flexible, we are going to use the length of the days array to access the largest element. Makes you wish JavaScript was allowed negative indicies on arrays (d[-1]), but that would of course break [] object notation for the key “-1”, which as we all know is super useful ;-p

The loop that follows just does the same carry algorithm on all the days blocks.

Bug?

At this point I noticed something I thought might be a bug. In the tens of hours decrement block I saw that h[0] is set to “5”. I’m not sure that is possible, since there are only 24 hours in the day, so it seems that at midnight a tick would mess up the count. I’m set up a quick test script for this, since I don’t want to stay awake until midnight watching :-)

After running it, I got bad output, sure enough.

Corrected code would be this, which actually reaches backwards into h[1] to fix it.

Update: I shot Drew an email and he fixed it. He even gave me credit!

Moving On

Ah ha! Now I finally see what go is for!

It’s an accumulator to know when to stop this whole thing! Evidently there is a timeout set on timer, which we haven’t seen yet.

When all the digits in the countdown are 0, the accumulator is also 0. Tricky!

I would usually implement it as a boolean, but I like it this way. It’s trickier, but it cleans up a lot of potential if statements. Neat!

Now we are back to animate in all the various numbers, if they need animating. Again we iterate over the digits in each span, and then apply a really cool custom animation on them if they have changed.

The animation takes advantage of hidden overflow in the CSS to animate the digit down and thus out of view, then moves it up above the visible area and animates in back down and into view again. It’s a great combination of CSS and jQuery for a nice effect.

Since the bulk of this is stuff we’ve seen, I’ll just let show the code (though I am curious what’s going on in that “Messy” comment, crazy chaining!

Lastly we declare and kick off the timer for the count down. I would probably have moved this up, it’s a bit weird to see timer on line 151 before it’s declared. It’s a little thing, and just my preference.

Conclusions

So that was neat!

Since I found a bug Drew has fixed that and went back and tweaked the code a bit more. Most notably he pulled out and boiled down the countdown animation into this function.

This is a really great effect, I can’t emphasize how cool it looks.

Reading another persons code exposed a few things about my own coding style that I wasn’t conscious of. What surprised me most is that I wasn’t bothered by any formatting issues. I’m rather opinionated on formatting, for instance I claim to despise cuddled else’s (no newline between a closing curly brace and an else), but Drew does that and I breezed right over them.

It was also interesting to see a totally different approach to this problem. I implemented a countdown in the middle of last year, but it was day only and not JavaScript. My natural approach would be handling time pieces as plain integers, so it was cool and kind of mind bending to see this approach of partitioning digits into arrays.

I think I will try to make a habit of this process, and I like the brain dump while reading, I think it helped it stick better to express it in writing while fresh in my mind.

Does anyone have any suggestions for what to read next? A favorite piece of code? Preferably something I can do in one sitting.

Naive Search with JavaScript

February 2, 2012 » Geek

On a recent project I had the need to implement a basic search functionality into a web interface. This isn’t unusual, but what was different is that the data in this project was fairly static, and was kept in JSON on disk (and cached into memory). Additionally, there would be a point where it was moved off of our servers, onto an entirely different stack (hence the JSON).

There was minimal server side processing so far, and I did not want to add more overhead that would need to be ported. So, I decided to implement my search in JavaScript (with some help from Python). My idea was to do very basic string matching on a pre-built index. In this article I am going to lay out my implementation, but on a dummy data set.

The Process

This search breaks down into three steps:

  1. Build Search Index
  2. Perform Search
  3. Connect To Interface

Before we start writing that, let’s cover the data.

The Data

Here is the data I will be sifting through. To keep it simple, I’m just searching through some sentences with associated ID’s. It’s short and simple, but with some tweaks you can apply this to bigger data sets, as I did.

My data set will be represented in JSON for portability and easy interpretation from JavaScript.

data.json (92278f)

The Index

To facilitate easy matching, I’m going to build a search index that will be a dictionary, with individual words (tokens) as the keys, and arrays of ID’s as the values.

I’ve decided on Python for my index builder. Handling JSON from Python is easy, though if you have a version older than 2.6 you will need to use a different import, such as simplejson.

The core of this functionality is the tokenizer. To build this, we need to determine our rules. Since this is a simple search, I’m going to tokenize on word boundaries, it will be case insensitive, and I will only accept the characters A-Z, single quote and dash inside of a word.

Here is my implementation of the tokenizer, that follows these rules:

build_index.py (92278f)

The remainder of the Python is simply opening and parsing the JSON.

build_index.py (92278f)

Run that, and we get the dictionary we wanted:

output.txt (92278f)

JavaScript

Now that we have an index to work with, let’s convert it to JSON. This just takes a little tweak to the Python.

build_index.py (89ba77 )

That gives us a JSON file to work with.

index.json (89ba77)

Now we’re going to load that JSON with an AJAX call, so we have the index to work with.

search.js (89ba77)

Once we have that data, finding matches is just a matter of looking them up in Search.index. Below I’ve used a loop that will search through an array of terms, and then accumulate the matches.

search.js (89ba77)

It works!
It works!

Hook It Up

Just a little bit remains to connect this thing. First we need to port over the tokenizer from Python to get a consistent result. Pretty easy port.

search.js (216a15)

Tokenize!
JavaScript tokenize in action.

Last we just need to apply our functions to some inputs, which we hook up with some jQuery.

index.html (216a15)

Hooked Up

Make It Better

This is not, by any means, a complete system. You would want to grab the results and match up the ID’s to a more useful output. Additionally, you could drop the AJAX call and use a script tag to bring in the index.

There are plenty of improvements you can make, and I’d love to know if you make them!

For reference, here is the complete source: https://gist.github.com/1673557.

Fundamentals: Hashing

November 20, 2011 » Geek

What the hash?

This post is about a fundamental programming concept, hashing.

“A hash function is any algorithm or subroutine that maps large data sets to smaller data sets, called keys.”

Hash function
Wikipedia – 2011/11/15

So, a hashing function can turn this:

Hello, my name is John Hobbs. I like hashes.

into this:

8ced81aa42cd91930024054f26ed92ea

Doesn’t seem super useful, does it? The content looses its semantic value. 8ced81aa42cd91930024054f26ed92ea is not the same as a personal introduction.

That’s okay though, we can find a use for a hash that won’t be seen by a human.

But first, let’s try our hand at writing a hash function.

Mapping to keys

So, the principal function of a hashing algorithm is to reduce a large block of data into a smaller block of data.

Let’s set up some arbitrary limits for developing our function.

For simplicity, let’s work at hashing values into a one byte integer, so eight bits.

Let’s also focus on only hashing ASCII strings, since ASCII characters are conveniently one byte long.

Now that that is established, let me propose a hashing function in pseudo-code:

For those of you not familiar with XOR, I mean exclusive or. This means comparing the bytes bit by bit. If the bits are the same, the result is zero. If they are different, the result is one. The common notation for XOR is ^. Here’s an example of 5 ^ 117:

Working at a byte by byte level like this ensures that our hash will not exceed 8 bits in length, or an integer value of 128.

Buckets

With our shiny little hash function in mind, let’s use find a use.

One great use is lookup tables. These are sometimes called dictionaries, associated arrays, hash tables, etc.

Basically, they map a key to a value. So you tell a table that “my name” is “john”, and then later, if you ask it for “my name” is tells you “john”. Great tool for amnesiacs.

Hashing is perfect here for faster lookups, and for giving bound memory performance.

Think of it as a finite line of buckets. When you want to store something you hand it to the walrus, and say “Please keep this fish in a bucket called ‘Salmon’.”

Now, the walrus only has so many buckets, and none of them are labeled “Salmon”.

So, he uses a special walrus method (a hashing function) that deterministically says “Salmon == 5”, and puts the fish in the fifth bucket.

Later, when you come back and ask for the fish from the “Salmon” bucket, he knows to get the fish out of the fifth bucket because he can derive that “Salmon == 5” using the same method as before.

Unfortunately, he ate the fish already, so there is nothing to give you

Walrus are selfish that way.

Implementation

Now that we have a use case, let’s implement our hashing function.

I chose JavaScript here.

Let’s give it a try:

Cool! Now there is just a little more glue to make this complete. We have to have a get and a set for values, and an array to store them in.

The get and set functions are pretty similar. You hash the key, and use that integer as the index to either set, or retrieve a value in the table array.

Let’s give it a try!

Perfect!

Collision course

Now let’s use our SimpleHashTable to store some stuff!

Wait, what the heck just happened?!

It shouldn’t have printed the value for q, we asked for salutations!

This is known as a collision, and it happens.

What happened is that the hash value we compute for q is the same value as the hash for salutations. It’s bound to happen, we only have 128 possible buckets and an infinite amount of possible keys.

So we have several ways to improve this situation.

One option is more buckets.

We can also improve the hashing function, but this only buys us fewer, more predictable collision rates.

Another common fix is a second level to each bucket keeping the exact key, value tuple, and doing a scan on collisions.

That would be like the walrus putting a sticky note that says “Salmon” on the fish before tossing it into the bucket. Then, when you ask for the “Salmon”, he rifles through the bucket, checking each fish until he finds one labeled “Salmon”.

I’ll leave these as an exercise for the reader, because it’s time we got moving on.

Real hashes

Okay, so now you have an example of a simple hash you could use for table lookups. That’s cool and everything, but in general we should leave details like that to the neckbeards. It’s good to know roughly how it works, but we don’t need to do it ourselves.

What else are hashes good for? Well, there is a specie of hash called a cryptographic hash. These hashes are cryptographic primitives, with strong promises of distribution and collision avoidance.

Examples you may have come across are MD5, or the SHA family of algorithms.

Implementation

These algorithms are considerably more complex than the hash function we created earlier, so I would urge you not to write your own except as an exercise.

Implementations are available in a variety of languages, so Google wisely.

Uses

This class of hash has several every day uses that apply to developer-types.

Data Integrity

Since hashes are deterministic functions, the output can be used as a fingerprint for the input. Passing data and the hash of that data allows for others to quickly check and make sure that the data has not become corrupted or tampered with in transit.

This assumes a secure channel for the hash, so use it wisely.

If you are on a Linux machine, you can quickly get the fingerprint for a file. In this example I’ll use the sha1 algorithm to verify the Redis 2.4.2 source code.

Comparing that hash to the Redis website, it looks like my download completed without errors. Huzzah!

Secret Verification

Another common use of hashing is to verify secrets, without storing the secret in the clear.

Say I was writing a system login, and I needed to use usernames and passwords.

I know it’s bad to store passwords in plaintext on my server. What if my server gets hacked and the user table is stolen?

I can’t store my passwords in the clear, but I need to reliably verify them.

Hashes make for a great solution to this problem.

You hash the password and store it in your database. Then, when a user wants to login, you hash the password they provide and compare it to the one in the database.

If they match, you log them in.

Doing this properly is a bit more complicated than that, so if you need to work with passwords like this, I recommend you read up.

Data Authenticity

We can also use hashes to confirm the authenticity of a message.

Say we both know a secret phrase, and no one else does.

If you want to send a message to me, and want it to be provably from you, you can send me the message along with a hash of the message concatenated to the secret.

Then, when I get the message, I also take the hash of the message plus the secret and compare it to your provided hash.

If they match, then I know it’s from you.

This is encapsulated in something called a Hashed Message Authentication Code, or HMAC.

File De-duplication

One final case that I regularly use is file de-duplication.

If you host uploads for a website, it’s a nice trick to store upload meta data (user, original name, upload time, etc.) separate from the file itself.

By de-coupling these you can hash files on upload, and if you already have a copy of that content, discard it and point the meta data at the existing file.

Here is an example in pseudo-code:

Conclusion

Hashes are useful tools for programmers, so learn the concepts and the applications.

I hope you learned something from this post! How do you use hashes?