Updating S3 CORS With Boto

Sometimes you don’t have access to the S3 console, but you do have keys for a bucket. If you need to change CORS for that bucket, it turns out you can. Boto has API methods for this.

# -*- coding: utf-8 -*-

from boto.s3.connection import S3Connection
from boto.s3.bucket import Bucket

AWS_ACCESS_KEY = 'YOUR KEY HERE'
AWS_SECRET_KEY = 'YOUR KEY HERE'
S3_BUCKET = 'YOUR BUCKET HERE'

cors_xml = """

  
    *
    GET
    POST
    *
  

"""

connection = S3Connection(AWS_ACCESS_KEY, AWS_SECRET_KEY)
bucket = Bucket(connection, S3_BUCKET)

print "Current CORS:"
print bucket.get_cors_xml()

bucket.set_cors_xml(cors_xml)

print "New CORS:"
print bucket.get_cors_xml()

Addressing Nested Dictionaries in Python

I think that post title is right. Essentially I mean dynamically accessing attributes from nested dictionaries or tuples.

Let’s say you’ve got a JSON response like this:

{
  "error": false,
  "body": {
    "name": "John",
    "job": {
      "company": "What Cheer",
      "position": "Developer"
    }
  }
}

The JSON module would convert that into nested dictionaries, like this:

>>> print usr
{u'body': {u'job': {u'position': u'Developer', u'company': u'What Cheer'}, u'name': u'John'}, u'error': False}

In my case, I was trying to provide a runtime specified format string which could use any of the values at any depth.

There isn’t (to my knowledge) an easy way to address into a deep structure with a single string. I considered value based format strings ('%(name)s' but there is no way to descend from there either.

My solution was to use a dot notation and evaluate it for field values.

This requires a strict policy not to use dots in your keys, but that is not an issue for my use case.

Here is my code for the dot notation:

def getByDotNotation( obj, ref ):
  val = obj
  for key in ref.split( '.' ):
    val = val[key]
  return val

And here it is in use against the object above:

>>> getByDotNotation( usr, 'body.name' )
u'John'
>>> getByDotNotation( usr, 'body.job.position' )
u'Developer'
>>> getByDotNotation( usr, 'error' )
False
>>> 

The next (optional) step would be to create a wrapper object.

class DotAccessibleDict ( object ):
  def __init__ ( self, data ):
    self._data = data

  def __getitem__ ( self, name ):
    val = self._data
    for key in name.split( '.' ):
      val = val[key]
    return val

Which we can then use like so:

>>> wrapped = DotAccessibleDict( usr )
>>> wrapped['body.name']
u'John'
>>> wrapped['body.job.position']
u'Developer'
>>> wrapped['error']
False
>>> wrapped['nope']
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 7, in __getitem__
KeyError: 'nope'
>>> 

While this is just sugar, it does look nice doesn’t it? To be complete you would want to implement the other sequence methods such as __setitem__

So that’s my fix – what’s yours?

Replacing Kohana 3 Auth module hashing

The password hashing in the Auth module provided with Kohana 3.1 is not very good. By default it is a simple sha256 hmac with a global salt.

modules/auth/classes/kohana/auth.php

public function hash($str)
{
  if ( ! $this->_config['hash_key'])
    throw new Kohana_Exception('A valid hash key must be set in your auth config.');

  return hash_hmac($this->_config['hash_method'], $str, $this->_config['hash_key']);
}

This isn’t strong. If you loose the hashes and the salt it’s just a matter of winding up a GPU.

So how can we fix this? Well, thanks to Kohana’s structure we can easily override the Auth class and tweak it. However, due to Auth’s structure, we can’t drop the global salt. The hash function has to stand alone, so no passing in salts from the database.

That leaves us with key stretching.

Now, I don’t want to deal with a custom key stretching implementation, I’m not a cryptographer. So, let’s find an existing algorithm.

One that pops to mind is PBKDF2. This is a pretty simple algorithm, so it was easy to find and spot check a PHP implementation

We just take some test vectors from RFC 3962 and run them against the code we found.

 1,
      'bits' => 128,
      'expected' => "cd ed b5 28 1b b2 f8 01 56 5a 11 22 b2 56 35 15"
    ),
    array(
      'rounds' => 1,
      'bits' => 256,
      'expected' => "cd ed b5 28 1b b2 f8 01 56 5a 11 22 b2 56 35 15 0a d1 f7 a0 4b b9 f3 a3 33 ec c0 e2 e1 f7 08 37"
    ),
    array(
      'rounds' => 2,
      'bits' => 128,
      'expected' => "01 db ee 7f 4a 9e 24 3e 98 8b 62 c7 3c da 93 5d"
    ),
    array(
      'rounds' => 2,
      'bits' => 256,
      'expected' => "01 db ee 7f 4a 9e 24 3e 98 8b 62 c7 3c da 93 5d a0 53 78 b9 32 44 ec 8f 48 a9 9e 61 ad 79 9d 86"
    ),
    array(
      'rounds' => 1200,
      'bits' => 128,
      'expected' => "5c 08 eb 61 fd f7 1e 4e 4e c3 cf 6b a1 f5 51 2b"
    ),
    array(
      'rounds' => 1200,
      'bits' => 256,
      'expected' => "5c 08 eb 61 fd f7 1e 4e 4e c3 cf 6b a1 f5 51 2b a7 e5 2d db c5 e5 14 2f 70 8a 31 e2 e6 2b 1e 13"
    ),
  );

  foreach( $tests as $test ) {
    print $test['rounds'] . ' rounds at ' . $test['bits'] . ' bits ' . "\n";
    $start = microtime( TRUE );
    $result = trim( preg_replace( '/(..)/', '\1 ', bin2hex( pbkdf2( 'password', 'ATHENA.MIT.EDUraeburn', $test['rounds'], $test['bits']/8, 'sha1' ) ) ) );
    $diff = microtime( TRUE ) - $start;
    print 'Expected: ' . $test['expected'] . "\n";
    print '     Got: ' . $result . "\n";
    if( $result == $test['expected'] ) {
      print "MATCH\n";
    }
    else { 
      print "NO MATCH\n";
    }
    print 'Took ' . number_format( $diff, 10 ) . "\n\n";
  }

Run it, and everything checks out:

1 rounds at 128 bits 
Expected: cd ed b5 28 1b b2 f8 01 56 5a 11 22 b2 56 35 15
     Got: cd ed b5 28 1b b2 f8 01 56 5a 11 22 b2 56 35 15
MATCH
Took 0.0000329018

1 rounds at 256 bits 
Expected: cd ed b5 28 1b b2 f8 01 56 5a 11 22 b2 56 35 15 0a d1 f7 a0 4b b9 f3 a3 33 ec c0 e2 e1 f7 08 37
     Got: cd ed b5 28 1b b2 f8 01 56 5a 11 22 b2 56 35 15 0a d1 f7 a0 4b b9 f3 a3 33 ec c0 e2 e1 f7 08 37
MATCH
Took 0.0000190735

2 rounds at 128 bits 
Expected: 01 db ee 7f 4a 9e 24 3e 98 8b 62 c7 3c da 93 5d
     Got: 01 db ee 7f 4a 9e 24 3e 98 8b 62 c7 3c da 93 5d
MATCH
Took 0.0000147820

2 rounds at 256 bits 
Expected: 01 db ee 7f 4a 9e 24 3e 98 8b 62 c7 3c da 93 5d a0 53 78 b9 32 44 ec 8f 48 a9 9e 61 ad 79 9d 86
     Got: 01 db ee 7f 4a 9e 24 3e 98 8b 62 c7 3c da 93 5d a0 53 78 b9 32 44 ec 8f 48 a9 9e 61 ad 79 9d 86
MATCH
Took 0.0000200272

1200 rounds at 128 bits 
Expected: 5c 08 eb 61 fd f7 1e 4e 4e c3 cf 6b a1 f5 51 2b
     Got: 5c 08 eb 61 fd f7 1e 4e 4e c3 cf 6b a1 f5 51 2b
MATCH
Took 0.0019500256

1200 rounds at 256 bits 
Expected: 5c 08 eb 61 fd f7 1e 4e 4e c3 cf 6b a1 f5 51 2b a7 e5 2d db c5 e5 14 2f 70 8a 31 e2 e6 2b 1e 13
     Got: 5c 08 eb 61 fd f7 1e 4e 4e c3 cf 6b a1 f5 51 2b a7 e5 2d db c5 e5 14 2f 70 8a 31 e2 e6 2b 1e 13
MATCH
Took 0.0144000053

So now all that’s left is to drop it in, which is pretty simple. One thing to note is that I wanted this to stay compatible with the default auth config file, so I just extended that a little bit.

application/classes/auth.php

_config['hash_key'] )
        throw new Kohana_Exception( 'A valid hash key must be set in your auth config.' );

      if ( 'pbkdf2' == $this->_config['hash_method'] ) {
        return base64_encode( self::pbkdf2( 
          $str, 
          $this->_config['hash_key'], 
          Arr::get( $this->_config['pbkdf2'], 'rounds', 1000 ),
          Arr::get( $this->_config['pbkdf2'], 'length', 45 ),
          Arr::get( $this->_config['pbkdf2'], 'method', 'sha256' )
        ) );
      }
      else {
        return parent::hash( $str );
      }
    }

    /** PBKDF2 Implementation (described in RFC 2898)
     *
     *  @param string p password
     *  @param string s salt
     *  @param int c iteration count (use 1000 or higher)
     *  @param int kl derived key length
     *  @param string a hash algorithm
     *
     *  @return string derived key
     *
     *  @url http://www.itnewb.com/tutorial/Encrypting-Passwords-with-PHP-for-Storage-Using-the-RSA-PBKDF2-StandardL
    */
    public static function pbkdf2 ( $p, $s, $c, $kl, $a = 'sha256' ) {

        $hl = strlen(hash($a, null, true)); # Hash length
        $kb = ceil($kl / $hl);              # Key blocks to compute
        $dk = '';                           # Derived key

        # Create key
        for ( $block = 1; $block <= $kb; $block ++ ) {

            # Initial hash for this block
            $ib = $b = hash_hmac($a, $s . pack('N', $block), $p, true);

            # Perform block iterations
            for ( $i = 1; $i < $c; $i ++ )

                # XOR each iterate
                $ib ^= ($b = hash_hmac($a, $b, $p, true));

            $dk .= $ib; # Append iterated block
        }

        # Return derived key of correct length
        return substr($dk, 0, $kl);
    }

  }

application/config/auth.php

 'orm',
    'hash_method'  => 'pbkdf2',
    'hash_key'     => 'zomg',
    'lifetime'     => 1209600,
    'session_key'  => 'auth_user',
    'pbkdf2'       => array(
      'method'  => 'sha256',
      'rounds'  => 1000,
      'length'  => 45,
    )
  );

One item to note is that I am packing these with base64_encode. This is to fit into the default field type for the ORM driver. That is also why my length is stunted to 45. If you really want to go all out, alter your table to use a TINYBLOB, up the length to 256 bit and up the rounds.

So that is how I replace weak hashing in K3 with something a bit better.

How do you do it?

iCloud Shimmer Effect

Today Alex pointed out the new iCloud website had lot’s of fancy effects. One I liked best was the polished metal effect on the login box that shimmered when you moved your mouse.

I went ahead duplicated it as best I could in a short time. There are some obvious differences in approach, but it’s essentially the same.

View The Demo

One thing I did not do was the easing on the mouse move. I really like that, but it would be time consuming to get it running.

Also, I’m not browser compatible. I only tested it in Chrome 14.

Most of the work is done in two functions.

mousemove takes the mouse position and converts it to a degree of rotation.

mousemove: function ( event ) {
  // Use the mouse x coordinate conbined with the window width to 
  // come up with a degree to rotate.  You can make it more responsive
  // by decreasing the reduction.
  var reduction = 200;
  var deg = ( window.innerWidth / 2 - event.clientX ) * -1 / reduction;
  if( deg != Shimmer.current_rotation ) { Shimmer.draw( deg ); }
}

draw rotates the canvas and draws the image onto it.

// Rotate the metal background
draw: function ( deg ) {
  Shimmer.current_rotation = deg;

  // Clear the canvas
  Shimmer.context.clearRect( 0, 0, Shimmer.canvas.width, Shimmer.canvas.height );
  Shimmer.context.save();

  // Set the rotation point at 50% from left and & 80px from top
  Shimmer.context.translate( Shimmer.center.x, 80 );

  // Rotate by degrees (convert to radians)
  Shimmer.context.rotate( deg * Math.PI / 180 );

  // Draw metal
  Shimmer.context.drawImage( Shimmer.image, Shimmer.offset.x, Shimmer.offset.y );

  // Clear transforms
  Shimmer.context.restore();
}

That’s essentially it. Simple, but visually powerful. The source is embedded in the demo and commented.

Kohana 3 OAuth & Twitter Demo Code

The Internet seems a bit sparse when looking for good demo code for using the Kohana 3 OAuth module with Twitter.

I think the main issue is that the OAuth module isn’t very well documented, and doesn’t do API requests. For that you need an API implementation, like shadowhand/apis.

Anyway, here is a gist I put together with an example controller for Twitter OAuth:

https://gist.github.com/1267793