Update: 2010-02-18
I put this onto the Storytlr wiki a while back. I highly recommend viewing that version instead of this one. This version will not be updated further.
Update: 2010-01-19
Edited in a minor fix, I learned a bit more when I wrote my foursquare plugin.
The correct revision to follow along with is now 75c520df.
I recently got interested in the lifestream application Storytlr.
One of the first things I wanted to do was add a plugin for github, as that is a fairly large part of my online life. Unfortunately there is very little documentation, which is understandable for a private project that just went open.
As such I had to work through some code, but all in all it wasn’t too tough. The source is very readable, just not set up with any auto-documentation.
So, as a way of giving back, I’m going to walk through the Storytlr plugin system, as I understand it. If you want to play along at home, you can grab the source from my github, and the commit we will be working from is 23136196
First things first, we need to copy off another plugin, that’s the easy way to get started. I cloned the RSS one, but they are all pretty similar to start with. Here’s what my final file tree looks like, it’s pretty similar to the starting tree, just with “Github” instead of “RSS”. This would reside at path_to_app/protected/application/plugins/github
.
├── database.sql
├── github.png
├── models
│ ├── GithubItem.php
│ └── GithubModel.php
└── views
└── scripts
├── rss.phtml
├── story.phtml
└── timeline.phtmlSo what are each of these and what do we have to do to them? Well, I like to drive my development by my data structures, so let’s start with database.sql. This is pretty straightforward SQL, and I’ll highlight some key fields after you have a look.
database.sql
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | DROP TABLE IF EXISTS `github_data`; SET @saved_cs_client = @@character_set_client; SET character_set_client = utf8; CREATE TABLE `github_data` ( `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `source_id` INT(10) UNSIGNED NOT NULL, `github_id` VARCHAR(255) NOT NULL, `title` text NOT NULL, `content` text, `repository` text, `link` VARCHAR(255) NOT NULL, `published` VARCHAR(45) NOT NULL, PRIMARY KEY USING BTREE (`id`), UNIQUE KEY `DUPLICATES` USING BTREE (`source_id`, `github_id`), FULLTEXT KEY `SEARCH` (`content`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8; SET character_set_client = @saved_cs_client; |
So what are our key items to look at here? Well, how about table naming convention? The existing tables are named as their plugin name, all lowercase, then _data. Why get fancy? Let’s use the convention.
What else is important? I kept the id and source_id pieces intact. These are actually fairly important, as I learned when I tried to make source_id into a varchar. source_id is actually the key that is tied to the sources table. That seems obvious now, but trust me, it didn’t when I was writing this the first time.
Aside from that, there isn’t too much to worry about. Just make sure to add something that you can use to identify individual updates on, github_id in this case, and then add your data fields and set your indexes. The naming of the FULLTEXT "SEARCH" key is probably important, though I didn’t test it any other way. You’ll probably find that all of your plugin tables will look more or less alike.
Now what? Well, let’s go ahead and define the model for an item, since we are already thinking about the data types. Here is a partial listing of GithubItem.php, have a look.
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | class GithubItem extends SourceItem { protected $_prefix = 'github'; protected $_preamble = 'Github activity: '; public function getContent() { return $this->_data['content']; } public function getTitle() { $title = str_replace( $this->_data['repository'], '<a href="http://github.com/' . $this->_data['repository'] . '">' . $this->_data['repository'] . '</a>', html_entity_decode( strip_tags( $this->_data['title'] ) ) ); return $title; } public function getLink() { return $this->_data['link']; } public function getType() { return SourceItem::LINK_TYPE; } public function getBackup() { $item = array(); $item['SourceID'] = $this->_data['source_id']; $item['Title'] = $this->_data['title']; $item['Content'] = $this->_data['content']; $item['Repository'] = $this->_data['repository']; $item['Link'] = $this->_data['link']; $item['Published'] = $this->_data['published']; return $item; } } |
This is a very stripped down implementation. All of your data is available to the views through the inherited SourceItem::toArray method, so you really only need to override a few methods and put in any mangle logic, as I did in the getTitle method.
One other method I was sure to override was getType. The types are enumerated in /protected/application/admin/models/SourceItem.php, and are as follows:
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | const IMAGE_TYPE = 'image'; const AUDIO_TYPE = 'audio'; const VIDEO_TYPE = 'video'; const STATUS_TYPE = 'status'; const BLOG_TYPE = 'blog'; const LINK_TYPE = 'link'; const OTHER_TYPE = 'other'; const STORY_TYPE = 'story'; |
I am not sure how they play into the rendering process at this point, but better safe than sorry, right?
Let’s move on to the views. These are all very similar, and are only rendered in different, well, views. I’ll go over timeline.phtml, but if you copy the others you should be able to piece it together in a jiffy.
1 2 3 4 | <div class="github"> <div class="title"><?= $this->item->getTitle(); ?></div> <div class="content"><?= $this->item->getContent(); ?></div> </div> |
Pretty brutal, huh? You just get your title (mangled by GithubItem::getTitle in this case) and get your content, and you are as good as done.
Finally, I’m going to address the engine that drives all of this, GithubModel.php. A lot of this is just editing boilerplate, so let’s start with that. Comments are added in-line, but aren’t in the git source.
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | class GithubModel extends SourceModel { // What is the table named? protected $_name = 'github_data'; // What is the plugin directory named? protected $_prefix = 'github'; // What fields are searchable? This is comma delimited, i.e. "content, title" protected $_search = 'content'; // What is a format string for update tweets? protected $_update_tweet = "Did %d things at github.com on my lifestream %s"; // What is the service name shown on the backend/widgets public function getServiceName() { return "Github"; } // ? public function isStoryElement() { return true; } // What is the URL for the widget links? public function getServiceURL() { return 'http://github.com/' . $this->getProperty('username'); } // Brief description for the admin interface public function getServiceDescription() { return "Github is social coding."; } // What is the name on the account (for the admin interface mostly) public function getAccountName() { if ($name = $this->getProperty('username')) { return $name; } else { return false; } } // What is the title of the account (for the front end) public function getTitle() { return $this->getServiceName(); } // The initial data import function public function importData() { $items = $this->updateData(); $this->setImported( true ); return $items; } |
Okay, now we are ready to dig into the meaty part, the data update functions. This is split into two parts, updateData and processItems. Here is a (hopefully) self-explanatory updateData implementation, more or less cloned from existing plugins.
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | public function updateData() { $url = 'http://github.com/' . $this->getProperty('username') . '.atom'; $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_HEADER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); curl_setopt($curl, CURLOPT_USERAGENT,'Storytlr/1.0'); $response = curl_exec($curl); $http_code = curl_getinfo($curl, CURLINFO_HTTP_CODE); curl_close ($curl); if ($http_code != 200) { throw new Stuffpress_Exception( "Github returned http status $http_code for url: $url", $http_code ); } if (!($items = simplexml_load_string($response))) { throw new Stuffpress_Exception( "Github did not return any result", 0 ); } if ( count( $items->entry ) == 0 ) { return; } $items = $this->processItems($items->entry); $this->markUpdated(); return $items; } |
Pretty easy, right? Just acquire the content (any way you wish, this one uses cURL), parse it out into an array and pass it on. If all goes well in processItems, then mark it as a successful update with SourceModel::markUpdated. Let’s see what processItems does.
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | private function processItems($items) { $result = array(); foreach ($items as $item) { $data = array(); $data['title'] = $item->title; $data['repository'] = substr( $item->title, strrpos( $item->title, ' ' ) + 1 ); $data['published'] = strtotime( $item->published ); $data['content'] = $item->content; $data['link'] = $item->link['href']; $data['github_id'] = $item->id; $id = $this->addItem( $data, $data['published'], SourceItem::LINK_TYPE, array( $data['repository'] ), false, false, $data['title'] ); if ($id) $result[] = $id; } return $result; } |
Again, pretty simple. We take the passed in items, mangle them as needed and stuff them into a data array for storage. About the only unknown there is the addItem call, which is in the SourceModel class. Let’s take a look at the first part of that to understand our parameters.
/protected/application/admin/models/SourceModel.php
168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 | public function addItem($data, $timestamp, $type, $tags=false, $location=false, $hidden=false, $title=false) { $data['source_id'] = $this->_source['id']; $columns = array(); $keys = array(); $timestamp = ($timestamp>=0) ? $timestamp : 0; foreach($data as $k => $v) { unset($data[$k]); if (!$v) continue; $columns[] = "$k"; $keys[] = ":$k"; $data[":$k"] = "$v"; } $sql = "INSERT IGNORE INTO {$this->_name} (".implode(',', $columns).") " . "VALUES(".implode(',', $keys).")"; |
The most important thing for us to note are the names of the arguments: $data, $timestamp, $type, $tags, $location, $hidden, $title. These are self-explanatory and help us understand why the existing plugins pass what they do. Some other pieces to note is the override of source_id on line 169, and how it builds the query from your $data arguments, on lines 174-183. Naming matters!
So, now we are back to GithubModel, and we just have a few more methods to go. What remains below are the form generators and processing for the admin interface. Github only needs one piece of information to work, the username, so that’s all we are asking for below.
GithubModel.php
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | public function getConfigForm($populate=false) { $form = new Stuffpress_Form(); // Add the username element $element = $form->createElement('text', 'username', array('label' => 'Username', 'decorators' => $form->elementDecorators)); $element->setRequired(true); $form->addElement($element); // Populate if($populate) { $values = $this->getProperties(); $form->populate($values); } return $form; } public function processConfigForm($form) { $values = $form->getValues(); $update = false; if($values['username'] != $this->getProperty('username')) { $this->_properties->setProperty('username', $values['username']); $update = true; } return $update; } } |
getConfigForm creates the form fields using the Stuffpress_Form class, which is described in /protected/library/Stuffpress/Form.php. Github only has the one element, so it is added and set to required, then we are done.
processConfigForm is similarly simple, we get the value of our field, then make sure it is valid. If it is, we save it into our model’s properties, which is a SourcesProperties class, which is in turn a Stuffpress_Db_Properties class. Essentially think of it as a persistent key/value store. Or don’t think about it at all, just use it.
At this point you should have a working plugin!
Debugging Storytlr can be tough sometimes, so make sure your config file has debug = 1, and keep an eye on /protected/logs/.
If you have any questions, comments or corrections, please let me know!


Comments
Thanks for the writeup. I’m planning on making a few plugins myself.
Cool, hope it helps!
I got a fair way into integrating Zend Captcha but it doesn’t work for some reason. I’m not very good at PHP so I’m not surprised. I still have the code but I need to get a new PSU for my development server before I go hacking again.
John, there something about storytlr that just makes you interested isn’t there? Not sure what it is but I really like storytlr even though it is missing some very important features.
I am a better administrator than I am a developer so maybe we can set up public instance of storytlr based on your branch sometime.
Hey Daniel, do you have a github or anything like that? If you get one and put your source in there I can clone it and use it. Even better, fork my branch and then I can pull in your patches. Eventually (I hope) it will all get pulled back into the core.
There definitely is something special about Storytlr, I just like being in the code and I like the way it pulls together all of my actions on the web for the big picture.
Feel free to shoot me an e-mail and maybe we can set something up.
I am still fiddling with Storytlr a bit. I am looking at a better way to manage static content. The way I REALLY would like for this to happen is for storytlr to have some type of in-built wiki, but I am playing with the idea of just making static pages more manageable.
All my code so far has been very nooby hacky stuff not worth sharing. When I do come up with some good code I will almost certainly get a github account