devdev2050

Merged in a bunch of patches from fangel and added support for RSA-SHA1 to the library.

There are still some gotches related to keys, they are marked in the code but for the moment you’ll have to extend the OAuthSignatureMethodRSASHA1 class in order to use it because the method for fetching your public and private keys is being skipped.

You can go test using the test client and test server

The keys being used are down at the bottom when RSA-SHA1 is selected as the current signing method.

As always, code is available at http://code.google.com/p/oauth/source/browse/code/php

Wee, some of the code I worked on has been released (and there’s a blog post about it), it’s an SDK that lets you use the Django development process with App Engine.

Works really well already and there is an active google code project where you can report bugs and get your patches reviewed.

Seems like every time I try to make use of it I am hacking around it.

To deploy to different environments, one must either do a switch or define an extra task to set the necessary variables. It’s a bit of code duplication but not really very bad. It gets a little bit worse when you also would like to be able to deploy from a branch or tag. The smarter way, of course, would be to have a built-in feature for defining different deployment environments, I don’t know anybody who has only one environment.

Where it starts to get annoying is when you want to deploy similar things that have some small difference between them, for example a debug version of a website as well as staging and production. Staging and production are just duplicating some settings and changing some paths, for debug currently you need to mark a server as debug and do something different for it, you end up setting a global variable and adding a branch anywhere that needs to check that code. For some reason using something like

task :stage do
  role :web, "staging.example.com"
end

task :dev do
  role :web, "debug.example.com", :debug => true
end

task :set_debug, :only => { :debug => true } do
  set :debug, "1"
end

namespace :deploy do
  task :before_symlink do
    set_debug

    p debug # WRONG: always prints "1"
  end
end

doesn’t work because it is always called regardless of this “only” business, so you have to resort to branching and global variables set at the top. Even if somebody offers up a solution or workaround to that particular bug the issue remains that one should simply be able to do something based on properties of the current host rather than having to define a sub-task to filter actions.

Furthermore, the /real/ annoyance begins when you have to deploy differing environments from the same code in the same deployment. Due to the same reasons as above I cannot do something with an intention like

role :front, "front.example.com", :readonly => true
role :web, "web.example.com"

task :set_debug, :only => { :debug => true } do
  set :readonly, "1" # this will fail because it sets it for both
end

namespace :deploy do
  task :before_symlink do
    set_readonly

    p readonly # WRONG: always prints "1"
  end
end

without resorting to environment variable hackery based on role or hostname checking.

Obviously what we all want from a deployment tool is a way to declaratively define the servers and their attributes and capistrano just falls flat on its face when faced with any of these very average deployment scenarios.

Update Hmm, Vlad the Deployer sounds nice

Added some client code to this beast so that I can start testing interop with other servers, the only site claiming support at the moment is Ma.gnolia but I can’t get it to work yet (thread). The client is part of the example code too: http://term.ie/oauth/example/client.php I’ve only tested it against my own stuff so far because I can’t get any debug info out of Ma.gnolia yet, hopefully they help out tomorrow :)

Tonight seems to OAuth night, Ma.gnolia just released their endpoint, Twitter is planning on updating theirs, I’m working on getting Jaiku’s into place and I just released my Public OAuth Test Server so go hammer it and help me fix the bugs :)

I’ll be working on stuff for the next couple hours if anybody wants to add andyster on gtalk to their buddy list and co-hack. You’re also welcome to come to my house if you are in SF and near the mission, ping me online.

I really feel like I looked everywhere and never found a decent how to on getting Ganglia set up on a pretty simple system and had to examine other people’s configs and just make guesses until I came up with something useful.

First a quick overview of the architecture and then we’ll walk through it with config examples.

Overview

Monitoring Server

This is the server where we are going to be aggregating the data, in the example this is a development server. It not only aggregates the data from other nodes but provides data for itself as well and as such it must run both gmetad and gmond.

DB Server and Web Server

Both of these servers are only reporting data for themselves and as such only need to run gmond.

gmetad

A little bit of a weird naming scheme Ganglia has, basically gmetad can be thought of as the “server” in an architecture like the above, it’s purpose is to periodically query a list of hosts (“clusters”) and aggregate their data using RRDTool and make pretty graphs.

/* Example section of gmetad config for the Monitoring Server */
data_source "monitor" localhost
data_source "db"    192.168.0.10
data_source "web"   192.168.0.11

gmond

This is sort of like the client in this architecture, it runs on every host you want to track. It is designed to be used for clusters and as such all the examples you see use multicast and expect there to be a bunch of hosts all sharing data. In this example, however, we are treating it like a single node that only provides the tracking data for host, itself.

This all seems straightforward until you look at the config and see “udpsend” and “udprecv” option, my first thought was that since I wasn’t talking to other servers I could just comment these directives out but that was wrong, they actually should be configured to send and receive from themselves.

/* Example section of the gmond config for all the servers */
/* Feel free to specify as many udp_send_channels as you like, 
   they specify who you push data out to, in this case the
   only one specific is to push to itself at localhost */
udp_send_channel {
   host = 127.0.0.1
   port = 8649
}

/* You can specify as many udp_recv_channels as you like as well,
   they specify the channel that you will listen for data on, in
   this case the only one specified is for localhost */
udp_recv_channel {
   port = 8649
   bind = 127.0.0.1
}

/* These are the IPs allowed to query this server's current state */ 
tcp_accept_channel {
  port = 8649
  acl {
    default = "deny"
    access {
        ip = 192.168.0.2 /* the ip of Monitoring Server */
        mask = 32
        action = "allow"
    }
    access {
        ip = 127.0.0.1
        mask = 32
        action = "allow"
    }
 }
}

One small issue with this example config is that servers will report themselves at localhost, this can probably be dealt with by using the public interface for bind and host above, but that may also require setting up some additional firewall rules.

Thought I’d never have to write libraries for it -hat tip sxip- but I guess everybody has to get over losing at some point.

The following is the product of about 6 hours in the middle of the night staring at various OpenID libraries (mostly the JanRain ones) and attempting to squeeze the code into Twisted’s asynchronous magicland. Nothing super fancy yet, I haven’t even tested the “verifyResponse” method yet, but I think it is probably a decent reference for somebody looking at integrating OpenID but stuck on the ridiculousness of the spec, the libraries and the various extensions they require to compute big numbers you’ve never heard of.

The code is in my devjavu, requires the JanRain OpenID python library (2.x) and Twisted (2.5, I guess)

Also, thanks to intertwingly for this code that gave me a place to start tearing apart JanRain’s super-deluxe-sized library.

My first Greasemonkey script!

Copied basically completely from Last.fm with Allmusic, this shows a link next to artists that will take you to an oink search for that artist.

You can get it at userscripts and in my greasemonkey section on Devjavu

And now, back to our regularly scheduled program…

This is a library using the Twisted framework that gives you a nice little service integrating with the Spread Toolkit. Also includes ParallelSpreader which is a basic concept for creating services around spread that run in parallel for some small distributed computing / redundancy tasks.

I guess it is no even a proper release because I’m just going to point you at spreader in my trac svn repo.

ParallelSpreader is expecting to be using SAFE_MESS level for everything.

Through a series of unfortunate events I ended up on a new server with a broken Drupal installation (I had an old version installed, but the new server had a high PHP version requirement and certain extensions and so on) that I just don't have the time to dig through and fix, maybe in the future.

Anyway, if you got a url pointing here from somewhere else that you didn't find, you might want to try replacing "devdev" with "devdev.old" in the url and trying again. The site is still there but is now full of spam and you won't be able to log in.