devdev2050

14:59 < termie> you know what is broken for me? java as a language
14:59 < verbal> just for you? :)
14:59 < termie> a lot of other people seem to be using some kind of workaround
15:00 < verbal> haha

Gave a bit of a explanation about how JaikuEngine intends to use OAuth via an XML-RPC interface over on Jaiku so I figured I’d mostly duplicate it here.

Quick background:

JaikuEngine expects the request coming in over XML-RPC to be in the form of a single parameter which is a struct of strings (emulated in python as a dictionary of strings)

First off using XML-RPC obviously isn’t covered in the spec, the spec only talks about headers and POST and GET, what we are doing however is following the spec for signing a set of request parameters.

Step 1: Get the parameters.

The spec describes three sources for parameters, the header, POST application/x-www-form-urlencoded and GET query string. The reason it specifies a content type there is to explicitly say this is data expected as key-value pairs, no mime attached files, no atompub, only things that are made of the same kind of key-values as one would expect in a query string.

In our case we choose to also allow those key-value pairs to come in via xml-rpc. When you boil it all down, what you are trying to sign is the important part of the request, in atompub the organization of the actual xml can have significance, tags will have attributes, etc; however, in xml-rpc (when used the way we are using it) the “information,” the important part of the request, is just the key-value pairs, we completely ignore all that extra xml that makes up the xml-rpc protocol: all we want is the content.

Step 2: Normalize the parameters

Once you have those key-value pairs you need to apply predictable and repeatable transformations to it to make sure that each key and value looks exactly the same as it does to the person who sent them, in the case of oauth that is a specific type of urlencoding.

Regardless of where you got the variables from, be it POST, GET, headers, xml-rpc, you will have a key and a value and your web app will have successfully decoded them from whatever format they were in to the actual values of the key and the value without any escaping, that is inherently necessary if you plan on using those values to make changes to your site. From there it is a simple matter of allowing the OAuth library to normalize them for use in the signature.

When writing the OAuth libraries, I explicitly included the option of passing a parameters dict or associative array into the request constructors so that the libraries can be transport protocol agnostic.

Step 3 and onward:

I’m pretty confident you’ve got the rest from there, it plays out according to the spec without any hitches.

Merged in a bunch of patches from fangel and added support for RSA-SHA1 to the library.

There are still some gotches related to keys, they are marked in the code but for the moment you’ll have to extend the OAuthSignatureMethodRSASHA1 class in order to use it because the method for fetching your public and private keys is being skipped.

You can go test using the test client and test server

The keys being used are down at the bottom when RSA-SHA1 is selected as the current signing method.

As always, code is available at http://code.google.com/p/oauth/source/browse/code/php

Wee, some of the code I worked on has been released (and there’s a blog post about it), it’s an SDK that lets you use the Django development process with App Engine.

Works really well already and there is an active google code project where you can report bugs and get your patches reviewed.

Seems like every time I try to make use of it I am hacking around it.

To deploy to different environments, one must either do a switch or define an extra task to set the necessary variables. It’s a bit of code duplication but not really very bad. It gets a little bit worse when you also would like to be able to deploy from a branch or tag. The smarter way, of course, would be to have a built-in feature for defining different deployment environments, I don’t know anybody who has only one environment.

Where it starts to get annoying is when you want to deploy similar things that have some small difference between them, for example a debug version of a website as well as staging and production. Staging and production are just duplicating some settings and changing some paths, for debug currently you need to mark a server as debug and do something different for it, you end up setting a global variable and adding a branch anywhere that needs to check that code. For some reason using something like

task :stage do
  role :web, "staging.example.com"
end

task :dev do
  role :web, "debug.example.com", :debug => true
end

task :set_debug, :only => { :debug => true } do
  set :debug, "1"
end

namespace :deploy do
  task :before_symlink do
    set_debug

    p debug # WRONG: always prints "1"
  end
end

doesn’t work because it is always called regardless of this “only” business, so you have to resort to branching and global variables set at the top. Even if somebody offers up a solution or workaround to that particular bug the issue remains that one should simply be able to do something based on properties of the current host rather than having to define a sub-task to filter actions.

Furthermore, the /real/ annoyance begins when you have to deploy differing environments from the same code in the same deployment. Due to the same reasons as above I cannot do something with an intention like

role :front, "front.example.com", :readonly => true
role :web, "web.example.com"

task :set_debug, :only => { :debug => true } do
  set :readonly, "1" # this will fail because it sets it for both
end

namespace :deploy do
  task :before_symlink do
    set_readonly

    p readonly # WRONG: always prints "1"
  end
end

without resorting to environment variable hackery based on role or hostname checking.

Obviously what we all want from a deployment tool is a way to declaratively define the servers and their attributes and capistrano just falls flat on its face when faced with any of these very average deployment scenarios.

Update Hmm, Vlad the Deployer sounds nice

Added some client code to this beast so that I can start testing interop with other servers, the only site claiming support at the moment is Ma.gnolia but I can’t get it to work yet (thread). The client is part of the example code too: http://term.ie/oauth/example/client.php I’ve only tested it against my own stuff so far because I can’t get any debug info out of Ma.gnolia yet, hopefully they help out tomorrow :)

Tonight seems to OAuth night, Ma.gnolia just released their endpoint, Twitter is planning on updating theirs, I’m working on getting Jaiku’s into place and I just released my Public OAuth Test Server so go hammer it and help me fix the bugs :)

I’ll be working on stuff for the next couple hours if anybody wants to add andyster on gtalk to their buddy list and co-hack. You’re also welcome to come to my house if you are in SF and near the mission, ping me online.

I really feel like I looked everywhere and never found a decent how to on getting Ganglia set up on a pretty simple system and had to examine other people’s configs and just make guesses until I came up with something useful.

First a quick overview of the architecture and then we’ll walk through it with config examples.

Overview

Monitoring Server

This is the server where we are going to be aggregating the data, in the example this is a development server. It not only aggregates the data from other nodes but provides data for itself as well and as such it must run both gmetad and gmond.

DB Server and Web Server

Both of these servers are only reporting data for themselves and as such only need to run gmond.

gmetad

A little bit of a weird naming scheme Ganglia has, basically gmetad can be thought of as the “server” in an architecture like the above, it’s purpose is to periodically query a list of hosts (“clusters”) and aggregate their data using RRDTool and make pretty graphs.

/* Example section of gmetad config for the Monitoring Server */
data_source "monitor" localhost
data_source "db"    192.168.0.10
data_source "web"   192.168.0.11

gmond

This is sort of like the client in this architecture, it runs on every host you want to track. It is designed to be used for clusters and as such all the examples you see use multicast and expect there to be a bunch of hosts all sharing data. In this example, however, we are treating it like a single node that only provides the tracking data for host, itself.

This all seems straightforward until you look at the config and see “udpsend” and “udprecv” option, my first thought was that since I wasn’t talking to other servers I could just comment these directives out but that was wrong, they actually should be configured to send and receive from themselves.

/* Example section of the gmond config for all the servers */
/* Feel free to specify as many udp_send_channels as you like, 
   they specify who you push data out to, in this case the
   only one specific is to push to itself at localhost */
udp_send_channel {
   host = 127.0.0.1
   port = 8649
}

/* You can specify as many udp_recv_channels as you like as well,
   they specify the channel that you will listen for data on, in
   this case the only one specified is for localhost */
udp_recv_channel {
   port = 8649
   bind = 127.0.0.1
}

/* These are the IPs allowed to query this server's current state */ 
tcp_accept_channel {
  port = 8649
  acl {
    default = "deny"
    access {
        ip = 192.168.0.2 /* the ip of Monitoring Server */
        mask = 32
        action = "allow"
    }
    access {
        ip = 127.0.0.1
        mask = 32
        action = "allow"
    }
 }
}

One small issue with this example config is that servers will report themselves at localhost, this can probably be dealt with by using the public interface for bind and host above, but that may also require setting up some additional firewall rules.

Thought I’d never have to write libraries for it -hat tip sxip- but I guess everybody has to get over losing at some point.

The following is the product of about 6 hours in the middle of the night staring at various OpenID libraries (mostly the JanRain ones) and attempting to squeeze the code into Twisted’s asynchronous magicland. Nothing super fancy yet, I haven’t even tested the “verifyResponse” method yet, but I think it is probably a decent reference for somebody looking at integrating OpenID but stuck on the ridiculousness of the spec, the libraries and the various extensions they require to compute big numbers you’ve never heard of.

The code is in my devjavu, requires the JanRain OpenID python library (2.x) and Twisted (2.5, I guess)

Also, thanks to intertwingly for this code that gave me a place to start tearing apart JanRain’s super-deluxe-sized library.

My first Greasemonkey script!

Copied basically completely from Last.fm with Allmusic, this shows a link next to artists that will take you to an oink search for that artist.

You can get it at userscripts and in my greasemonkey section on Devjavu