And the Pièce de résistance in the series of Dr. Horrible Act I lyrics and chords/tabs: A Man's Gotta Do, What a Man's Gotta Do. ([wikipedia](http://en.wikipedia.org/wiki/Dr._Horrible's_Sing-Along_Blog) suggests that [this page](http://en.wikipedia.org/wiki/John_Wayne) is related)
If you've not seen and enjoyed the newest Joss Whedon creation [Dr. Horrible](http://drhorrible.com/), you should certainly check it out. What can be better than a musical with Neil Patrick Harris? I've spent a bit of time to write out the lyrics and guitar chords/tabs to the first song that appears in Act I which I would imagine is entitled: "With My Freeze Ray" The chords aren't by any means perfect, but if you fake like I do on the piano you'll have no problem convincing people you're right.
Intro -- 4 beats to each chord, right hand stays positioned on G for the first 3 chords
In [part 1](/posts/linode-api-python-part-1) I discussed the creation of the base api, in [part 2](/linode-api-python-part-2) the creation of the interactive shell, and now in part 3 the creation of the previously mentioned command line driver. To wet your whistle here's how you could create a specific A record query it and update it
Here are the goals I had for the command line driver, in no particular order:
* dynamically generated list of valid actions (without ugly hacks that depend on dir and sanitizing names)
* dynamically generated list of valid parameters
* useful getopt style parsing
* dynamically generated help messages
There are some fundamental changes to the base api to achieve this. First, I removed `__api_required`, I knew from the get go that this functionality should have been in `__api_request` there was no need to chain them. Second, every argument listed in the api is now considered to be required. This second change seems like quite a drastic move, the rationale is this:
* when adding a new api method you only want to have to define parameters once (this is to fulfill goal 2 on above list)
* when adding parameters the results should be obvious and not have arbitrary notation/syntax indicating optional from required
* even if a parameter is listed as optional not including it will reset that field to the default value (but you may think you're preserving the previous value).
Now todays trickery/hackery, you have to be able to discern between when a class is compiled and decorated versus when decorators are invoked at runtime. Last night when I started the refactoring I initially put the population of the commands and parameter lists inside the wrapper method. But this will only be reached when the method is actually invoked (of course). However, a method is decorated at compile time, so my work around was to create a class ApiInfo with class variables, so long as ApiInfo is compiled before Api I can add populate the parameter and command lists.
Here's what I end up being able to do:
Shnazzy, plenty of room for improvement of course. Being able to pull out a short help string from `__doc__` for each valid command would be pretty, and making --help from the command line look to see if an action is defined and then display the whole `__doc__` from that method. Also there's no reason that [irgeek's high-level-commands](http://git.irgeek.com/cgi-bin/gitweb.cgi?p=linode;a=shortlog;h=refs/heads/high-level-commands) can't be added to the right places in ApiInfo to be populated, so you'll be able to get his recent method additions such as domainResourceUpdate which behaves more like you'd want if you're planning on doing dyndns with linode.
As usual the code is available for you to `git clone git://github.com/tjfontaine/linode-python.git` and for you to browse at [http://github.com/tjfontaine/linode-python](http://github.com/tjfontaine/linode-python)
Some of us are CLI junkies, and would prefer their hands never have to leave the keyboard to perform tasks. With that in mind I did a quick work up of an interactive shell for the API. One of the goals I had for the bindings was to introduce as few dependencies outside the core python libraries as possible, so the interactive console logically inherits from code.InteractiveConsole, which is for the most part a modified version of HistoryConsole from [this example](http://docs.python.org/lib/readline-example.html). The example claims that the console (presumably when used with .interact()) would have tab complete and history enabled. Much to my dismay and wasted time debugging, this is not the case. The reality is the console is sufficiently sandboxed (my guess) as to not inherit what you may setup of readline by the time you need it (as far as I could tell). The solution I came up with was to force my will upon the new interactive shell before the user enters it.
There. Now everything seems kosher, vars, modules, methods and what not will tab complete (finally). Geeze, why didn't someone just tell me that in the first place?! Ok, but almost certainly you're going to want to modify the completion routine to return only what you see fit:
This kludgy code looks for 'private' like results and prevents them from being returned when you press tab, when properly motivated I'm sure I'll prettify that such that only the Api class is kludged with. There really isn't much more to the code that should be interesting, I make sure that a method pp is available to you that wraps print simplejson.dumps and your default object for accessing the api is called 'linode'. On startup the shell looks for the environment variable LINODE_API_KEY if it's not set it will ask you to input your api key.
So what you end up with (after pressing tab twice):
and you can do normal python-y things:
[James Sinclair (irgeek in irc parlance)](http://git.irgeek.com/cgi-bin/gitweb.cgi?p=linode;a=summary) has been helping out, and in his branch you'll find interesting additions including some potential caching and more high level operations like searching through your domains for records with specific parameters (e.g. "Type='MX', Priority=5) pretty neat stuff, I'm sure this kind of stuff will eventually make it upstream and into the interactive shell.
The next step for the shell script is to have a means to execute actions without firing up the interactive portion:
`linode-shell --action=domainResourceSave --ResourceID=54 --Name=mail --Target=18.104.22.168 --Type=A`
Ideally whatever the shell inherits from, base api or higher level, it should all just work the same and handle parameters properly. But if you do have the higher level one you could have a nice action domainResourceUpdate that could lookup by FQDN (which performs some aggregate of domainList and domainResourceList). Hell, we can probably have an rc file that you can stick custom defined functions in as well
and then ... profit!
As usual the code is available from my git repo
`git clone git://github.com/tjfontaine/linode-python.git`
and viewable at
The great [Linode](http://linode.com) is preparing an api for customers to manipulate their account and twiddle their dns zone settings etc. Apparently I was supposed to write the [Python](http://python.org) bindings, the reality is somewhere between volunteer and victim I suspect. So I spent a little bit time working up an example how they might work.
A little background on the API, it is generously simple, you can submit via GET or POST, send your api_key in HTTP auth or in the GET/POST, and you can get results in JSON/WDDX/Human readable formats. It's still a bit in flux as far as what methods are available etc. The bindings are deliciously simple (if I do say so myself), however they rely on a certain amount of evil to the casual python fan. When designing the python bindings I wanted a way to quickly add and remove api methods (re: flux), but also minimize the copy and paste effort required. The flow for a request boils down to create an http request, determine the action (the api method), append necessary state vars, submit, read response, return json decoded result. The system I've devised uses the powerful python decorators to achieve this.
Here's an example of how a method is defined in the api:
Couple of notes about the above method definition:
The privateness of the decorator relates to another hack utilized elsewhere to determine the list of available api methods quickly (the reason for this will be in the follow up post)
Because there's no other real instructions to this method the doc string declaration is important so as not to annoy the python block definitions (stupid whitespace idiocy [I'm more of a ruby lover at heart])
You may ask, how is it exactly that you achieve such simple definitions of methods? Python decorators of course! Decorators are a way of manipulating a function/method before the actual function/method is invoked. Let's look at the definition of \__api_request
You'll notice that the input to the method \__api_request is the method we're decorating, and what's returned is a separate callable method that we've defined dynamically. Oh the wonder of it all! There's more than just your normal bit of trickery going on here, if you wanted to determine the number of arguments that were passed to domainList() you'd change the parameters for "def decorator", and the number and types of parameters don't even have to match what we defined as domainList() anyway. In fact, we defined domainList initially to take self and request, however we called it with no parameters. That's why my decorator function uses `*__args` and `**__kw`, it's a heavy handed var_args for python.
The rest of the decorator function relates to how the internals of generating an api request work, it translates `func.__name__` into the action to be performed, takes any named parameters `[method(ParamName=foo)]` adds them the dictionary (it also looks for 1 dictionary argument that may or may not be present and adds any of those parameters to the request, the reason for this will be discussed in a later blog). The decorator then calls the original method definition, checks to see if anything was returned, and then calls the base `__send_message` and returns that result to the callee. Phew!
The reason the decorator calls the original method is to allow for some input sanitization and other sanity checks before allowing the request to go out. You could add default values or check for more complex relationships between required parameters. Speaking of required parameters, you can also use another decorator to achieve that:
Checks the named parameter list (and the odd regular argument) for the required parameters and raises an exception if they're not found, pretty snazzy. To use it:
If you're interested in seeing the rest of the code you can visit [http://github.com/tjfontaine/linode-python](http://github.com/tjfontaine/linode-python) or clone it yourself with `git clone git://github.com/tjfontaine/linode-python.git`
I learned about using python decorators by reading [this](http://www.ddj.com/web-development/184406073) Dr. Dobbs article, and by looking at more [examples](http://wiki.python.org/moin/PythonDecoratorLibrary).
From my forum [post](http://www.linode.com/forums/viewtopic.php?t=3258):
> I'm using a secondary L360 for a few specific tasks, one of which is clamav.
> I figure others can benefit from this. The service is free and I make no
> guarantees as to uptime, speed, or catching every virus. I will keep it
> running the latest Debian has to offer (at the time of this writing 0.93~dfsg-1)
> with freshclam keeping the definitions up to date. If you do opt to use the
> service for your MTA/Spam/AV solution it behooves you to make sure you fall
> back to some other form of scanning should it not be available. (This is
> your responsibility!) The service is located at snafo-local.atxconsulting.com
> (which resolves only to a dallas private ip) on port 3310. ClamAV uses separate
> ports to handle data transfer that are negotiated though the initial connection,
> the port range my service uses is not the default instead it is 11024 to 12048.
> Make sure your firewall allows this communication. Should you have any questions
> I'm easiest to reach on oftc irc as tjfontaine I also lurk in #linode (where you
> should be as well) Donations to subsidize the effort are always welcome And last
> but not least thanks to Linode for their spectacular service!
I love my [Linode(s)](http://linode.com), I have 2 personal and 1 for $Employer;
sometimes a GoodThing(tm) is just not enough. I've been getting a few more
requests to do hosting and I don't like to overload my boxen, that's what
prompted the increase of personal linodes. One of the tasks I split off was
clamav since it can get memory hungry pretty fast. If you have a linode in the
Dallas/ThePlanet DC with private IP enabled you can and probably should take
advantage of not having to run clamd on your own.