Recently I was asked to explore means to monitor/audit network traffic of a virtual host (i.e. all traffic on the Dom0 and DomUs) without the monitoring software running inside the privileged domain (Dom0). If you are ok with running the monitoring software inside the privileged domain then you need not read the rest of this blog.
Normal network auditing software comes in the form of tcpdump, SNORT, or even a collection of software such as OSSIM. Most of this software gleans its information by setting a network interface into promiscuous mode and then using a hub or port mirroring on your switch to duplicate all packets to that network interface for auditing. Hubs are used in lieu of switches because they operate merely as repeaters, where as a switch tries to limit network congestion by sending packets to only the port that a given destination resides behind. This same switching technology is employed by the bridge interface in Linux and most other virtualization platforms.
The most common way to achieve network auditing inside of a virtual machine guest (DomU) is to assign the physical interfaces from the bare metal to the guest (pci mapping) such that the guest sees the physical hardware, thus bypassing the bridge normally used at the Host/Hypervisor level. There are a few cons to this approach. In this deployment you can't just add hardware and use port mirroring on your switch to capture all traffic from the Host. This will only capture traffic from sources and to destinations external to the host you're monitoring, and won't capture traffic among the guests running on your host. To get around that obstacle you could use this guest then as the route of all traffic to and from and among the host and virtual machine guests. However you're then relying on a specific guest to stay available to handle the traffic and audit, and if that guest dies all other guests on the host will be without network connectivity. Some permutation of this solution is probably acceptable if you're simply interested in monitoring traffic to and from your gateway where rogue traffic is more likely to initiate.
Another possible solution is to "simply" duplicate all packets in the privileged domain and send them "over the wire" to another virtual machine for auditing. The solution is what I'll dub virtual port mirroring, and can be achieved with iptables (or probably more suitably ebtables) and extensions from [http://xtables-addons.sourceforge.net/](http://xtables-addons.sourceforge.net/)
`iptables -t mangle -A PREROUTING -j TEE --gateway 192.168.0.100`
You'll probably want to make sure this rule is at the top of your PREROUTING chain and you're likely going to want to make the rule considerably more specific (not the least of which is preventing a feedback loop from the destination because of its own traffic). For instance, on the network I intend to deploy this on there's a backend network handling DRBD replication, those are probably packets you're not going to want to duplicate unless you're extra paranoid. But this solution is simple, straight forward, and if I do say so myself elegant. Traffic in and out of the host and among the guests are all duplicated and sent to a specific destination and if that destination is down the packets merely drop. I still need to perform some stress testing to see just how far you can push the network stack before it falls over, but it's certainly advantageous to keep the destination on the same host and not push that traffic external for security and network congestion related issues.
Note that this will not audit the rest of the traffic on your presumably switched network. You'll want to devise a means to audit on each of your physical hosts and then collect that data in a central location later. I performed all this on the Xen stack provided from Debian in Lenny and unfortunately xtables-addons is only a Squeeze package right now (because it deps on iptables >= 1.4.3) so I ended up building these packages on my own. If there are others out there who would like to implement a similar solution in their Lenny stacks and don't want to build the packages leave a comment and I'll create a repo for the packages on i386 and amd64 platforms.
I've been in a Muppet mood recently, as such I've found myself playing along with soundtracks. I was quite unsatisfied with the chords I found for Gonzo's lament from The Muppet Movie so here is my transcription, the end of the bridge may not be 100% but it doesn't sound terrible
Not long ago I wrote an imap polling script for work that used Python's [imaplib](http://docs.python.org/library/imaplib.html). Shortly after that [HoopyCat](http://blog.hoopycat.com/) wrote an excellent [imap backup script](http://blog.hoopycat.com/index.php/2009/07/04/imap2maildir-a-tool-for-mirroring-imap-t) that also used imaplib. We exchanged some stories and opined on wanting a simpler mechanism to get imaplib to return an [email.Message](http://docs.python.org/library/email.message.html) object (a feature that should be available from the core).
I happened to have to look at my code again this week and decided to work up a quick example of the interface I expected to actually find in the core imaplib
The SimpleImap and SimpleImapSSL classes should be drop in replacements for your existing usage of IMAP4 and IMAP4_SSL. You'll notice the use of generator objects (the yield keyword), this means that each iteration the message you're working on is pulled from the server right then. That's useful when you have a lot of messages that you don't necessarily want to cache into memory or when they'll potentially have large attachments. On the other hand, it will result in considerably more imap commands and traffic than if you were to just pull all the messages at once.
Here's a sample usage that prints out the subject of every message in your inbox:
For far too long now, you have been limited by those 140 characters of microblogging sites like twitter. I present you with twuuenc, take your tweet that is longer than 140 characters and stuff it into fewer unicode characters.
Take for instance the beginning of the [Gettysburg Address](http://en.wikipedia.org/wiki/Gettysburg_Address):
Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.
That's a total of 175 characters, twitter just won't have it. But if you run it through twuuenc you get:
That tweet is only 128 characters (130 with markers), there's still 12 characters left for you to insert a wise crack!
What's better is that this allows you send binary data over twitter, imagine the possibilities! Just for starters, let's add some additional compression to the address through Zlib
Now our tweet only takes up 93 characters (95 with markers), You have a full 47 characters to be clever!
You can also optionally include the markers around the message to signify that the following message should be twuuenc decoded. A message wrapped in ☹ means it is twuuenc encoded but not compressed; while a message wrapped in ☺ means it's encoded and compressed with zlib.
The alphabet twuuenc uses only has 2048 characters, but if you can get that up to 4096 that's another whole bit you can store per character in your tweet.
You can find the source for the encoding and decoding [here](http://git.atxconsulting.com/cgi-bin/gitweb.cgi?p=twuuenc;a=blob_plain;f=tw_uuencode.py;hb=HEAD) with an [MIT](http://www.opensource.org/licenses/mit-license.php) license. The code relies on [http://code.google.com/p/python-bitstring/](http://code.google.com/p/python-bitstring/) which is also in the git repo (similarly licensed).
Now for the Act III tension builder where our favorite villain (anti-hero?) gets caught monologuing, these are the tabs for "Slipping"
I don't have much prowess for tabs as it is, and the darker/minor the song the more difficult it is for me to transcribe, lemme know of errors or clarifications you can spot
Act III the phenomenal finish to Dr Horrible starts off with this catchy ditty "So They Say", thanks again to all the folks at [http://whedonesque.org/](http://whedonesque.org) for starting me off with the lyrics
To finish off Act II here is "Brand New Day" from Dr. Horrible's Sing-Along Blog. Not the best I've done in the series, would probably sound better if I played guitar. Everything is mostly in fifths and not full chords I don't know if that equates to power chords or not. The slow section talking about Penny needs some more attention as well.
Act II's sentimental sexual-tension building Penny's Song has now been transcribed, it has a nice and easy "Seasons of Love"(Rent)/"What's going to happen"(Scrubs) feel to it. The opening chords are the same to "Will You Lend a Hand" sung by Penny in Act I, it wasn't long enough song for me to transcribe, but I was glad the theme was reprised in Act II there are certainly good elements of musical theater in this show.
Continuing our series of Dr. Horrible's Sing-Along Blog lyrics and chords/tabs I present the first song from Act II: On The Rise. I found some other like minded folks transcribing the lyrics at [http://whedonesque.org/viewtopic.php?t=1725](http://whedonesque.org/viewtopic.php?t=1725) they make getting the words right the first time easy. I've also decided to try the more inline approach of the dueling lyrics, hopefully this will make changes clearer.