Ataraxia Consulting

Giganews VyprVpn on Linux with IPSEC and L2TP

Sun, 01 May 2011 12:58:00 -0400

I'm not a fan of PPTP, but unfortunately that's the only listed configuration option for giganews' VpyVpn service (http://www.giganews.com/vyprvpn/setup/linux/pptp.html). So the following are a few configuration files you can use to connect to vyprvpn using ipsec and l2tp. I tested with Ubuntu 10.04, OpenSWAN, and xl2tpd.

The /etc/ipsec.conf stanza

conn giganews
        authby=secret
        pfs=no
        rekey=yes
        keyingtries=3
        type=transport
        left=%defaultroute
        leftprotoport=17/1701
        right=us1.vpn.giganews.com
        rightid=@us1.vpn.giganews.com
        rightprotoport=17/1701
        auto=add

The /etc/ipsec.secrets stanza

%any us1.vpn.giganews.com: PSK "thisisourkey"

The /etc/xl2tpd/xl2tpd.conf stanza, be sure to replace giganews_username with your username

[lac giganews]
lns = us1.vpn.giganews.com
require chap = yes
refuse pap = yes
require authentication = yes
; Name should be your giganews username
name = giganews_username
ppp debug = no
pppoptfile = /etc/ppp/options.l2tpd.client
length bit = yes

The /etc/ppp/chap-secrets stanza, be sure to replace giganews_username and giganews_password with your username and password respectively

giganews_username us1.vpn.giganews.com "giganews_password" *

The /etc/ppp/options.l2tpd.client file

ipcp-accept-local
ipcp-accept-remote
refuse-eap
noccp
noauth
crtscts
idle 1800
mtu 1410
mru 1410
defaultroute
debug
lock
#proxyarp
connect-delay 5000

You can replace us1.vpn.giganews.com with any of the following end points, just make sure you replace all instances in the previous

us1.vpn.giganews.com for Los Angeles, CA
us2.vpn.giganews.com for Washington, DC
eu1.vpn.giganews.com for Amsterdam
hk1.vpn.giganews.com for Hong Kong

To connect you run the following commands ipsec auto --up giganews when that's successful connect l2tp echo "c giganews" > /var/run/xl2tpd/l2tp-control

If that's successful ppp will have replaced your default route to go out over ppp0 which represents your vpn connection.

Most of the instructions adapted from http://www.jacco2.dds.nl/networking/linux-l2tp.html

Steve Earle -- Every Part Of Me

Fri, 29 Apr 2011 23:55:00 -0400

Steve Earle recently released a new album entitled "I'll Never Get Out of This World Alive" (the title of an old Hank Williams song). I'm particularly infatuated with this album. Mostly for its simplicity. All the songs were recorded live, with little overdubbing. Just reinforcing that music doesn't have to be complicated and layered to be enjoyable and moving. The song that represents this ethos the best is "Every Part Of Me", the following are the lyrics and chords as I interpret them. Here's a video with him performing it and a brief audio clip of me playing the two major themes.

Every Part Of Me
Steve Earle
I'll Never Leave This World Alive
C

[C] [G] [Am] [G] [C]
[C]I love you with all my heart
[G]all my soul [Am]every [G]part of me
[C]it's all I can do to mark
[G]where you end and [Am]where I [G]start you see

[Am]living long in [G/B]my travails
I [C]left a trail of [G/B]tears behind me
[Am]been in love so [G]many times
didn't [F]think this kind would [Am]ever [G]find me

[C]I love you with everything
[G]all my weakness [Am]all my [G]strength
[C]I can't promise anything
[G]except that my last [Am]breath will [G]bear your name

[Am]and when I'm gone they'll [G/B]sing a song
a[C]bout a lonely [G/B]fool who wandered
[Am]around the world and [G]back again
[F]but in the end he [Am]finally [G]found her

[C]I love you with all my heart
[G]all my soul and [Am]every [G]part of me
[C] [G] [Am] [G]
[C] [G] [Am] [G]

[Am]cross the univer[G/B]se I'll spin
un[C]til the ending and [G/B]then I wonder
[Am]if we should get a[G]nother chance
could [F]I have that dance
for[Am]ever [G]under

[C]a double moon and sky lit stars
[G]shining down on [Am]where you [G]are
[C]and I love you with all my heart
[G]all my soul and [Am]every [G]part of me
[C] [G] [Am] [G] [C]

Linsides - LinCached - 3rd Party Linode Service

Thu, 24 Mar 2011 19:01:00 -0400

Prior to today I had considered this site, while not terribly popular (or frequently updated), to be relatively quick with little effort on my part. I may not host many high traffic sites, but I have my personal sites and a handful of others. I have a server specifically to handle the HTTP traffic, and a server to handle RDBMS. They use the private network afforded to me by Linode.com (my VPS provider of choice) so communication between the two servers is quick and doesn't count against my monthly bandwidth quota. It's worked for years, so I've had little desire to muck with the formula.

That is until I learned about Linsides.com -- a company that offers services specifically for Linodes over the private network.

The offering is young, but the service is delivered with slick ease. They currently offer NTP, APT Caching, and LinCached (a memcached frontend). The services are only available over the private network, so that means you have to be a Linode.com customer before you can take advantage. NTP and APT caching are offered for free and are conveniences to provide fast responses and to keep load on public mirrors low. That is you get the same quality as if you were connecting to them publicly but they're generally delivered faster and don't count against your monthly bandwidth quota.

LinCached is a private network memcached instance, which you can configure to be one of the following sizes: 32MB, 64MB, 96MB, or 128MB. Linsides uses a prepaid credit system for managing payments. Each size memcache instance costs a certain amount of credits per day, a 32MB instance is 1 credit, 64MB are 2 credits and so on. When you sign up you get 5 free credits, so you can get a free trial for 5 days of a 32MB instance. There's a dashboard that lets you see how many instances you have, what their sizes are, and your current usage on that instance. If you drill down further you can even see a snazzy progress bar to that gives you a visual way to identify how much of your instance you're using. You can even quickly flush the specific instance.

After you've created your LinCached instance, you simply need to add the private IP address of the node that you want to grant access to and boom, you're done. All in all it took about 10 or 15 seconds of simple data entry on Linsides.com clean site to add a new instance, and it was instantly available to me. I made the necessary changes so this site would take advantage of memcache and just as instantly I started to see the usage appear on the Linsides dashboard.

Simple, dead easy. A perfect no hassle way for me to increase performance of my site in under 15 minutes (realistically under a minute).

Now I'm perfectly capable of running my own memcached instances. But what's key here is that it's not using memory on my web servers or my database server, and I didn't have to spin up yet another node to achieve that. Pricing is affordable as well, credits come in packages that range from $10 for 100, to $300 for 6K. They are also planning on offering more services in the future. I'm excited!

Linode.com and Linsides.com -- A match made in heaven!

Automatically generate AirPrint Avahi service files for CUPS printers

Sun, 21 Nov 2010 00:13:00 -0500

Last weekend I read Ryan Finnie's excellent article about getting CUPS printers to work with AirPrint (http://www.finnie.org/2010/11/13/airprint-and-linux/). I got a bit angry at Ubuntu/avahi/CUPS/Apple regarding some silliness involving the APIs that are used by CUPS internally for DNSSD announcement (like the fact it's broken in 10.04 and 10.10 because Apple changed APIs and the new API calls weren't packaged (yet?)). So after I finished ranting to myself I created the service file and boom, I could print from my iPhone.

Neat.

Sucks to have to create these .service files manually though, if only something could just talk to CUPS and spit out these files for me. So this weekend I decided to whip up a small script to do just that.

https://github.com/tjfontaine/airprint-generate

It's a small python script that can talk to a CUPS server (by default the local socket) and write out some xml suitable for use with avahi. It doesn't do much special, just grabs all configured printers that are marked shared and create files that when avahi exports them will make the printers visible from an iOS device. You are responsible to make sure your printer is configured properly in CUPS. You should make sure CUPS can send and print a test page, if it can't do that it's unlikely that the print you send it from your iOS device will work either. Your CUPS server should also have a working PDF filter, since most times that's what the iOS device will send.

Without any options, it will communicate with your local CUPS instance (or that is to say, will do what ever the cups client library will do by default, there may be environment variables at play here), after it learns about each printer it will generate an xml file AirPrint-[name of printer in CUPS].service, putting this file somewhere where avahi knows to load it will automagically make the printer available to iOS devices (the directory in my experience is /etc/avahi/services). You can also specify -p [PREFIX] if you aren't a fan of AirPrint-. There is also -d [TARGET DIRECTORY] if you wanted to specify the avahi services directory, if you supply this parameter all the xml files will be generated in that directory, otherwise they will be generated in the current working directory.

DNSSD has a limit of 255 characters for a txt-record entry, currently not all fields are verified for this, the one place where it is checked is the entry that generates the "pdl=" record, this is the hint record that specifies what content-types the printer will accept. There is an internal priority list (in the future you will be able to influence this) that keeps important content-types at the head and experimental/unnecessary ones out all together. The resulting entry is truncated to fit into 255 (without also creating malformed entries). If you're curious to see what will be truncated make sure to run the script with the rather ugly --verbose option.

In the future (with proper motivation) I will add the other hint fields that include things like duplexing, but it wasn't immediately obvious to me which CUPS printer attributes store that information in a consistent way.

It would also be trivial to take this script and instead of generating avahi service files directly, use a python binding for dnssd/avahi/bonjour to do the announcements directly (at least as a stop gap until CUPS >= 1.4 + Debian (and derivatives) get the packaging solidified [and add airprint announcments]).

Linode API Python3 and GitHub

Thu, 05 Aug 2010 13:01:00 -0400

Josh Wright today contributed a Python3 branch of the API. I've pushed this to the repo and cherry picked a few of the commits that apply to master as well. I've also created a github repo that will serve as the main access point for the repo from here on out. Please if you have issues, file them. Also you hopefully will find the examples useful, I'll be moving those over to the wiki as well. If you want to contribute please don't hesitate, Josh already identified the need for unit tests. Thanks to Linode for being a great resource, and thanks to everyone who has used and/or contributed to the bindings!

Updated Linode API

Tue, 09 Feb 2010 16:39:00 -0500

I've updated the python bindings to support the new Linode StackScripts method calls. An excellent feature to aid in the deployment of your new nodes. The documentation is also up to date, albeit in need of some verbosity. You can browse the source at my gitweb, or as usual you can access the source directly with git git clone git://github.com/tjfontaine/linode-python.git

Monitor All Network Traffic On A Virtual Host Within A Virtual Machine On That Host

Sat, 12 Dec 2009 17:32:00 -0500

Recently I was asked to explore means to monitor/audit network traffic of a virtual host (i.e. all traffic on the Dom0 and DomUs) without the monitoring software running inside the privileged domain (Dom0). If you are ok with running the monitoring software inside the privileged domain then you need not read the rest of this blog.

Normal network auditing software comes in the form of tcpdump, SNORT, or even a collection of software such as OSSIM. Most of this software gleans its information by setting a network interface into promiscuous mode and then using a hub or port mirroring on your switch to duplicate all packets to that network interface for auditing. Hubs are used in lieu of switches because they operate merely as repeaters, where as a switch tries to limit network congestion by sending packets to only the port that a given destination resides behind. This same switching technology is employed by the bridge interface in Linux and most other virtualization platforms.

The most common way to achieve network auditing inside of a virtual machine guest (DomU) is to assign the physical interfaces from the bare metal to the guest (pci mapping) such that the guest sees the physical hardware, thus bypassing the bridge normally used at the Host/Hypervisor level. There are a few cons to this approach. In this deployment you can't just add hardware and use port mirroring on your switch to capture all traffic from the Host. This will only capture traffic from sources and to destinations external to the host you're monitoring, and won't capture traffic among the guests running on your host. To get around that obstacle you could use this guest then as the route of all traffic to and from and among the host and virtual machine guests. However you're then relying on a specific guest to stay available to handle the traffic and audit, and if that guest dies all other guests on the host will be without network connectivity. Some permutation of this solution is probably acceptable if you're simply interested in monitoring traffic to and from your gateway where rogue traffic is more likely to initiate.

Another possible solution is to "simply" duplicate all packets in the privileged domain and send them "over the wire" to another virtual machine for auditing. The solution is what I'll dub virtual port mirroring, and can be achieved with iptables (or probably more suitably ebtables) and extensions from http://xtables-addons.sourceforge.net/

iptables -t mangle -A PREROUTING -j TEE --gateway 192.168.0.100

You'll probably want to make sure this rule is at the top of your PREROUTING chain and you're likely going to want to make the rule considerably more specific (not the least of which is preventing a feedback loop from the destination because of its own traffic). For instance, on the network I intend to deploy this on there's a backend network handling DRBD replication, those are probably packets you're not going to want to duplicate unless you're extra paranoid. But this solution is simple, straight forward, and if I do say so myself elegant. Traffic in and out of the host and among the guests are all duplicated and sent to a specific destination and if that destination is down the packets merely drop. I still need to perform some stress testing to see just how far you can push the network stack before it falls over, but it's certainly advantageous to keep the destination on the same host and not push that traffic external for security and network congestion related issues.

Note that this will not audit the rest of the traffic on your presumably switched network. You'll want to devise a means to audit on each of your physical hosts and then collect that data in a central location later. I performed all this on the Xen stack provided from Debian in Lenny and unfortunately xtables-addons is only a Squeeze package right now (because it deps on iptables >= 1.4.3) so I ended up building these packages on my own. If there are others out there who would like to implement a similar solution in their Lenny stacks and don't want to build the packages leave a comment and I'll create a repo for the packages on i386 and amd64 platforms.

Muppet Movie -- Im Going to Go Back There Someday -- Guitar Tabs Chords

Sat, 05 Dec 2009 20:52:00 -0500

I've been in a Muppet mood recently, as such I've found myself playing along with soundtracks. I was quite unsatisfied with the chords I found for Gonzo's lament from The Muppet Movie so here is my transcription, the end of the bridge may not be 100% but it doesn't sound terrible

G          C   G     Em      A
This looks familiar, vaguely familiar,
G        C     G         Em          A
Almost unreal, yet, it's too soon to feel yet.
Am          D             Bm     E
Close to my soul, and yet so far away.
    Am          D              G
I'm going to go back there someday

G          C     G      Em            A
Sun rises, night falls, sometimes the sky calls.
G         C    G          Em     A
Is that a song there, and do I belong there?
Am         D               Bm         E
I've never been there, but I know the way.
    Am          D              G
I'm going to go back there someday.


Em          A             D
Come and go with me, it's more fun to share,
Em               A          D
We'll both be completely at home in midair.
      E           C#m         F#m         F#m/E
We're flyin', not walkin', on featherless wings.
       D         Am/C         Am        G/D    D
We can hold onto love like invisible strings.


G             C    G       Em                 A
There's not a word yet for old friends who've just met.
G            C     G        Em           A
Part heaven, part space, or have I found my place?
Am           D          Bm        E
You can just visit, but I plan to stay.
    Am          D              G
I'm going to go back there someday.
    Am          D              G
I'm going to go back there someday.

Making imaplib simple

Thu, 23 Jul 2009 17:06:00 -0400

Not long ago I wrote an imap polling script for work that used Python's imaplib. Shortly after that HoopyCat wrote an excellent imap backup script that also used imaplib. We exchanged some stories and opined on wanting a simpler mechanism to get imaplib to return an email.Message object (a feature that should be available from the core).

I happened to have to look at my code again this week and decided to work up a quick example of the interface I expected to actually find in the core imaplib

import imaplib
import email

class __simplebase:
  def get_messages_by_folder(self, folder, charset=None):
    ids = self.get_ids_by_folder(folder, charset=charset)

    for m in self.get_messages_by_ids(ids):
      yield m

  def get_ids_by_folder(self, folder, charset=None):
    self.select(folder)
    status, data = self.search(charset, 'ALL')
    if status != 'OK':
      raise Exception(data)

    return data[0].split()

  def get_messages_by_ids(self, ids):
    for i in ids:
      yield self.get_message_by_id(i)

  def get_message_by_id(self, id):
    status, data = self.fetch(id, '(RFC822)')

    if status != 'OK':
      raise Exception(data)

    return email.message_from_string(data[0][1])

class SimpleImap(imaplib.IMAP4, __simplebase):
  pass

class SimpleImapSSL(imaplib.IMAP4_SSL, __simplebase):
  pass

The SimpleImap and SimpleImapSSL classes should be drop in replacements for your existing usage of IMAP4 and IMAP4_SSL. You'll notice the use of generator objects (the yield keyword), this means that each iteration the message you're working on is pulled from the server right then. That's useful when you have a lot of messages that you don't necessarily want to cache into memory or when they'll potentially have large attachments. On the other hand, it will result in considerably more imap commands and traffic than if you were to just pull all the messages at once.

Here's a sample usage that prints out the subject of every message in your inbox:

c = SimpleImap('mail.example.com', 143)
c.login('someluser', 'stupersekrit')
for m in c.get_messages_by_folder('INBOX'):
  print m['subject']
c.logout()

twuuenc the uuencode of web2.0

Wed, 01 Jul 2009 19:34:00 -0400

For far too long now, you have been limited by those 140 characters of microblogging sites like twitter. I present you with twuuenc, take your tweet that is longer than 140 characters and stuff it into fewer unicode characters.

Take for instance the beginning of the Gettysburg Address: Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.

That's a total of 175 characters, twitter just won't have it. But if you run it through twuuenc you get:

ɹҺξɌǶᖜᘫᓿϧѶ✢Εǚᖣ¼Ј|ᓅɋöᖛӪ❡Ѯ9ᒡ✲ᐑᖨᙃ¥ѰҺΨᙦɖǞᗍʼО;◆ɌɺǪᙃ¦јѺĖ◐ΑȕөŤѐƢᘟ◐ýǪᗌᘋŇğ◆ᘕ˼ᖸᒋӽШѥÛᙵŶᖨ᙮ŰŇ{ᒧɋɸᕰᔋоҀƞ✲❼ȖᓫᓿѠᑺ◊◡ǹᖰᔌ❚ŇᓹĆ✮Pᖐᕣ°ЈѥĆ✒˘ǒᘫᕀҀƞ`ᘶýᕤᕤᗟ

That tweet is only 128 characters (130 with markers), there's still 12 characters left for you to insert a wise crack!

What's better is that this allows you send binary data over twitter, imagine the possibilities! Just for starters, let's add some additional compression to the address through Zlib

ңᙦ˥ᖐąᒟǐТ;ӤУᗔ○Õ¢Ñ◀ᓫӆᕪ✼ᙜ❉ӆʃ%ᗽᐉᘰҫᑻǧɪȰŠᖔᙱ¯ķ▭ᑜᔨɳĺᘗŐ◇ᔇђʗᑵËᘎᙁᑶʓǿ⟘nΗˀЧЏȜ➒⟘Ɲᖁʉᘧ➳ːȰ✩❈❢✜¨Ȁѣ➫ᗧңᔯᓞϓ❯ᒑȰ❰˒Ӝń

Now our tweet only takes up 93 characters (95 with markers), You have a full 47 characters to be clever!

You can also optionally include the markers around the message to signify that the following message should be twuuenc decoded. A message wrapped in ☹ means it is twuuenc encoded but not compressed; while a message wrapped in ☺ means it's encoded and compressed with zlib.

The alphabet twuuenc uses only has 2048 characters, but if you can get that up to 4096 that's another whole bit you can store per character in your tweet.

You can find the source for the encoding and decoding here with an MIT license. The code relies on http://code.google.com/p/python-bitstring/ which is also in the git repo (similarly licensed).