You might have noticed that I'm running a little Raspberry Pi, acting as a server for my website as well as some other small server-ish tasks. This machine is actually on my home network and I also use it as the front-face to that network for incoming connections. There are other machines on this network, and while they are behind a NAT and so not addressible from the outside world, this is fine most of the time. But on the odd occasion where I'd like to directly address any other machine on that network, I have to do so through the Raspberry Pi. Depending on what it is I'm trying to do, exactly, that can be tricky.

I've just discovered sshuttle. It acts similarly to a VPN, using SSH under the hood to transport TCP packets through a server that you specify. The cool thing is that it doesn't require any complicated pre-configuration on the server—just Python. All you have to do is run it on the client machine you want to connect from. Nifty!

Debian has it packaged: run sudo aptitude install sshuttle. The binary itself apparently installs under /usr/sbin, which doesn't seem to be in a [...]

Read this post

Recent events gave me cause to consider my personal data backup and redundancy strategy for my Debian installs. Or, more accurately, it caused me to amend my half-baked and semi-implemented existing approach so that I won't lose data or have to reconfigure things from memory/scratch in the event of a hard disk failure.

My present “backup” approach is really somewhere between a time-limited backup and redundant storage. Essentially, I use Unison to synchronize my home folder (with certain sub-folders ignored, e.g. certain git repositories, config and thumbnail cache folders) between my desktop and my netbook. I have to run Unison manually, so I end up synchronizing my data every week, give or take. This effectively kills two birds with one stone: I get to have local copies of my important data as up-to-date as my last sync for when I'm on the road and using my netbook, and the delayed redundancy gives me some protection should anything go wrong with either hard drive.

I'll be clear: what I'm doing here is something not-quite a backup (safe storage of historical editions of files) and not quite proper redundancy (up-to-date duplication of data). The frequency of synchronization is the key [...]

Read this post

I recently upgraded my home desktop's hard drive, because the old one was getting a bit full. Googling for instructions about how to transfer an existing system onto a new drive, many posts suggest using the cpio command, and that's what I tried. While this command does the job for the most part, there is one caveat which I encountered that makes cpio not the ideal tool to use.

Don't use cpio to clone filesystems. Why? Because GNU cpio doesn't support access control lists (ACLs) or extended attributes (xattrs).

Using cpio will end up a little screwy in some edge cases because of this. The particular case I ran into involved folders in /media that are managed by udisks2. udisks2 creates personal mount folders under /media with tailored ACLs to help properly manage permissions for permission-capable filesystems mounted by regular users. If you already have one of these personal mount folders, cpio will break it, and you'll lose permission to view any filesystems that you mount under it henceforth.

A better solution than cpio: rsync -caHAX src/ dest/

(Edit: Add -v to also see progress.)

As a side-note: if applicable, don't forget to run sudo dpkg-reconfigure grub-pc and set the [...]

Read this post

Lately at work I've been doing some data analysis in LibreOffice Calc that requires the binary entropy function. The function itself looks like H₂(x) =  − xlog₂(x) − (1 − x)log₂(1 − x), where 0log₂(0) is taken to be 0. It's this latter point that makes things a little tricky. LibreOffice Calc doesn't have this function built-in, sadly, and you have to explicitly guard for the case where x is 0 or 1, which is not easy to pull off inside a cell.

So I wrote a basic macro that implements it:

Function BINENT(x)
    If x = 0 Or x = 1 Then
        BINENT = 0
    ElseIf x > 0 And x < 1 Then
        BINENT = -(x*Log(x) + (1 - x)*Log(1 - x))/Log(2)
    Else
        BINENT = Null
    End If
End Function

To be able to use BINENT in Calc, go to the “Tools” menu, “Macros”, “Organize Macros”, and select “LibreOffice Basic...”. You'll be shown a window with a library hierarchy on the left. Under the “My Macros” collection, select the “Standard” library, then click the “New” button on the right. This will open up the editor. Copy and paste the BINENT code above into the editor window [...]

Read this post

A few of the interests I have and projects I follow use Google Groups to keep in touch. Essentially, these are just mailing lists that Google maintains. At any rate, while it's possible to subscribe to a group without having a Google account by sending an email to (group name)+subscribe@googlegroups.com, if you do have a Google account and use GMail with aliases, it can be tricky to ensure that you subscribe using your preferred address. This is because Google, in its infinite wisdom, will subscribe your GMail address if it detects you attempting to subscribe under any alias that's associated with your GMail account. That has the side-effect that you won't be able to post to the group with anything but your GMail address.

Luckily there's an alternative subscription mechanism (which I spotted here). If you instead use the Google Groups web interface, Google won't have the opportunity to inspect your email headers and determine if you're using a GMail alias. So, you will end up with your preferred address subscribed to the group, instead of your GMail address.

The URL you need is https://groups.google.com/forum/#!forum/(group name)/join where (group name) is [...]

Read this post

Sometimes, as I have, you'll find yourself working with colleagues who, through no fault of their own, are either not acquainted with the etiquette of Subversion repository use, or simply have an accident. What you may then end up with is a repository that contains one or more giant blobs of useless data that, really, should never have been added in the first place. Whether or not the culprit well-intentionedly removes these giant blobs in subsequent revisions, you're still left with a huge chunk of nothing-much wasting space on your server's hard drive.

Though a long-standing item on Subversion's wishlist, there is no command that will simply obliterate files from the repository's history. Nevertheless, there is a way to achieve this. Here's how.

The first step of the process is to determine which files need to go. (Some snippets in the following are derived from StackOverflow and Christosoft blog.) First, find the size of each revision in the repository. Replacing REPO appropriately, run this on your server:

REPO=[/absolute/path/to/repo]
for r in `svn log -q file://$REPO | grep ^r | cut -d ' ' -f 1 | tr -d r`
do
    echo "revision $r is " `svn diff -c $r file://$REPO [...]

Read this post