Listening to BBC Radio Shows on an MP3 player

In the name of bastard fuck, this was far fucking harder than it had any right to be.

You have Windows Media Centre installed on your PC. Either XP, or Vista or 7. Whatever. Fine. You record DVB-T1 broadcast TV programs and watch them quite happily. You record the odd radio program (also broadcast over DVB-T1), and you can play it back on the TV. But who wants to sit around the TV listening to radio programs, what do you think this is, the 1930's?

You have recorded an audio file. You want to listen to the audio on another device. How fucking difficult do you think this is going to be? Oh, boy you are in for a fucking treat.

So, how do we do this? I should interject here and say that there is a fairly straightforwards, albeit time-consuming, way to deal with this. You fire up audacity, change the input source to be the sound card audio out, then hit record and play whatever it is you want to record. The problem with this is that you are recording a compressed track so you are going to lose quality. In my view the BBC are the experts when it comes to digital audio compression and trasmission, and it is going to spoil whatever they broadcast if I recompress it. Its like sitting in the back row of the cinema with your digital video camera. You are not exactly going to walk out with a blue ray quality version of the film are you?

First thing. If using Vista or Windows 7, media centre spits out files with the extension .wtv. These are unreadable by anything but windows, so we need to convert them into a more common format. Helpfully if you right click on the file you will find an option to convert it to dvr-ms file.

What I would then ordinarily do is fire the file up in Video Redo. This is an excellent product for taking in a dvr-ms file, cutting out all the bits you do not want (ads and so forth) and then spitting out the original video and audio streams in an mpg file. You can then happily load that mpg file into avidemux, or your editor of choice and convert it to whatever format you desire.

However, Video Redo shits itself when it is given a dvr-ms file without any video. It just cannot handle it. The more recent versions of Video Redo may be different, but I am not paying for an upgrade for something this simple.

So we want to extract the audio in some other fashion. When something is difficult, then make it harder by doing it on the command line. If we want robust command line editing tools then we want Linux. I have tried ffmpeg under ubuntu with a command that looks a bit like this:

ffmpeg -i filename.dvr-ms -acodec copy -o filename.mp2

Doesn't fucking work. Apparently ffmpeg also cannot handle the earth shattering reality of a dvr-ms file without a fucking video stream.

How about:

mplayer -dumpaudio -dumpfile audio.mp2 filename.dvr-ms

No fucking joy. Cannot ever find an audio stream. It turns out that vlc is practically the only fucking thing on the planet that will spit the untouched audio out of a BBC Digital TV Broadcast Radio Recording. The command is this gem:

cvlc "[filename].dvr-ms" vlc://quit --sout '#transcode{vcodec=none}:duplicate{dst=std{access=file,mux=raw,dst="[filename].mp2"}}'

That's a fucking little beauty isn't it. Far more than you could ever realistically want to know about those options can be found here and here.

Acronis True Image Home 2011

Through several annoying, and a few heartbreaking, incidents I have learned well the IT lesson that "Data you do not have backed up is data you do not want". So I back stuff up. I back stuff up in a number of ways.

Firstly, I have an external drive from seagate (the clicky clicky special referred to in my post about encryption). It has a 5 year warranty, which is a good sign (and the clicky clicky sound has gone away). On that drive I mirror My Music, My Documents, My Videos, My Pictures and My Dropbox folders. I use Allway Sync for this, and it is pretty good. It is available as a "free" version, but is actually a nagware version, because after you have synced so many files it starts hassling you to buy the full version. It is, however, very intuitive to use. File synchronization software is not, by and large, written by people who have an innate talent for user interface design. When you run Allway Sync it pops up suggestion tip boxes telling you what to click and where to type to do a basic sync. You can completely ignore these if you like, and you will do so once you get to grips with it, but it is very welcome at the start.

With this backup method, what I end up with is a copy of all my data on an external disk. All I need to do is connect up the external drive, fire up Allway Sync and run the saved settings. The length of the backup just depends on how much stuff has changed since the last time I did a backup. It can be very very quick in deed. I am now into the habit of running one everytime I buy some new iTunes music, or upload some new digital photos from my camera. I treat all new material as not really existing until such time as I have run a sync.

It is also worth pointing out that you can set the software up to sync in one direction or both directions. Both directions is useful if you are moving the external drive between machines, where you may be adding data to it from both machines. The two way sync makes sure that every machine has the same data on it as the other machines. Ideal for sharing an iTunes library around the house. If running with a one way sync - just to back up stuff from one machine, you can also choose NOT to replicate deletions, so you keep a copy of all files on the external drive even if you delete them (accidentally!) on the main machine.

The second backup method I use is Dropbox. This allows you to select a folder on your machine which is then replicated on the Dropbox company's servers. If you install the software on another machine you get a copy of the folder on that machine as well. It works with Windows and Linux. I have it installed on every machine I use (netbook, HTPC, Desktop, and both of my work machines). That is a mixture of Ubuntu and Windows 7 machines. I get exactly what it says on the tin - one folder which is on all of those machines.

Dropbox, like AllwaySync, comes in a free version. You get 2Gb of space to use before you have to part with any cash. I just use the free version, and keep the folder contents to the really important stuff. Like, for instance, password safe databases, bitlocker recovery keys and so on. This backup method is very useful because it is off site storage. So if the house burns to the ground, I still have all the data in my dropbox folder. Also, if the dropbox company goes out of business I have all the data in the folder on 5 separate machines in three different buildings. Quite, quite, secure.

The third type of backup I do is to take an image of my system partition on various machines. If you are connected to the internet, you are going to get some sort of problem software installed at some point. Or you are going to change a stupid setting in the OS which is a pain to put right. Or you may just have a good old fashioned hardware failure. The BEST way of resolving the foregoing issues that I have found is not to bother with manuals, or anti-malware programs, but just to say "fuck it" and restore the last working image of the system partition. As a rough rule of thumb every gigabyte of data takes about 1 minute to restore. So if you install windows and a couple of bits of software you are only looking at 10-15 minutes to get back to a fresh working machine. You can EASILY spend that amount of time trying to get rid of whatever rootkit has infected your machine.

The software that I use for this task is True Image Home from Acronis. Unlike the other options I outlined above, this is not free (although there is a free trial). It does a couple of very useful things which made me spend money on it. Firstly, it lets you mount the archives it creates as virtual disks, so if you just want to grab a file or two out of it you can. Secondly, it lets you build a bootable USB stick so you can boot from that and use the software without having installed it. Go on, use Windows Backup to restore an image to a bare metal HDD. Not possible. You need to install Windows FIRST. Also the USB Key approach will happily back up and restore Linux partitions as well.

I had an older version of the software which I had to replace because it did not recognise my SATA connected hard disks. When browsing in the shop, I found two different versions of the 2011 software: a netbook "optimized" version which came on a USB key, and a Desktop version. The Desktop version cost £10 more. Now, I decided that I could put up with the inconvenience of using software designed for a netbook screen on a larger screen in exchange for a ready made USB key and the £10 price difference.

It turns out, however, that when Acronis say netbook "optimized" they actually mean netbook "crippled". Because you see (and it mentions this nowhere on the fucking box) it ONLY WORKS ON NETBOOKS. Yes, the software starts up on the desktop and says to itself, "hang the fuck on, this is not a netbook processor, I am going to go in a huff". The software was therefore completely fucking useless. Apart from on my netbook. Worked fine there.

So back to the shop. Explain predicament. Part with £10. Get full version. Go home. Install full version. Run a system disk backup.

Now, fast forward to last Saturday. I was downloading some hefty multipart files and the machine kept crashing. Odd, I thought. Possibly a hardware issue. I broke out in a cold sweat with thoughts of the Hardware Fuckup trilogy. Thankfully though, it did not seem to be hardware related. All temps were fine, and it kept going for a few more hours before crashing again. Hmm. Quick browse with Malwarebytes Anti-Malware (again the free version) revealed that yes, a nasty infection of something or another had taken hold. Bugger.

First solution is the "fuck it" solution. Fire up Acronis, tell it to restore from the previous Wednesday's backup. It tells me it needs to restart the machine to restore to the system drive. Fine, fine, just get on with it. It restarts ... back into Windows. No sign of a backup being restored. Arse. Next, grab a USB stick, set it up as bootable media and boot from that. Fine. Select archive to restore. It cannot be restore because there may not be enough space left to boot. What. The. FUCK. is that about? This is a mirror of a whole disk. It is not possible to fit more stuff in the image than was on the disk in the first fucking place.

I have never seen this message on earlier version of True Image. So I take a punt and download the trial version of the 2010 software. It complains that the image is corrupted. A Ha! says I. Maybe the error message is some sort of default and actually the archive is bollocksed. So I reinstall the 2011 version and validate the archive. The archive is fine. Fucking arseholes. This must be a backwards compatibility issue. So I then use the 2011 software to convert the archive from True Image format to Windows Backup format, boot to the USB key version 2010 and ... it complains about the image being too big for the blah blah. However, it does at least let me IGNORE THE BULLSHIT ERROR and just restore the image anyway. Which is what it is doing now. [UPDATE] It has now finished and the restore went fine. So if you backup using TIH2011, all you need to do to restore an image is to convert it to a Windows Backup Image, remove TIH2011, install TIH2010, create a USB rescue disk, boot from the same and restore the Windows Backup Image. Simples.

I also discovered a little gem of a bug in the software. I like incremental backups, where only the changes since the last backup are recorded. It lets you keep lots of different versions for relatively little hard disk space. There is only one problem with this option in the 2011 version. It does not fucking work. With one archive, which extends to 2Gb, I watched as it made the first archive file, then a day later made the second (7Kb - no changes), then a day later made the third file (also 7Kb no changes) by OVERWRITING THE FIRST FILE. Fucking brilliant. All data gone. Even when all of the data is demonstrably there it complains about being unable to find the original volumes. Fucking useless. I now have two 7Kb files and no 2Gb data anywhere.

Apparently both of these problems are known in the world of Acronis Users. This has left me fucking pissed off. I have backup software which, at a fairly fundamental level, does not fucking work. If it wasn't bad enough that it overwrites backup files, it also DOES NOT LET YOU RESTORE FROM BACKUPS THAT YOU HAVE MADE. Backup software which just backs up and does not restore IS NOT BACKUP SOFTWARE; it is a useless pile of crap.

I loved the previous version of Acronis I had. I can't even remember where I got it - it may have come bundled with the Seagate. It worked fine. I used it for months doing regular backups of my important stuff, and I did restores from time to time. It just would not detect my SATA hard disks on my new motherboard. Now I have the latest 2011 version which is useless.

Now, what about this Norton Ghost stuff then?

get_iplayer

get_iplayer is a very useful command line utility to download, perfectly legally, programs from the BBC's iplayer facility.

Ordinarily, you can download the programs using iplayer's built in p2p system. This may or may not be attractive for you. It is useless if the program you are interested in is not available for download in a format you want for example. The alternative is watching the programs in the website's flash player. This is great, unless your ISP comprises a greater number of arseholes than a proctologist's final year exams. If so, you are likely to find your ISP bandwidth throttles real time video. You can find out if your ISP is doing this here.

If you suffer from this problem, then get_iplayer could be the solution. There are various packages for distributions etc etc available on the website, and it even works in windows. To install it from scratch though, use the following commands from your home folder, or where ever:

wget http://linuxcentre.net/get_iplayer/packages/get_iplayer-current.tar.gz
tar -xzvf get_iplayer-current.tar.gz
cd get_iplayer*
wget http://download.savannah.gnu.org/releases-noredirect/flvstreamer/source/flvstreamer-1.8k.tar.gz
tar -xzvf flvstreamer-1.8k.tar.gz
cd flvstreamer
make flvstreamer
make streams
cd .. 

The software will now be installed, with the important flvstreamer addon. Download the iplayer database by running this command from the isntall folder:

./get_player

You can configure default options by running these commands:

./get_iplayer --prefs-add --modes=flashhd,flashvhigh,flashhigh,flashnormal,iphone
./get_iplayer --prefs-add --flvstreamer "./flvstreamer/flvstreamer_x86" 

You can check and remove these default options by running:

./get_iplayer --prefs-show
./get_iplayer --prefs-clear

By way of example, you can now download a clip embedded in a BBC webpage by running this command:

./get_iplayer --url="http://news.bbc.co.uk/sport1/hi/motorsport/formula_one/8304965.stm"

You can download live F1, and presumably other sports, with a little more effort. First, install the Firebug add-on for firefox. Now, activate the add-on and browse to the live website. Use the “All Panels” option from the bug icon in the lower right corner of Firefox. Then, left click the bug to bring up the information, select the 'Net' tab, and look for a connection like this:

http://www.bbc.co.uk/mediaselector/4/gtis/?server=cp52115.live.edgefcs.net&identifier=sport6a_hq@s5444&kind=akamai&application=live&cb=59251

Copy that URL, and paste it into the command above as the text between the quotes.

Using Evolution with Exchange Server

Evolution is to Outlook, as Thunderbird is to Outlook Express. While Thunderbird will let you get your email from your ISP, and from Google (and sometimes, when it fecking works, hotmail), Evolution's big selling point is that it works with Exchange. Exchange being Microsoft's email server.

To get Evolution working in either Ubuntu or Arch you need to install the following packages:

evolution evolution-exchange

When you have installed these, you configure Evolution as follows:
Add an account and specify type as Exchange Server.
You then set these variables:
Receiving E-mail → Username = [domain]/[logon_name]
Receiving E-mail → OWA URL is https://[mail_server_name]/exchange
Receiving Options → Global Catalogue Server Name = [mail_server_ip_address]
Default → Sent Messages Folder = set to Exchange not Local
The system should auto find the exchange account if you click authenticate.
Public Folders are accessed by going Folder → Subscriptions, and choosing Exchange Server.
[domain] examples would be, smoobandit.com, smoobanditprivate.local

Once you pop in your password you should find that Evolution connects and starts downloading all your mail.

However, this is buggier than a trap for Indiana Jones. Every so often, including more than once per day, it will fubar its connection to the exchange server. It either denies there is any mail, or tells you you have 3 messages in the last 3 months. Sometimes this is accompanied by an error saying that it has lost connection to the backend something or other. Sometimes not. The only way to fix this, that I have found, is to open a terminal and run these commands:

evolution --force-shutdown
cd ~/.evolution/mail/exchange
sudo rm -rfv ./[domain]

Then re-run Evolution and it will re-sync to Exchange. This is OK, unless you have something stored locally and not on the server. For instance, if you were drafting an email when the connection fubarred, then you would lose that because it would only be stored locally. Bastard.

Testing Windows 7

So, there is a release candidate of Windows 7 out. I want to test this by using it in a virtual machine. It installs fine, but will not see the VMWare network, audio or USB drivers. Goddammit.

To solve the network problem you need to ensure that you are using the e1000 virtual hardware. In order words, your .vmx file needs to include:

ethernet0.virtualDev = "e1000"

To get the USB working, you need to ensure that both scsi and ehci are running, with these lines in your .vmx file

scsi0.present = "TRUE"
scsi0.pciSlotNumber = "18"
ehci.present = "TRUE"
ehci.pciSlotNumber = "19"

I have no fucking idea how to get the audio to work.

[UPDATE]
This software fucking hates me. Once I had setup all of the above, USB worked flawlessly in the Virtual Machine. However, once I shut it down and restarted, it was fubarred. I checked the .vmx file and found to my horror that the .pciSlotNumbers for scsi- and ehci were now '-1'. Brilliant, why the fuck has it done that? Solution - change it back to the text above and WRITE PROTECT the .vmx file. Job done.

Mount Windows Shares in Arch

Oh for fucks sake this is difficult. It should be easy, really it should be easy, but the combination of shitty documentation, and crap network authentication standards leads to a mess.

It should work, on a temporary basis with these commands:

mkdir /media/network
mount -t cifs "//server/share/" /media/network -o username=myusername,password=mypassword

There should be a way to do all this automatically with smbnetfs, but that software is currently fucked.

Otherwise, this might have stood a chance:

pacman -S smbnetfs
modprobe fuse
nano /etc/fuse.conf #uncomment 'user_allow_other'
/etc/rc.d/smbnetfs start

This does stand a chance, actually. Just not with anything that should, god forbid, use some kind of username/password security. Configuration is in:

/etc/smbnetfs/.smb/smbnetfs.conf

Which failing a forced manual mount would look like this:

cd /mnt/smbnet/DOMAIN:myusername:mypassword@server

But that fails with the delightfully informative "Transport endpoint is not connected", which makes about as much intuitive sense as "Your wurfle strip has sturbified, you must re-cognificate your panzel-struder".