Downloading videos from La7.tv

I have a netbook, and I really enjoy watching La7.tv (also because Rai has an idiotic policy for people like me living abroad: only RaiNews24 is available for us emigrants).

However, things being what they are, I am not always connected to the Internet. Therefore, I’d like to download some replicas of TV shows they air (like news, or the incomparable Crozza’s Italialand!) on my hard disk, and watch them while waiting (say) at the airport or in the train.

So, I wrote a script to download videos from www.la7.tv directly. Just go there, open the episode you’d like to download, and run this Ruby script passing it the URL:

---

#!/usr/bin/env ruby

require 'uri'
require 'net/http'
require 'rexml/document'

# Constants
LA7TV_URL = 'www.la7.tv'
DESCRIPTOR_URL = 'http://www.la7.tv/repliche/content/index.php?contentId=%s'
ARCHIVE_URL = 'rtmp://yalpvod.alice.cdn.interbusiness.it:1935/vod/%s'

if ARGV.empty? or not ARGV[0].include? LA7TV_URL
  $stderr.puts <<-EOD
This script downloads videos from www.la7.it, for your personal use only.
The idea is that you can watch what you see online for free, but offline too.
Please be sensible and don't use this method for breaking the law.

Please make sure rtmpdump is installed.

Usage: #{$0} <http://www.la7.tv/richplayer/?assetid=#######>
EOD
  exit -1
end

asset_number = ARGV[0].split('assetid=')[1]
xmldoc = Net::HTTP.get URI.parse(sprintf DESCRIPTOR_URL, asset_number)
xml = REXML::Document.new xmldoc
videos = REXML::XPath.match(xml, '*/videos/video')
video = videos.max_by { |video| video.elements['quality'].text.to_i }

local_filename = video.elements['originalName'].text
remote_filename = video.elements['fms'].text.gsub(/^mp4:\//, '')
remote_url = sprintf ARCHIVE_URL, remote_filename

puts "Downloading #{local_filename}\n\tfrom #{remote_url}"
Kernel.exec "rtmpdump -e -o '#{local_filename}' -r '#{remote_url}'"

---

It’s maaagiiiic! 🙂 Remember to install rtmpdump (available in most distros), and of course Ruby.

PLEASE USE THIS RESPONSIBLY. You are consuming a lot of bandwidth in a short timeframe on their server by downloading things instead of streaming them, so: a) do it only for personal use, and b) don’t do it for things you will not watch.

Cheers,
Matteo

Having Cherokee playing well with RVM and Ruby on Rails

Update: now it works with Rails 3.0.

Today I went down to try and have Cherokee working well with RVM. I wanted being able to switch the Ruby version with ease, in order to allow for a painless upgrading when patches are released upstream. More, I wanted to be able to create gemsets and such. Cherokee is fast as hell, and much easier to maintain than Apache.

After a little bit of fiddling, I came up with a nice and easy solution, which roughly goes like this:

1. Create a rails user on your system. My advice is to lock it down with “passwd -l rails” after creation.
2. If you installed any gems as root, it’s best to remove them. Then, follow the normal instructions to install rvm su-ing as the rails user. Compile and set as default a ruby instance of your choice (“rvm use –default ruby-1.8.7“, for example).
3. Always logged in as the rails user, install any gem you may need. You can do this later, if you prefer. Test if your website starts manually, by calling script/server, or if it complains about missing gems.
4. chmod -R your rails project to rails:rails. I keep my production sites under /var/www, but you can put ’em in /home/rails, for example.
5. Use the standard wizard that comes with Cherokee to prepare the sources for your website.
6. Under the “Interpreter command” text field of each of the three newly created sources, prepend the command that’s already there with (“/home/rails/spawner.sh“). For example: “/home/rails/spawner.sh example-website script/rails server -b 127.0.0.1 -e production -p 38161“. I omitted “/var/www/“, but you can put it there if you want.
7. For each of the sources, set the user and the group the site will be served with to “rails“.
8. Create a new file /home/rails/spawner.sh, which will do the simplest magic we need:

#!/bin/bash

if [ "$(whoami)" != "rails" ]; then
echo "Cannot run this script as root. You must sudo to the 'rails' user."
exit -1;
fi

export HOME="/home/rails"

if [[ -s "$HOME/.rvm/scripts/rvm" ]]; then
source "$HOME/.rvm/scripts/rvm";
fi

cd "/var/www/$1"
exec ${@:2}

Now, if someone of the Cherokee project would be so kind to fix that ugly “Bad gateway” error the first time you try to access a Rails site and the interpreter hasn’t been spawned yet, I’d be immensely grateful. 🙂

My thesis: Gallows

This week I finally managed to get my Bachelor’s degree in Computer Science with a final mark of 101/110. Here you’ll find my thesis about the project I implemented during my final internship: Gallows.

Gallows is a scaffold generator that uses ExtJS 3.0 to achieve CRUD actions on a editable grid, as well as on some associated controller views. The Gallows generator produces a grid view of records starting from a pre-existing model. The capability of modifying, creating and deleting items is achieved via AJAX, by setting up ExtJS 3.0 to use a JSON renderer.

It is necessary to run the generator two times: the first round it creates a stub of configuration file, which you can alter to achieve the wanted result in views. The second time, it generates a controller and the right JSON-based views.

The advantage of this approach is to allow you to get automatically some join capabilities across different tables/models, and to associate the right widget to the right field when more than one may apply. (Hopefully) sensible defaults will be auto-generated. The controller name (which can be namespaced) must match the model name.

Feel free to download and share my thesis: Gallows: a scaffold generator for Ruby on Rails employing ExtJS 3.0.

ExtJS 3.0, XTemplate and fields with square brackets

I’m working a lot with Ruby on Rails and ExtJS 3.0 in this period, and one of the problem I faced was with the management of the standard Rails field names when outputted for a form.

As you may know, Rails generates fields in the form with names like this:

artist[lastname]

…so that when in the controller you call @artist.update_attributes!(params) all the right magic works, since params is a hash in the form:

{ offset => 0, _method => 'put', artist => { 
    firstname => 'James', lastname => 'Hetfield' } 
}

However, ExtJS doesn’t like these kind of parameter names inside Ext.form.Combobox and Ext.DataView, to name two, since they use an XTemplate to render a list of items. The standard regular expression which is used to catch parameters inside a template allows only for digits, letters, ‘-‘ and ‘#’ (plus a little bit of other magic you really don’t want to read).

It turns out that you can override the ‘tpl’ configuration parameter to use XTemplate syntax for inline evaluation. For example, for a Combobox, you could move from the default to:

'<tpl for="."><div>{[values["' + this.displayField + '"]]}</div></tpl>'.

And there you go, problem solved – hopefully.

Cheers,
Matteo