Downloading videos from

22 October 2011

I have a netbook, and I really enjoy watching (also because Rai has an idiotic policy for people like me living abroad: only RaiNews24 is available for us emigrants).

However, things being what they are, I am not always connected to the Internet. Therefore, I’d like to download some replicas of TV shows they air (like news, or the incomparable Crozza’s Italialand!) on my hard disk, and watch them while waiting (say) at the airport or in the train.

So, I wrote a script to download videos from directly. Just go there, open the episode you’d like to download, and run this Ruby script passing it the URL:

#!/usr/bin/env ruby

require 'uri'
require 'net/http'
require 'rexml/document'

# Constants
LA7TV_URL = ''
ARCHIVE_URL = 'rtmp://'

if ARGV.empty? or not ARGV[0].include? LA7TV_URL
  $stderr.puts <<-EOD
This script downloads videos from, for your personal use only.
The idea is that you can watch what you see online for free, but offline too.
Please be sensible and don't use this method for breaking the law.

Please make sure rtmpdump is installed.

Usage: #{$0} <>
  exit -1

asset_number = ARGV[0].split('assetid=')[1]
xmldoc = Net::HTTP.get URI.parse(sprintf DESCRIPTOR_URL, asset_number)
xml = xmldoc
videos = REXML::XPath.match(xml, '*/videos/video')
video = videos.max_by { |video| video.elements['quality'].text.to_i }

local_filename = video.elements['originalName'].text
remote_filename = video.elements['fms'].text.gsub(/^mp4:\//, '')
remote_url = sprintf ARCHIVE_URL, remote_filename

puts "Downloading #{local_filename}\n\tfrom #{remote_url}"
Kernel.exec "rtmpdump -e -o '#{local_filename}' -r '#{remote_url}'"

It’s maaagiiiic! 🙂 Remember to install rtmpdump (available in most distros), and of course Ruby.

PLEASE USE THIS RESPONSIBLY. You are consuming a lot of bandwidth in a short timeframe on their server by downloading things instead of streaming them, so: a) do it only for personal use, and b) don’t do it for things you will not watch.


What to do if the gherkin gem does not compile

7 February 2011

On my GNU/Linux box, the gherkin gem doesn’t want to compile – and it is a dependency of cucumber, so that’s pretty annoying.

The errors returned when trying to build it are:

matteo@orchid:~$ gem install gherkin
Building native extensions.  This could take a while...
ERROR:  Error installing gherkin:
ERROR: Failed to build gem native extension.

/home/matteo/.rvm/rubies/ruby-head/bin/ruby extconf.rb
checking for main() in -lc... yes
creating Makefile

compiling gherkin_lexer_ar.c
cc1: warnings being treated as errors
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl: In function ‘unindent’:
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:252:3: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl: In function ‘store_multiline_kw_con’:
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:285:3: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl: In function ‘store_pystring_content’:
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:322:3: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl: In function ‘CLexer_init’:
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:364:3: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl: In function ‘CLexer_scan’:
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:378:3: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:381:3: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:180:5: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:230:9: error: ISO C90 forbids mixed declarations and code
/Users/ahellesoy/scm/gherkin/tasks/../ragel/i18n/ar.c.rl:230:9: error: ISO C90 forbids mixed declarations and code
make: *** [gherkin_lexer_ar.o] Errore 1

Fortunately, the error is pretty self-explanatory. You can have it install by just doing:

gem install gherkin -- --with-cflags="-std=c99"

That will fix it. Cheers,

Having Cherokee playing well with RVM and Ruby on Rails

5 May 2010

Update: now it works with Rails 3.0.

Today I went down to try and have Cherokee working well with RVM. I wanted being able to switch the Ruby version with ease, in order to allow for a painless upgrading when patches are released upstream. More, I wanted to be able to create gemsets and such. Cherokee is fast as hell, and much easier to maintain than Apache.

After a little bit of fiddling, I came up with a nice and easy solution, which roughly goes like this:

  1. Create a rails user on your system. My advice is to lock it down with “passwd -l rails” after creation.
  2. If you installed any gems as root, it’s best to remove them. Then, follow the normal instructions to install rvm su-ing as the rails user. Compile and set as default a ruby instance of your choice (“rvm use –default ruby-1.8.7“, for example).
  3. Always logged in as the rails user, install any gem you may need. You can do this later, if you prefer. Test if your website starts manually, by calling script/server, or if it complains about missing gems.
  4. chmod -R your rails project to rails:rails. I keep my production sites under /var/www, but you can put ’em in /home/rails, for example.
  5. Use the standard wizard that comes with Cherokee to prepare the sources for your website.
  6. Under the “Interpreter command” text field of each of the three newly created sources, prepend the command that’s already there with (“/home/rails/“). For example: “/home/rails/ example-website script/rails server -b -e production -p 38161“. I omitted “/var/www/“, but you can put it there if you want.
  7. For each of the sources, set the user and the group the site will be served with to “rails“.
  8. Create a new file /home/rails/, which will do the simplest magic we need:

if [ "$(whoami)" != "rails" ]; then
echo "Cannot run this script as root. You must sudo to the 'rails' user."
exit -1;

export HOME="/home/rails"

if [[ -s "$HOME/.rvm/scripts/rvm" ]]; then
source "$HOME/.rvm/scripts/rvm";

cd "/var/www/$1"
exec ${@:2}

Now, if someone of the Cherokee project would be so kind to fix that ugly “Bad gateway” error the first time you try to access a Rails site and the interpreter hasn’t been spawned yet, I’d be immensely grateful. 🙂

My thesis: Gallows

19 December 2009

This week I finally managed to get my Bachelor’s degree in Computer Science with a final mark of 101/110. Here you’ll find my thesis about the project I implemented during my final internship: Gallows.

Gallows is a scaffold generator that uses ExtJS 3.0 to achieve CRUD actions on a editable grid, as well as on some associated controller views. The Gallows generator produces a grid view of records starting from a pre-existing model. The capability of modifying, creating and deleting items is achieved via AJAX, by setting up ExtJS 3.0 to use a JSON renderer.

It is necessary to run the generator two times: the first round it creates a stub of configuration file, which you can alter to achieve the wanted result in views. The second time, it generates a controller and the right JSON-based views.

The advantage of this approach is to allow you to get automatically some join capabilities across different tables/models, and to associate the right widget to the right field when more than one may apply. (Hopefully) sensible defaults will be auto-generated. The controller name (which can be namespaced) must match the model name.

Feel free to download and share my thesis: Gallows: a scaffold generator for Ruby on Rails employing ExtJS 3.0.

ExtJS 3.0, XTemplate and fields with square brackets

21 August 2009

I’m working a lot with Ruby on Rails and ExtJS 3.0 in this period, and one of the problem I faced was with the management of the standard Rails field names when outputted for a form.

As you may know, Rails generates fields in the form with names like this:


…so that when in the controller you call @artist.update_attributes!(params) all the right magic works, since params is a hash in the form:

{ offset => 0, _method => 'put', artist => { 
    firstname => 'James', lastname => 'Hetfield' } 

However, ExtJS doesn’t like these kind of parameter names inside Ext.form.Combobox and Ext.DataView, to name two, since they use an XTemplate to render a list of items. The standard regular expression which is used to catch parameters inside a template allows only for digits, letters, ‘-‘ and ‘#’ (plus a little bit of other magic you really don’t want to read).

It turns out that you can override the ‘tpl’ configuration parameter to use XTemplate syntax for inline evaluation. For example, for a Combobox, you could move from the default to:

'<tpl for="."><div>{[values["' + this.displayField + '"]]}</div></tpl>'.

And there you go, problem solved – hopefully.