Kevin Deldycke

Fix big HTML table rendering in IE

Kevin Deldycke — Mon, 17 Jun 2013 12:32:00 +0200

It is well-known since the burst of the dot-com bubble that tables are bad for layout. But tables still makes perfect sense to present huge quantities of tabular data.

The thing is, Internet Explorer can't cope with big HTML tables. So when it encounters a huge number of cells, IE9 starts to shift them randomly:

You can go here to reproduce this issue.

I absolutely don't care about these issues. Life is short. I have better things to do than fixing IE bugs.

Still, at work, things are different. You're trading time for money. And it's up to your customer to choose what you should do with your time. I guess that's the tragedy of becoming a part of the software workforce.

That's how I started to dive into IE hell.

After some research, I found out the root cause of this issue. It's the presence of extra spaces and line returns between cells. Any sane browser should ignore these. But IE9 simply can't.

This issue seems to have been known since April 2011 (at least), but hasn't been addressed by Microsoft yet.

And you know how I fixed this ? I fearlessly added the following piece of JavaScript in my initialization code:

// Fix IE9 cells misalignment
matrix.find("table tr").contents().filter(function() {
    return this.nodeType == 3;
}).remove();

The code above just strips all extra spaces from tables it find. You can see this hack in action in my matrix widget for OpenERP 6.0.

This is ugly, but makes my customer happy. Life is full of contradictions. And that's normal.

Support La Quadrature du Net !

Kevin Deldycke — Sun, 09 Jun 2013 12:41:00 +0200

6 months ago I gave 100€ to La Quadrature du Net. It's an advocacy group defending the rights and freedoms of citizens on the Internet. Its reputation is well established and its actions are internationally recognized by the Electronic Frontier Foundation and the Free Software Foundation.

They didn't reached their fundraising goal last year, but it doesn't matter: donation are accepted all year long. If you care about your rights & freedom in a digital society, you should contribute.

As a gift, you'll received decimals of Pi:

maildir-deduplicate.py moved to its own repository

Kevin Deldycke — Tue, 04 Jun 2013 12:48:00 +0200

In 2010 I created maildir-deduplicate.py, a one-of script to detect and remove duplicate copies of emails.

Since then, Adam added loads of features and refinements to it, promoting himself to the de-facto maintainer. 3 months ago he privately asked me if I could split this script out of its messy home. Which makes sense.

This post is just a message to all maildir-deduplicate.py users. The script is no longer available in my generic repository. It now has its own dedicated repository on GitHub, for which Adam has full administration rights.

Behind the Scenes of Omashay's "Where is she ?"

Kevin Deldycke — Mon, 20 May 2013 12:32:37 +0200

Back from holidays, and it's time for me to clear my backlog of draft articles. Here is the last and missing piece of my 2012 production notes series.

Let's rewind to year ago, when Where is she ? was released:

Pre-production

Tomasito planned this video and most of its shots in advance. Here is the basic outline of the scenes we arranged during a brainstorming session at my apartment in the end of 2010:

We postponed the shooting so many times that these sticky papers were hanging on my wall for 2 years.

Eventually this analog timeline led to some sketches and spreadsheets to prepare our shooting:

Shooting

The following gear was involved in the making of this video:

I shot the video in 2012 in two days (1 & 18 May) with some help from Robin (who makes a cameo appearance as the upset tourist). Here is the behind the scenes video:

And some extra photos of the shooting:

The wedding entrance video was the first time I used my Glidecam HD-2000. But Where is she ? was the first publicly released video featuring my new toy. And while filming with it in Monrmartre, a police patrol car paid us a visit:

No need to say the music video was produced in guerilla style, without any warning nor permission... ;)

Editing

Tomasito edited alone the source footage (1080p, 23.976 fps, 1/50s shutter) in Kdenlive:

At that stage, I just helped him by creating the seamless split screens:

Color correction

As I said in Kdenlive's forum, the color correction was a first. I never worked on a project for which any serious color correction was done. Until Where is she ?.

I was worried by the final look of it because, as Marko pointed out in the thread, the footage was captured in all sorts of lighting conditions. It's hard to keep a consistent exposure between all these locations, especially with the tight latitude of a Canon 7D (even with a fine-tuned neutral color profile).

Robin did all the color correction in Kdenlive and for him, it was a first too. The goal wasn't to apply create a strong style. Color grading was more or less a technical mean, to keep the exposure jumping from shot to shot. Robin invested lots of time in this project and the result exceeded our expectations. The final video is fairly consistent and the cut between scenes is smooth compared to the raw footage.

I'd love to show you screenshots of the timeline with all its color parameters. Unfortunately we used an old development version, and when I try to re-open the project with the current version I have on my machine, I end up with this errors before completely crashing Kdenlive:

But by looking at the XML source of the project, I can assert that the whole color correction was entirely made with a combination of these 3 filters only:

RGB adjustment (frei0r.coloradj_RGB)
Curves (frei0r.curves)
Brightness

If I can't show you all the details, I can still show you a comparison between the raw footage and the color correction pass (the video below has no audio on purpose):

Addon to restore search range in OpenERP 6.1

Kevin Deldycke — Tue, 23 Apr 2013 12:25:36 +0200

In OpenERP 6.1, the default search widget for dates and floats no longer feature a range:

This was uncovered by users in the community. But it's not a bug, it's a feature, a design decision.

Worse, datetime search field in OpenERP 6.1 no longer let you set the time:

To fix these issues, I've created web_smile_search_range a module which is available since 2012, but I never advertised it on this blog.

This module is based on an open-source addon from Credativ. I extended the later to:

Replace single date, datetime and float search fields to a range.
Allow selection of time in datetime search fields.

Thus restoring the default behavior from OpenERP 6.0 that was ditched in 6.1.

And if you need to dig deeper in the code, you'll quickly find out that the core of the module lie in the search.js file, which extend in Javascript the search fields models of our interests.

Run OpenERP's ir.cron from command-line

Kevin Deldycke — Thu, 18 Apr 2013 12:46:34 +0200

Here is a simple Python script to trigger an ir.cron job from the command-line system.

I created this to let my customer use his proprietary job scheduler to trigger OpenERP tasks. It was tested on OpenERP 6.1.

#!/usr/bin/python

import datetime
import xmlrpclib
import traceback
import sys
from optparse import OptionParser

parser = OptionParser()
parser.add_option("-d", "--db", dest="db_name", help="OpenERP database name", metavar="DB_NAME", default="openerp")
parser.add_option("-p", "--password", dest="password", help="OpenERP admin password", metavar="ADMIN_PASSWORD", default="admin")
parser.add_option("-n", "--name", dest="cron_name", help="OpenERP ir.cron object name", metavar="CRON_NAME")

(options, args) = parser.parse_args()

assert options.cron_name

try:
    sock = xmlrpclib.ServerProxy('http://localhost:8069/xmlrpc/object')
    ir_cron_ids = sock.execute(options.db_name, 1, options.password, 'ir.cron', 'search', [('name', '=', options.cron_name)])
    if not ir_cron_ids:
        raise "No cron found in OpenERP."
    for ir_cron in sock.execute(options.db_name, 1, options.password, 'ir.cron', 'read', ir_cron_ids, ['model', 'function', 'args']):
        print "%s, %s - (%s) %s %s%s" % (options.db_name, datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), ir_cron['id'], ir_cron['model'], ir_cron['function'], ir_cron['args'])
        sock.execute(options.db_name, 1, options.password, ir_cron['model'], ir_cron['function'], *eval('tuple(%s)' % (ir_cron['args'] or '')))
except:
    traceback.print_exc(file=sys.stdout)
    sys.exit(8)

sys.exit()

Pelican effects

Kevin Deldycke — Fri, 05 Apr 2013 12:03:26 +0200

Here is a screenshot from Google's webmaster tool monitoring the average page download time of this site. Can you spot when I switched from WordPress to Pelican ? :)

Tale of Two: production notes

Kevin Deldycke — Tue, 26 Mar 2013 12:32:37 +0100

This is the fourth video I've work on for Omashay:

This video is a timelapse of a painting Tomasito made in 2007. It's based on a series of photos he took every 10 minutes:

slowmoVideo

I tried to produce a timelapse out of these images a year ago. In fact, that was the original project I was referring to in my previous article, the project which triggered my initial interest into slowmoVideo.

But the experiment failed and I abandoned this endeavor. Instead of slow-motion, and because of the low timing resolution of the photos, I assumed a simple slideshow would do it.

Blender

The original photos were not consistent. To make them work as a slideshow, they required some stabilization. I tried the new tracking features of Blender:

While I could feel the power of the tracking tools, my limited knowledge of Blender stopped me. I abandoned this approach. But one day for sure, I'll give Blender the time it deserves.

vid.stab & Kdenlive

My plan B for stabilization was Transcode's vid.stab plugin.

But it can't read images: it only takes videos for input. So we'll produce a video file of a simple slideshow in Kdenlive, then feed the result to vid.stab.

First, in a separate project, I created a stupid slideshow of 1 photo per frame, without any transition:

To keep details, I exported the project to a high-quality h264 stream composed of I-frames only:

I wanted to use a true lossless codec here, but after several trials and errors, this profile was the only one vid.stab was able to digest.

I can now apply the stabilization, with a high shakiness parameter to make the algorithm ignore the lack of fluidity:

$ transcode -J stabilize -i ./slideshow-redux.mp4 -y null,null -o dummy
$ transcode -J stabilize=shakiness=8 -i ./slideshow-redux.mp4 -y ffmpeg -F huffyuv -o ./slideshow-redux-stabilized.avi

Then we extract all frames of the stabilized video to get a new set of photos:

$ mkdir ./timelapse-stabilized
$ cd ./timelapse-stabilized
$ cp ../slideshow-redux-stabilized.avi
$ ffmpeg -i ./slideshow-redux-stabilized.avi -f image2 stab%05d.png

From the produced images, I created a new slideshow with proper transitions.

I presented the final rendering to Tomasito, and we agreed that the result, while not incredible, was a good excuse to distribute one of his song on YouTube.

He planned to published the video, but the final editing step was postponed by several months. And then we both forgot the project.

slowmoVideo, again

Until last January when, after some efforts, I managed to produce something with slowmoVideo. We resurected the project.

I applied the raw slowmoVideo transformation on the initial set of photos. And the result was good enough. But Tomasito wanted more, and stabilized all the 70 images of the original set by hand !

After this herculean task, I cropped & resized the images to fit the 1080p resolution:

$ convert -resize 1920x1080 -background black -gravity center -extent 1920x1080 ./manually-stab-keyframes/* pict%04d.png

As for Goodnight, we tried to get rid of the wide black bars on the sides. QPX created for us a mask made of paint strokes:

Then I used that mask to blend the photos with a background fiber texture from Subtle Patterns:

$ find ./ -iname "pict*.png" -exec composite "{}" ./stressed_linen-1080p.png ./Masque-02.png "{}"-composed.png \;

Finally I send these pre-rendered keyframes to slowmoVideo to produce a 4 minutes ultra-slow-motion. Then Tomasito added a title and credits.

There is no magic and the final result could have been better if properly shot. What's missing was a steady canvas and good camera positioning, and of course a lower delay between photos. These are the perfect conditions to produce a proper timelapse, and I'm sure we'll demonstrate that in a future project.

libvirt commands

Kevin Deldycke — Wed, 20 Mar 2013 12:25:36 +0100

List system virtual machines:

$ virsh -c qemu:///system list --all

List session VMs:

$ virsh -c qemu:///session list --all

Dump a VM configuration to XML file:

$ virsh -c qemu:///system dumpxml vm-name > vm-name-conf.xml

Create a new VM by importing its XML configuration:

$ virsh -c qemu:///system define vm-name-conf.xml

OpenERP 6.1 web client hacks: hide buttons with Javascript

Kevin Deldycke — Thu, 14 Mar 2013 12:25:36 +0100

Last year I published two articles on hacking OpenERP 6.0 user interface, one to demonstrate how to monkey-patch native Javascript behavior, the other to show how to tweak widgets

Both methods are based on an injection of inline Javascript code via XML view definition. In OpenERP 6.1, as highlighted by Timothy you can no longer leverage this dirty trick: Javascript is stripped out of the XML stream.

But I found new ways of hacking OpenERP's web client in 6.1. Following these discoveries, I've created a the experimental web_smile_hide_buttons addon to hide the hard-coded create and duplicate buttons on form views.

This module is not generic: it just demonstrate how to distribute dirty patches to the web client. It currently:

Hide create buttons on all list views (affect tree views and many2one search view pop-ups),
Hide the save button on form views,
Hide create and duplicate button on all read-only form views,
Only apply these transformations to a configurable subset of models.

The core of the module lies in the /static/src/js/custom.js file. Here is an excerpt of that file, which sums-up my hacks:

openerp.web_smile_hide_buttons = function(openerp) {

    // Models for which we'll hide create and duplicate buttons
    var MODELS_TO_HIDE = ['res.partner'];

    // Hide the create button on all list views, which affect tree views and many2one pop-up search view
    openerp.web.ListView.include({
        start: function() {
            var self = this;
            var ret = this._super.apply(this, arguments);
            var res_model = this.dataset.model;
            if ($.inArray(res_model, MODELS_TO_HIDE) != -1) {
                self.options.addable = false;
            };
            return ret;
        },
    });

    // Hide the save button on form views
    openerp.web.FormView.include({
        on_loaded: function(data) {
            var self = this;
            var ret = this._super.apply(this, arguments);
            var res_model = this.dataset.model;
            if ($.inArray(res_model, MODELS_TO_HIDE) != -1) {
                this.$element.find('button.oe_form_button_save').remove();
            };
            return ret;
        },
    });

    // Hide the create and duplicate button on all page views (i.e. read-only form views)
    openerp.web.PageView.include({
        on_loaded: function(data) {
            var self = this;
            var ret = this._super.apply(this, arguments);
            var res_model = this.dataset.model;
            if ($.inArray(res_model, MODELS_TO_HIDE) != -1) {
                this.$element.find('button.oe_form_button_create').remove();
                this.$element.find('button.oe_form_button_duplicate').remove();
            };
            return ret;
        },
    });

};

As you can see in the full version of the code above, I tried to hide create entries of many2one context menus, but failed to.

And to make the module really complete, I still have to find a way to hide create & modify entries of many2one drop-down menus. If you have some working code that's doing this, feel free to send me a pull request on GitHub.