Today I stumble upon a bug in the Kdenlive 0.7.8 running on my Kubuntu 10.10: the crop filter was messing with the display ratio of my video clips. Digging the web I found a bug report that was really close to my problem. According to the comments, this issue was fixed in the upcoming version of MLT. Is that bug the one I encountered ? The only way to find out was to install the development version of MLT. Here is how I did it…
The idea is to keep the version of Kdenlive installed above, and replace the pre-packaged MLT on our system with a custom development version of our choice.
But first, we’ll install all the libraries required to build MLT from sources:
Let’s now remove the installed MLT. If we use apt-get or KPackageKit, this will remove Kdenlive. So we’ll use the following command to remove MLT while ignoring all the dependencies:
At this point, and every time we try to use it, apt will complain of broken Kdenlive dependencies, and will try to remove it. This mean we can’t upgrade other packages on the system.
To avoid this issue, I tried to freeze the state in which Kdenlive and MLT are, by setting the hold flag on kdenlive, kdenlive-data, libmlt2, libmlt++3, libmlt-data and melt packages. I tried with both dpkg and aptitude, but unfortunately it doesn’t work as expected. So we’ll continue our hack anyway…
Let’s get MLT sources:
git clone git://mltframework.org/mlt.git
The command above will give you the latest development version. But if you target a particular revision (like commit 21a3f68 in my case), you have to use this additional command:
Last week I was called by one of my co-worker from Uperto (the open-source division of Devoteam). He knew I worked on some video projects for my band, so he asked me if I wanted to help him create one. The video was meant to be released 5 days later, so we clearly were in a hurry. However this was a great oportunity to play with my Canon EOS 7D, so I accepted !
The goal of the video was to show off an Android application we made, for a contest organized by SNCF (the french national railroad). The application, that we simply named Info TGV, is designed to centralize informations for on-board staff. This application will give train personnel better insights when delays or other kind of traffic perturbations arise.
Here is the final video we managed to create:
Now let’s talk about what happened behind the scenes !
Shooting
In all of my projects, I want to use free and open-source software, and push their usage for every step of the creation workflow. For video recording, there is no open-source firmware for the 7D. Sadly, the Magic Lantern community seems to have reached a dead-end regarding 7D support. I’m quite sad as I choosed the 7D a year ago for 2 things: its video capabilities and the potential hackability of this camera (extrapolating Magic Lantern’s success on the 5D Mark II). But this doesn’t stop me to get usable footages. Far from it.
For this video, I choose to use my Sigma 30mm f/1.4 lens as it’s the fastest lens I have today. I wanted to reduce the depth of field as much as I can to emphasize the screen of the HTC Desire, as all the important “action” takes place in the plane of the phone’s screen.
So I set my 7D to record in 1080p at 25 fps, open the lens at f/1.4 and set the shutter speed to a traditional 1/50s. I then set the ISO to 160 (which is the lowest native ISO value available on the 7D). Still, the final image was over-exposed so I attached a variable ND-filter to the lens and tuned it until I had an acceptable result. I finally customized the white balance to match the ambient light temperature.
Here is a snapshot of this first test on my Samsung Galaxy S (notice the strong vignetting created by the ND-filter):
Here is how this first test setup looked like:
All these parameters were calibrated for an afternoon shooting session, but we really started to record in the evening (look at the phone’s clock in the video!). Because of this delay, I lost the white stripe of natural light coming from the top of the background. At the last minute, I started to play with the white balance. Then I changed my mind and removed the ND-filter. And closed the lens to f/3.5 to get a more manageable depth of field. In a word, I messed up all my initial settings resulting in a final video looking worse (in my opinion) compared to my first test. The better is the enemy of the good…
But there is something I’m quite happy with from my last minute changes: dimming the phone’s screen brightness. Phone screens are so powerful nowadays that they create clipped highlights, thus reducing the readability of black characters on white background.
Here is a photo of the final setup, in which I marked the focus plan with black electric tape:
As you can see in the final video, I have some aliasing issues due to the pixelated nature of the phone’s screen and the line skipping happening on 7D’s CMOS sensor. I tried to reduce the aliasing by moving the phone to a distance were the natural softness of the Sigma lens will occur. I didn’t really succeed as it’s really hard to maintain a constant distance to the lens while holding the phone by hand.
Yes, I could have tried to put the phone on a stand but I really wanted to show the app on a real phone, into real hands, as to make it clear there was no special effects or compositing in action. The application is real, it’s running on bare metal, it’s not a mockup, showing that Uperto has tough engineers getting things done !
By the way, about hands: there a trick involved here. The left hand is mine, but the right one is my co-worker’s. There was a big advantage using this technic: with an eye on the 7D’s rear LCD monitor, I can fully concentrate on the image and micro-adjust the distance of the phone to the lens. In the same time, my co-worker can focus (pardon the pun) on the action and follow the script. The only time when you can see my right hand is when I take the Acer Liquid E to demonstrate the propagation of messages via Twitter:
Video editing
For video editing, I knew I’ll not be able to manipulate my 7D’s files natively. At least not with the default packages bundled with my Kubuntu 10.04. So monday morning I started to compile the trunk version of x264, FFmpeg and MLT. Then I realized a brand new version of Kdenlive (v0.7.8) was released. How did I missed such an important news about my favorite NLE ? Compliments must go to the Kdenlive team for providing up to date packages and all their dependencies !
So I did the entire video editing with Kdenlive. Here is what the final project looks like in the timeline:
Inter-title cards were created from scratch with Gimp:
I wanted to record the speach on my linux machine but I didn’t managed to compile the Line 6 open-source drivers. In fact the module compiled but refused to load:
Sep 20 22:02:47 kev-laptop kernel: [ 717.905187] line6usb: Unknown symbol snd_rawmidi_receive
Sep 20 22:02:47 kev-laptop kernel: [ 717.905321] line6usb: disagrees about version of symbol snd_ctl_add
Sep 20 22:02:47 kev-laptop kernel: [ 717.905323] line6usb: Unknown symbol snd_ctl_add
Sep 20 22:02:47 kev-laptop kernel: [ 717.905379] line6usb: disagrees about version of symbol snd_pcm_new
Sep 20 22:02:47 kev-laptop kernel: [ 717.905380] line6usb: Unknown symbol snd_pcm_new
Sep 20 22:02:47 kev-laptop kernel: [ 717.905439] line6usb: disagrees about version of symbol snd_card_register
Sep 20 22:02:47 kev-laptop kernel: [ 717.905440] line6usb: Unknown symbol snd_card_register
Sep 20 22:02:47 kev-laptop kernel: [ 717.905498] line6usb: disagrees about version of symbol snd_card_free
Sep 20 22:02:47 kev-laptop kernel: [ 717.905499] line6usb: Unknown symbol snd_card_free
Sep 20 22:02:47 kev-laptop kernel: [ 717.905557] line6usb: disagrees about version of symbol snd_pcm_lib_preallocate_pages_for_all
Sep 20 22:02:47 kev-laptop kernel: [ 717.905559] line6usb: Unknown symbol snd_pcm_lib_preallocate_pages_for_all
Sep 20 22:02:47 kev-laptop kernel: [ 717.905667] line6usb: disagrees about version of symbol snd_pcm_stop
Sep 20 22:02:47 kev-laptop kernel: [ 717.905668] line6usb: Unknown symbol snd_pcm_stop
Sep 20 22:02:47 kev-laptop kernel: [ 717.905917] line6usb: disagrees about version of symbol snd_ctl_new1
Sep 20 22:02:47 kev-laptop kernel: [ 717.905918] line6usb: Unknown symbol snd_ctl_new1
Sep 20 22:02:47 kev-laptop kernel: [ 717.906043] line6usb: Unknown symbol snd_rawmidi_transmit_ack
Sep 20 22:02:47 kev-laptop kernel: [ 717.906261] line6usb: disagrees about version of symbol snd_pcm_hw_constraint_ratdens
Sep 20 22:02:47 kev-laptop kernel: [ 717.906262] line6usb: Unknown symbol snd_pcm_hw_constraint_ratdens
Sep 20 22:02:47 kev-laptop kernel: [ 717.906379] line6usb: disagrees about version of symbol snd_pcm_lib_malloc_pages
Sep 20 22:02:47 kev-laptop kernel: [ 717.906381] line6usb: Unknown symbol snd_pcm_lib_malloc_pages
Sep 20 22:02:47 kev-laptop kernel: [ 717.906437] line6usb: disagrees about version of symbol snd_pcm_lib_ioctl
Sep 20 22:02:47 kev-laptop kernel: [ 717.906439] line6usb: Unknown symbol snd_pcm_lib_ioctl
Sep 20 22:02:47 kev-laptop kernel: [ 717.906546] line6usb: disagrees about version of symbol snd_pcm_lib_free_pages
Sep 20 22:02:47 kev-laptop kernel: [ 717.906547] line6usb: Unknown symbol snd_pcm_lib_free_pages
Sep 20 22:02:47 kev-laptop kernel: [ 717.906617] line6usb: Unknown symbol snd_rawmidi_transmit_peek
Sep 20 22:02:47 kev-laptop kernel: [ 717.906771] line6usb: disagrees about version of symbol snd_pcm_set_ops
Sep 20 22:02:47 kev-laptop kernel: [ 717.906773] line6usb: Unknown symbol snd_pcm_set_ops
Sep 20 22:02:47 kev-laptop kernel: [ 717.906963] line6usb: disagrees about version of symbol snd_pcm_suspend_all
Sep 20 22:02:47 kev-laptop kernel: [ 717.906965] line6usb: Unknown symbol snd_pcm_suspend_all
Sep 20 22:02:47 kev-laptop kernel: [ 717.907035] line6usb: Unknown symbol snd_rawmidi_new
Sep 20 22:02:47 kev-laptop kernel: [ 717.907095] line6usb: disagrees about version of symbol snd_card_disconnect
Sep 20 22:02:47 kev-laptop kernel: [ 717.907097] line6usb: Unknown symbol snd_card_disconnect
Sep 20 22:02:47 kev-laptop kernel: [ 717.907230] line6usb: Unknown symbol snd_rawmidi_set_ops
Sep 20 22:02:47 kev-laptop kernel: [ 717.907413] line6usb: disagrees about version of symbol snd_card_create
Sep 20 22:02:47 kev-laptop kernel: [ 717.907414] line6usb: Unknown symbol snd_card_create
Sep 20 22:02:47 kev-laptop kernel: [ 717.907474] line6usb: disagrees about version of symbol snd_pcm_period_elapsed
Sep 20 22:02:47 kev-laptop kernel: [ 717.907475] line6usb: Unknown symbol snd_pcm_period_elapsed
Against my will, and to not waste time, I resigned myself to use a Windows machine lying around the office. I installed Audacity and the Windows drivers, then plugged the pod. And voilà, I had a portable recording studio.
Again, as we were in a hurry, we didn’t paid close attention to the way Arnaud was speaking in the mic. Thus the quality of the original take was not fantastic. To me it was good enough for the intended purpose.
When I gave the raw recording to Thomas for mixing, he didn’t take long for him to realize how bad we were at recording ! Even without knowing how we proceed to record and what the setup looked like, he pointed out all the stuff we did wrong. I know him for a long time now, but he still amaze me with his technical and practical knowledge about audio stuff. I really want to thanks him for his help on this project !
By the way, if you’re looking to record/mix/master any audio material (from a simple voice-over to a full band), I really recommend you to contact his studio. And tell him you heard of him by reading my blog, he may give you a discount !
Epilogue
As I concentrated all my efforts towards the creation of the video, I don’t have many details about the application development itself. But coding the android app was, without a doubt, the biggest chunk of work of this project. It involves two of our best Andoid developpers (Jeremy and Paul) and our in-house Photoshop geek for the design. The project was lead by Arnaud.
The whole project was completed in no time and virtually no budget thanks to the dedication and effort of Uperto’s staff. Everybody in Uperto, from management to developpers, was fantastic. It really makes this team unique.
This project was a great oportunity for me to test my video skills. In a sense, it was my first professionnal assignment: I had a limited time, an inflexible deadline and a clear goal. But still, this type of activity falls into my hobby category. I have so much things to learn…
This project is also the first time I edit 1080p footage from my 7D in Kdenlive. This doesn’t mean it’s the first time I shoot with my 7D. In fact I shoot videos in various formats and setup for about a year. But my shooting skills only starts to be acceptable. My other projects involving 7D footages are currently on hold for various reasons (mostly more important stuff to do). That’s why I hadn’t published anything on the web since then. But I expect this situation to change soon !
The KDEnlive included in Mandriva 2007.0 is pretty old (version 0.2.4). Last night I decided to backport the 0.4 release from cooker to Mandriva 2007.0. It is available in my RPM repository.
Get help of a particular filter (eq2 in this example):
mplayer -vf eq2=help
Here is the filter I use to light up a video taken in the dark with my cheap camera. Of course it add noise but thanks to this we can distinguish shapes in the dark.
And this is the command to preview the result of the filter used above:
mplayer video.avi -vf eq2=1.61:1.95:0.54:2.43
This is how I convert raw videos taken with my digital camera into ISO standard MPEG-4 (DivX 5, XVID compatible) videos [to encode in grayscale, add :gray option to -lavcopts]:
Transcode the first video stream found in a m3u playlist to a 384 kbps MPEG-2 video and 48 kHz Vorbis audio, and serve the resulting stream ina Ogg container to http://localhost:8888. To save bandwisth we reduce by two the size of the video: