Tag Archives: Git

Apache commands

Posted on by
1
  • Hide Subversion and Git directories content (source):
    RedirectMatch 404 /\.(svn|git)(/|$)
  • Disable rendering of PHP files coming from imported third party Javascript submodules (context):
    RedirectMatch 404 js-(.*)\.php$
  • Redirect any request to current year sub-directory (I used this for a yearly-updated static web page):
    RewriteEngine on
RewriteRule !^/2010/ /2010/ [R=301,L]
  • Here is my template for domain-based virtual host routing:
    # Setup the main website access
<VirtualHost *:80>
  ServerName example.com
  DocumentRoot /var/www/example
  # Add extra capabilities to let CMS like WordPress manage redirections
  <Directory /var/www/example>
    Options +FollowSymLinks +SymLinksIfOwnerMatch
  </Directory>
</VirtualHost>
# Redirect all other access to the website from different domains to the canonical URL
<VirtualHost *:80>
  ServerName www.example.com
  ServerAlias *.example.com
  ServerAlias example.net *.example.net
  ServerAlias example.org *.example.org
  RedirectMatch permanent (.*) http://example.com$1
</VirtualHost>
  • Insert dynamic headers in HTTP responses depending on the browser:
    BrowserMatchNoCase ".*MSIE\s[1-6].*" IS_DISGUSTING_BROWSER
Header add X-advice-of-the-day "Save a kitten: use Firefox !" env=IS_DISGUSTING_BROWSER
  • Prevent WebDAV connexions (thanks Guillaume!):
    <Location />
  <Limit PROPFIND PROPPATCH MKCOL COPY MOVE LOCK UNLOCK PATCH>
    # Leaves GET (and HEAD), POST, PUT, DELETE, CONNECT, OPTIONS and TRACE alone
    Order allow,deny
    Deny from all
  </Limit>
</Location>
SetEnvIf Request_Method "OPTIONS" CLIENT_PROBE
Header set Allow "GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE" env=CLIENT_PROBE
  • At work, we had to engineer a convoluted software architecture for our intranet to fit the network security policy of our customer. This had a bad side effect of letting the web statistic collector delete all cookies but its own, thus breaking intranet’s authentication. So we (thanks Matthieu!) came up with this unmaintainable hack on Apache side to hide our intranet’s cookies to NedStat’s Javascript embedded code:
    <LocationMatch "/(.*)">
  LoadModule headers_module modules/mod_headers.so
  RequestHeader edit Cookie "(app_cookie_001=[^;]*(; )*)" ""
  RequestHeader edit Cookie "(app_cookie_002=[^;]*(; )*)" ""
  RequestHeader edit Cookie "(app_cookie_003=[^;]*(; )*)" ""
</LocationMatch>
  • Kill all apache processes and restart the service:
    /etc/init.d/apache2 stop ; pkill -9 -u www-data ; /etc/init.d/apache2 restart
  • Restart Apache service if no process found:
    [ `ps axu | grep -v "grep" | grep --count "www-data"` -le 0 ] && /etc/init.d/apache2 restart

Using latest stable Kdenlive with a development version of MLT

Posted on by
1

Today I stumble upon a bug in the Kdenlive 0.7.8 running on my Kubuntu 10.10: the crop filter was messing with the display ratio of my video clips. Digging the web I found a bug report that was really close to my problem. According to the comments, this issue was fixed in the upcoming version of MLT. Is that bug the one I encountered ? The only way to find out was to install the development version of MLT. Here is how I did it…

First, make sure to use the latest stable Kdenlive stack for you system. For me, the Sunab’s alternative repository for Kubuntu 10.10 was the ultimate source:

sudo apt-get update && sudo apt-get install kdenlive

The idea is to keep the version of Kdenlive installed above, and replace the pre-packaged MLT on our system with a custom development version of our choice.

But first, we’ll install all the libraries required to build MLT from sources:

sudo apt-get install libavdevice-dev libswscale-dev libvorbis-dev libsox-dev libsamplerate-dev frei0r-plugins-dev libdv-dev libavformat-dev libquicktime-dev libxml2-dev libsdl-dev libsdl-image1.2-dev

Let’s now remove the installed MLT. If we use apt-get or KPackageKit, this will remove Kdenlive. So we’ll use the following command to remove MLT while ignoring all the dependencies:

sudo dpkg --remove --force-depends libmlt2 libmlt++3 libmlt-data melt

At this point, and every time we try to use it, apt will complain of broken Kdenlive dependencies, and will try to remove it. This mean we can’t upgrade other packages on the system.

To avoid this issue, I tried to freeze the state in which Kdenlive and MLT are, by setting the hold flag on kdenlive, kdenlive-data, libmlt2, libmlt++3, libmlt-data and melt packages. I tried with both dpkg and aptitude, but unfortunately it doesn’t work as expected. So we’ll continue our hack anyway…

Let’s get MLT sources:

git clone git://mltframework.org/mlt.git

The command above will give you the latest development version. But if you target a particular revision (like commit 21a3f68 in my case), you have to use this additional command:

git checkout 21a3f68

We can now follow the procedure detailed in the Kdenlive manual:

cd mlt
./configure --prefix=/usr --enable-gpl
make clean
make
sudo make install

That’s it ! Now you can launch Kdenlive, and if you run the wizard, you’ll see that the MLT version on your system is the latest:

Oh, and by the way, it fixed my problem with the crop filter ! :)

Finally, if you want to revert the mess we created on the system, you have to remove the MLT we built in place:

sudo rm -rf /usr/lib/libmlt*
sudo rm -rf /usr/lib/mlt*
sudo rm -rf /usr/lib/pkgconfig/mlt*
sudo rm -rf /usr/include/mlt*
sudo rm -rf /usr/share/mlt*

I came with the list above by searching my system with the following command:

sudo find / -path "/home" -prune -or -iname "*mlt*" -print -or -iname "*melt*" -print

Then, we can let apt handle Kdenlive and MLT properly and get back to the pre-packaged binaries:

sudo apt-get remove kdenlive && sudo apt-get update && sudo apt-get install kdenlive

Subversion commits and mail activity stream in iCalendar

Posted on by
Reply

Last week I consolidated all my code in my GitHub repository. I stumble upon an old script I haven’t publicized yet: svn2ical.py.

This is a simple hack which get commit metadata out of a Subversion repository and generate an iCalendar file containing all commits of a given author. I used it back then to visualize in a calendar my commit activity. Nowadays this script is quite useless as services like Ohloh and GitHub provides great timeline and activity streams. But this script can still be useful for private repositories.

And in the same spirit of this script, I uncovered maildir2ical.py, a script that look in a maildir folder for mails sent by a particular author, then generate an iCal file based on mail dates.

Commit history reconstruction with Git

Posted on by
1

Here is something I wanted to do for 3 years. I wanted to migrate my code repository from this:

to a proper revision control system, like Subversion. And I wanted to reconstruct the commit history with all the proper dates. That’s something I can’t do with SVN.

Then came Git. I knew that Git was powerful enough to let me manipulate the history (at my own risks). So I studied it during the last weeks until I found an acceptable way to do exactly what I had in mind. Here are my notes regarding this journey.

First, I need to get a local copy of my GitHub repository. That’s the place where I want all my code to reside at the end of the process.

cd ~
git clone git@github.com:kdeldycke/kev-code.git

In gitg, my untouched repository looks like this:

Notice all the pre-existing code.

Let’s create a history-injection branch from the init tag. The later is the root of my repository, as explained in my previous post on how I initialize my Git repositories.

git branch history-injection init

Then switch to our brand new branch:

git checkout history-injection

We are now in a safe and contained environment in which we can do all our dirty stuff. Let’s move the file we want to add in our repository:

cp ~/kev-code/website-backup-2006_04_30.py ~/kev-code/website-backup.py

Commit this new file locally, as usual, but with a commit date set in the past:

cd ~/kev-code
git add --all
git commit --all --date="2006-04-30 23:17" -m "First version of a script to backup several remote websites via FTP and make bzip2 archives."

I can repeat the last steps to reconstruct the commit history of my website-backup.py script:

cp ~/kev-code/website-backup-2006_10_29.py ~/kev-code/website-backup.py
git commit --all --date="2006-10-29 23:13" -m "Delete previous backups if nothing has changed."
cp ~/kev-code/website-backup-2006_11_01.py ~/kev-code/website-backup.py
git commit --all --date="2006-11-01 23:14" -m "Keep monthly bzip2 snapshots of backups and incremental backups of the last 32 days thanks to rdiff-backup."
(...)

At last, the history-injection branch contain all version of website-backup.py:

Now I’ll use the rebase directive to insert the history-injection branch back in the main line (aka master). This insertion will take place just after the init tag. This translates to the following Git command:

git rebase --preserve-merges --onto history-injection init master

The --preserve-merges option is really important here to not let Git takes too much initiatives. Without this option, all our banches between the init tag and the head of the master branch will be rebased. Believe me, that’s not what we want.

I no longer need my temporary history-injection branch. Let’s remove it:

git branch -D history-injection

Now you should have a unique and straight history line from init tag to master head. Like this:

Commits appears to be ordered as they should but you may not be as lucky as me. In fact the recently merge commits are stuck at the “bottom” (just after the init tag, as we asked Git to do on rebase). And you may find you in a situation where commits of the whole master branch are not chronologically ordered.

Here is such an example. It happened when I tried to rebase the full history of my system-backup.py script:

I haven’t found a way to tell Git how to rebase by following commit dates. I know that something can be done with a command like:

git rebase --interactive init

But I haven’t succeeded yet. So I left these commits unsorted for now. I may write another blog post in the future if I find a way to cleanly sort them. In the mean time, If you have a solution, I’ll be happy to ear that !

Finally, when we have something that looks good, we can push our changes to our remote GitHub repository:

git push origin

But Git will complain: changing already-pushed commits is bad. As I explained several weeks ago, it’s dangerous but I don’t care. I’m the only user of this repository. So let’s bypass Git’s wise warnings:

git push origin +master:master

Et voilà ! By repeating these steps several times, I moved my code to GitHub, with a consistent and clean commit history.

How I initialize my Git repositories

Posted on by
6

The first few days I used Git, I messed up my repository. I had to reset and recreate it from scratch several times. With enough trials and errors, I came up with an idea of how I should initialize my repositories. Let me explain in this post why git init is not enough to me.

To create a Git repository, nothing else is absolutely necessary than these few trivial commands:

$ mkdir kev-code
$ cd kev-code/
$ git init

But after reading some documentation and user experiences on the web, it looks like Git has some limitations when dealing with the root of a repository history. As I plan to heavily manipulate the commit history (to do some kind of code archaeology and history reconstruction), I need to have the widest time latitude to play with commits.

In this situation, I came to the conclusion that it’s a good idea to create an empty commit at the start of your repository life, and date it to the start of epoch. In the future, I’ll be able to leverage this intial commit as an ordinary history point from which I can start a branch. Then in this branch I’ll be free to mess up the history, until merging my changes back in the mainline tree.

So, let’s create an empty commit:

$ git commit --allow-empty -m 'Initial commit'

Then get the commit hash:

$ git log
commit 395290bcdb8ffccfbff89e42cb976077fbd3c1b7
Author: Kevin Deldycke <kevin@deldycke.com>
Date:   Tue Dec 1 15:37:49 2009 +0100

    Initial commit

We now change the commit date of our first commit to epoch start:

$ git filter-branch --env-filter '
>     if [ $GIT_COMMIT = 395290bcdb8ffccfbff89e42cb976077fbd3c1b7 ]
>     then
>         export GIT_AUTHOR_DATE="Thu, 01 Jan 1970 00:00:00 +0000"
>         export GIT_COMMITTER_DATE="Thu, 01 Jan 1970 00:00:00 +0000"
>     fi' -- --all
Rewrite 395290bcdb8ffccfbff89e42cb976077fbd3c1b7 (1/1)
Ref 'refs/heads/master' was rewritten

And check that the previous operation did what we expected:

$ git log
commit 8fe2934d1552c97246836987f0ea08e10ba749ae
Author: Kevin Deldycke <kevin@deldycke.com>
Date:   Thu Jan 1 00:00:00 1970 +0000

    Initial commit

Looks good !

For convenience, we’ll now attach a tag to this initial commit. Let’s call it init:

$ git tag "init"

This will came handy later when we’ll need to create a branch from here.

It’s time to push all changes to our brand new public repository:

$ git remote add origin git@github.com:kdeldycke/kev-code.git
$ git status
# On branch master
nothing to commit (working directory clean)
$ git push origin master --force

Counting objects: 2, done.
Writing objects: 100% (2/2), 159 bytes, done.
Total 2 (delta 0), reused 0 (delta 0)
To git@github.com:kdeldycke/kev-code.git
 + 86bd2c7...8fe2934 master -> master (forced update)

And here is the result on GitHub:

Maybe this “first commit” trick is unnecessary. So, if you have a better understanding of the issue, or can explain me why this is stupid, please tell me ! :)

How-to fix bad commit authorship in Git

Posted on by
1

Several months ago I commited some code in my GitHub repository, but I did it from a temporary system. If I registered my authentication keys correctly to commit stuff, I forgot to create a minimal ~/.gitconfig file with the right stuff in it.

The result was not good looking, as my usual name and mail address were not attached to the commit:

Let’s fix this !

First, get a local copy of the remote Git repository:

git clone git@github.com:kdeldycke/kev-code.git

What was missing in my ~/.gitconfig file were the following options:

[user]
name = Kevin Deldycke
email = kevin@deldycke.com

These values can be set with Git command line with the following syntax:

--author 'user.name <user.email>'

The commit I want to change is the latest in history, so I’ll use the --amend directive to make my changes. Putting all things together, our final command becomes:

git commit --amend --author 'Kevin Deldycke <kevin@deldycke.com>'

After this, here is how the local branches looks like in gitg:

Using the git log -n1 command, we can compare the old commit:

commit 81a26f03901918ed4a954d964b2659187f1cc988
Author: kevin <kevin@laptop-kev.(none)>
Date:   Mon Mar 8 22:49:43 2010 +0100

    Update old shop logo with the brand new one

with the new one:

commit adf4620f3d8a89746dd643dcefc3f900f0f69878
Author: Kevin Deldycke <kevin@deldycke.com>
Date:   Mon Mar 8 22:49:43 2010 +0100

    Update old shop logo with the brand new one

Notice the fixed authorship. The commit ID was also updated as it’s just a hash depending on commit metadata.

Now we can push our changes back to the remote repository:

git push origin

But this doesn’t work and throw the following error:

To git@github.com:kdeldycke/kev-code.git
 ! [rejected]        master -> master (non-fast forward)
error: failed to push some refs to 'git@github.com:kdeldycke/kev-code.git'

This is Git protection mechanism in action. Modifying already-published commits like this is a bad idea. It can break updates of other developers’ repository (if they already have pulled the commit we’re trying to change).

In our case we will force the remote repository to take our changes:

git push origin +master:master

As I told you before this is bad, but nobody really cares: I’m the only person working on this repository ! ;)

Finally, you can contemplate the result on GitHub, a clean and tidy commit history: