Monthly Archives: March 2011

Feed Tracking Tool released under an Open-Source license

Posted on by
2

I’ve just open-sourced the Feed Tracking Tool project (aka “FTT”), my first (and only) Ruby on Rails experience.

This tool was developed within Uperto, the company I currently work for, for its internal needs. The project had an ancestor written in 2006 that was based on Pylons. It was a prototype and was barely working. Iterating over the abandoned Python code base was considered a waste of time. So in summer 2007, it was decided to rewrite this application from scratch.

As my co-worker was available and already played with Ruby on Rails, he was tasked to create the initial code base. I joined the project early on, as it was a great opportunity to play with the (then really trendy) Ruby on Rails framework.

At the end FTT was essentially a test project to explore Ruby on Rails. It was never deployed on a production server and was never used.

After roting for more than 3 years, and representing absolutely no business value in itself, I decided to release it under a GPLv2 license (with Uperto’s approval of course). My intention with this open-source release is to share back knowledge and code with the community.

FTT was living in a private Subversion repository at Uperto, but we unfortunately lost it. During the last few weeks I tried to rebuild the code history from old and partial backups. I then used my Git-based reconstruction method to consolidate everything in a Git repository. The code is now available on GitHub.

I don’t plan to maintain this project. But I may reboot it in the future if I need feed-related features, or if I need an excuse to play with Ruby on Rails again. But for now beware: the code is quite outdated and is only running on old Rails 1.2.x. This project should be considered as an ugly legacy code base. So please be indulgent while looking at FTT’s code: it was the work of unexperienced RoR developers ! ;)

Why I chose Ubercart instead of Magento or osCommerce.

Posted on by
4

About a year ago a friend asked me why I chose the Ubercart/Drupal combo as the engine for Cool Cavemen’s online shop. He specifically asked me why not choose Magento or osCommerce. I never really took the time to answer him. Let’s fix this.

First, I selected a couple of web eCommerce solution based on the same requirements upon which I choose WordPress some years ago. Namely:

  • Open-source free software
  • Active project
  • Healthy community

I excluded osCommerce right away. I quickly played with it in 2005 and in 2007 for projects that never came to fruition. And I keep a really bad memory of these experiences: both code and templates felt poorly designed and implemented. That’s why osCommerce is classified in my brain as a giant legacy code base, hard to maintain. Maybe the project has evolved a lot since then. I should have looked back at it, but was attracted by another project…

…which was Magento. So yes, I seriously considered to use Magento at some point. It looked great. And clean. But digging deeper I found that something was missing.

My shop was supposed to sell tee-shirts for the band. And a tee-shirt is a product that can have lots of variations:

  • different colors (white, black, yellow, …),
  • of different sizes (S, M, L, X, XL, …),
  • in different shapes (teeshirt, hoodie, sweatshirt, …),
  • for different peoples (guy, girl, baby, …).

I wanted to track stocks and prices for each combination. And this degree of complexity is not supported by Magento. To be fair, I remember to have heard of this feature, but was only available in a third party module (that I don’t remember the name). Alas, this module was proprietary. That is the main reason I discarded Magento.

And then I found Ubercart, which allowed me to do exactly what I wanted. Here is for example prices and stocks for a black Cool Cavemen tee-shirt:

I use Ubercart since 2008 and I’m really happy with it. The future is interesting, as Ubercart was forked as the Drupal Commerce project a year ago. I’m waiting for the 1.0 release of the latter to decide if it is worth switching to the fork or not. But having it based on Drupal 7 is good news, as Ubertcart still stick to the (old) Drupal 6.

Fixing messed-up encoding in MySQL

Posted on by
Reply

Currently working on my e107 Importer plugin, I was confronted today with badly-encoded data coming from my databases.

e107 migrated to full UTF-8 years ago, but I must have messed the upgrade process at the time. That was my conclusion when I took a close look to my tables: all of them seems to be set to Latin-1 but contain UTF-8 data. Here are screenshots from SQLBuddy (a great light-weight MySQL manager) showing just that:

To fix this, I first tried to use the following command I found on the web:

mysql --database=e107db -B -N -e "SHOW TABLES"  | awk '{print "ALTER TABLE", $1, "CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;"}' | mysql --database=e107db

But this doesn’t work, as it not only change the encoding of the table, but also transcode the data inside the table.

Let’s try something else. First, we’ll export the database to a dump file, of which the encoding is forced to Latin-1:

mysqldump -a -c -e --no-create-db --add-drop-table --default-character-set=latin1 --databases 'e107db' > ./e107-data.sql

Now the trick is to change the CHARSET parameter of all CREATE TABLE directives to UTF-8:

sed -i 's/CHARSET=latin1/CHARSET=utf8/g' ./e107-data.sql

We’ll also change the NAMES directive to force MySQL to handle imported data as UTF-8:

sed -i 's/SET NAMES latin1/SET NAMES utf8/g' ./e107-data.sql

Then we’re free to import the result in a new UTF-8 database:

sed -i 's/USE `e107db`;/#USE `e107db`;/g' ./e107-data.sql
mysql --execute="CREATE DATABASE e107db_new CHARACTER SET=utf8"
mysql --database=e107db_new < ./e107-data.sql

And now, accentuated characters appears as they should in our database, meaning we’ve fixed all the mess ! :)


PS: I found another alternative method (look at the end of the linked page) which consists of temporarily handling TEXT fields as BLOB, to have MySQL treat them as binary content (thus skipping character transcoding). Haven’t tested this but sounds tricky.

I’m Top 1% Open-Source Developer ! (and that’s a lie)

Posted on by
3

With all the recent active development on my e107 Importer WordPress plugin, I increased my public code contributions. This had the nice side effect of bumping my ranking on Ohloh from #8 (bronze level) to #9 (silver level):

Other interesting statistic is that I’m now ranked as open-source developer number 5673 on a population of 438276. Which place me in the top 1.3% of the population ! :D

That’s exciting, but irrelevant. Ohloh stats must be taken with a huge grain of salt.

  1. Some code are duplicated: my WordPress plugin is both present in my personal GitHub repository and in WordPress official plugin repository, inflating contributions on this project by 2.
  2. Not all open-source projects are tracked in Ohloh. Which make developer registered on Ohloh seems to be part of a smaller community.
  3. Not all developers on Ohloh have aggregated their repository accounts under one identity, making those who’ve done this looks like bigger contributors compared to others. This also artificially inflate the global population.
  4. Open-source contributions are not necessary in code repositories. Think about project promotion, maintenance of forges and websites, documentation, bug reports, testing, benchmarking, support (in mailing-lists, IRC and forums), …
  5. And most importantly, the best contributions are not always tied to high commit activity or number of added code lines. Thinks about removing old/dead/legacy code and refactoring. These may be the best code contributions a project will see.

That’s why Ohloh stats must not be taken for granted. But this doesn’t remove the fun you can get from them. Especially when they put me in a favorable light ! ;)

e107 Importer 1.1 available !

Posted on by
2

A month after the 1.0 release, here is my e107 Importer v1.1 for WordPress !

The biggest new feature is support of e107 forum import to the bbPress WordPress plugin. This plugin is still in alpha and was not released yet. This mean you have to fetch it from its Subversion repository. And be careful to get the recommended version (SVN release 2942).

Because of the experimental status of forum import, the default option of e107 Importer is to not import forums. I decided to include this feature anyway to get feedbacks. So please don’t consider forum import as a highly reliable. It may work for you or may not. And please write detailed bug reports.

Here is a detailed changelog between 1.0 and 1.1:

  • Add import of forums and threads to bbPress WordPress plugin.
  • Parse BBCode and e107 constants in forums and thread.
  • Add forums and threads redirections.
  • Make e107 user import optional. This needs you to set a pre-existing WordPress user that will take ownership of all imported content.
  • Parse BBCode in titles too.
  • Import images embedded in comments and forum threads.
  • Description update of existing users is no longer destructive.
  • Add an entry in the FAQ regarding script ending prematurely.
  • Disable all extra HTML rendering hooks like the one coming from e107 linkwords plugin.
  • Allow news and pages import to be skipped.
  • Add missing news category redirects.
  • Minimal requirement set to WordPress 3.1.
  • Some pages are not tied to a user. In this case, default to the current user.