Ultimate guide of Lotus Notes mail migration

The title may sounds pretentious but extracting mails out of Lotus Notes is soooo hard and complicated, that achieving such a task feels like winning an epic battle against the forces of evil.

Anyway. The goal of this post is to help you migrate all your Lotus Notes mails to a more convenient and standard format like maildir or mailbox.

There are several ways of extracting all your mail trapped in Lotus Notes’ proprietary databases.

Method #1: using integrated IMAP service

This is probably the simplest method. It consists in using the Lotus Notes desktop client (were your emails currently resides) as an IMAP client.

Essentially, what you have to do is just to create a secondary account linked to an IMAP server, like Gmail, etc. This works well and is explained in details in this tutorial.

But sometimes your Notes client is behind firewalls and proxys. So you can’t reach the Internet.

And some other times, Lotus Notes clients are crippled and don’t let you create an IMAP connexion. Unfortunately this happened to me:

So I had to found another approach.

Method #2: using nlconverter

This method is explored in details in my previous article.

But again, it seems that the Lotus Notes on my machine was crippled and/or corrupted. The nlconverter GUI gave me this error:

And the command line gave me this:

Traceback (most recent call last):
  File "notes2mbox.py", line 21, in <module>
    db = NlconverterLib.getNotesDb(notesNsfPath, notesPasswd)
  File "C:\winnlc-alpha-1\NlconverterLib.py", line 43, in getNotesDb
    session = win32com.client.Dispatch(r'Lotus.NotesSession')
  File "C:\Python26\lib\site-packages\win32com\client\__init__.py", line 95, in Dispatch
    dispatch, userName = dynamic._GetGoodDispatchAndUserName(dispatch,userName,clsctx)
  File "C:\Python26\lib\site-packages\win32com\client\dynamic.py", line 104, in _GetGoodDispatchAndUserName
    return (_GetGoodDispatch(IDispatch, clsctx), userName)
  File "C:\Python26\lib\site-packages\win32com\client\dynamic.py", line 84, in _GetGoodDispatch
    IDispatch = pythoncom.CoCreateInstance(IDispatch, None, clsctx, pythoncom.IID_IDispatch)
pywintypes.com_error: (-2147221231, 'ClassFactory ne peut pas fournir la classe demand\xe9e', None, None)

After these two failed attempts, I was quite depressed and not far from surrender to the evil power of proprietary software. Then I managed to setup a new (but complicated) strategy.

Method #3: using Lotus Notes client for Mac OS X

This is the only method that worked for me, and basically, is the same as the first one, but on Mac OS X. Additionally, it involves a local mail server. This procedure was tested several times on Leopard (OSX 10.5).

  1. First things first, download the trial version of Lotus Notes client for Mac OS X (unfortunately you have to register) and install it. I used Lotus Notes 7.0.3 for Mac OS X Leopard (10.5):
  2. You’ll be welcomed by a wizard:
  3. On the next screen, enter a dummy name and uncheck the “I want to connect to a Domino server” box:
  4. Then proceed to the next step in which you’ll uncheck “Setup instant messaging“:
  5. In the last screen, uncheck all boxes. We don’t want to setup any other service yet:
  6. Initial setup is now complete:

    You can now launch Lotus Notes:

  7. On first run, there will be the following screen, where you should click on the “No thanks, just give me the defaults“:
  8. You’ll end up on what will be your default Lotus Notes main page from now on:
  9. The next step is to go back to the machine (Windows for me) from which you’re running the Notes client containing all the mails you plan to migrate. From there, export your mail database:

    All details of this operation can be found in this dedicated article.
  10. Then go back to your Mac OS X machine and import your freshly exported .nsf database. This is as simple as opening the file via the File > Database > Open... menu and dialog:

  11. When Notes attempts to open the file, you’ll be prompted by several dialogs regarding the security attached to the database. If you get the “Create Cross Certificate” screen, then just answer “Yes” as below:

    And every time you get an “Execution Security Alert” message, always check the “Start trusting the signer to execute this action” option before clicking “OK“:
  12. The client will then rebuild the index before giving you a plain view of your inbox:

  13. Next step is to setup a local IMAP server:

    As you can see I used Dovecot, and all is explained here.
  14. Now it’s time to create a new account in Lotus Notes to access this local IMAP server. Click on the Address Book in your toolbar and add a new Account:

  15. Here is where you configure Notes to let it be aware of our local server existence. Only the first tab must be changed to your local parameters. You can left the last two tabs untouched:
  16. Open within Notes your local IMAP mailbox. It is found in the workspace, which you can access via the Databases icon on your toolbar:

  17. You’ll be welcomed by a useless help screen:

    Just close it to get your local IMAP mail view:
  18. While trying to opening the local IMAP mailbox, you may encounter this TCPIP port error:

    In this case, please have a look at my other article explaining how to open TCPIP port in Lotus Notes.
  19. For this step, just copy or cut, then paste, mails from your local .nsf database to your local IMAP account:




  20. While playing with copy’n'paste, you may encounter this error:

    A workaround can be found in this article.
  21. Finally, if like me you’ve played a lot with mails during the transfer step above, you may ends up with loads of duplicate mails. In this case have a look at the deduplication script I wrote. It will help you clean-up your Maildir folder.
  22. That’s it ! You now have a standard Maildir of your Lotus Notes mails, located in your user home directory (~/Maildir):

My ultimate action was to convert the Dovecot maildir to Kmail maildir, as I wanted to use Kmail to finally upload everything in Gmail. But you can use anything that suit your needs, like thunderbird or any mail conversion tools.

Conclusion

  • Lotus Notes sucks. Everybody knows that, but I feel liberated saying that ! ;)
  • The smartest thing to do is to avoid Notes like the plague in the first place. Sadly when working for the man, it’s not always possible… :(
  • The only method I found to work for me (the third solution in this article) is far from perfect from my point of view. What I dream about is a 100% automated solution, like a command line utility we can name nsf2maildir. And as I don’t plan to own Apple hardware and software in a near future, such a command should be 100% free software and running on Linux. I really think there is a “market” for a free software component able to read and understand .nsf files. Any motivated volunteer ? ;)

Convert Lotus Notes’ nsf files to mbox with nlconverter

There is a great piece of software called nlconverter. It’s a tool designed to convert Lotus Notes’ .nsf files to mbox. It rely on win32′s COM/DDE API so it can only be used on Windows.

If you want to extract mails out of your .nsf database, this might be the tool you’re looking for. Bonus point: it’s written in Python ! ;)

Installing nlconverter and its dependencies

Here is how I installed nlconverter on a Windows 2000 (SP4) machine:

  1. First I downloaded and installed the official Python builds for Windows (2.6.6 precisely):




  2. Then Python for Windows extensions (build 214 for Python 2.6 in my case):



  3. Finally I had to download the latest icalendar archive, then extract the \iCalendar-1.2\src\icalendar folder to C:\Python26\Lib\site-packages\:
  4. Next step is to download nlconverter itself and extract it:

nlconverter GUI

First thing you have to do is to create an export of your mails as a .nsf database. Follow the previous link to get the instructions.

Now let’s convert this nsf to a mbox. nlconverter’s FAQ tells you to run the gui.exe program to perform the conversion.

Unfortunately it didn’t work for me:

So I tried the alternative approach by using the command line.

nlconverter command line

Again, most of the things I’m writing here are based on nlconverter’s FAQ:

  1. First, we have to download the notes2mbox.py script from nlconverter’s mercurial repository, as this file is not distributed in the winnlc-alpha-1.zip archive I unzipped previously. Let’s put notes2mbox.py in C:\winnlc-alpha-1\:
  2. Now we’ll modify the notes2mbox.py script to set the password (via the notesPasswd variable) and location (notesNsfPath variable) of the .nsf file. Here are the modifications I applied:
    --- notes2mbox.py.orig	2010-09-02 13:49:58.000000000 +0200
    +++ notes2mbox.py	2010-09-02 13:51:24.000000000 +0200
    @@ -14,8 +14,8 @@
     import NlconverterLib
    
     #Constantes
    -notesPasswd = "foobar"
    -notesNsfPath = "C:\\archive.nsf"
    +notesPasswd = "XXXXXXXXXXXXX"
    +notesNsfPath = "C:\\winnlc-alpha-1\\kevin-notes-big-backup-part-1.nsf"
    
     #Connection à Notes
     db = NlconverterLib.getNotesDb(notesNsfPath, notesPasswd)
    
  3. Before running the script, we have to register a Notes DLL used by nlconverter:
    regsvr32 "C:\Program Files\Notes\nlsxbe.dll"
    


    And make the Python interpreter available system-wide:

    C:\winnlc-alpha-1>SET Path=%Path%;C:\Python26
    
  4. Now we can run the notes2mbox.py script:
    C:\winnlc-alpha-1>C:\Python26\python.exe notes2mbox.py
    

If you’re lucky, you’ll get a nice mbox at the end of the process.

But I was not and the notes2mbox.py ended up with the following error:

Traceback (most recent call last):
  File "notes2mbox.py", line 21, in <module>
    db = NlconverterLib.getNotesDb(notesNsfPath, notesPasswd)
  File "C:\winnlc-alpha-1\NlconverterLib.py", line 43, in getNotesDb
    session = win32com.client.Dispatch(r'Lotus.NotesSession')
  File "C:\Python26\lib\site-packages\win32com\client\__init__.py", line 95, in Dispatch
    dispatch, userName = dynamic._GetGoodDispatchAndUserName(dispatch,userName,clsctx)
  File "C:\Python26\lib\site-packages\win32com\client\dynamic.py", line 104, in _GetGoodDispatchAndUserName
    return (_GetGoodDispatch(IDispatch, clsctx), userName)
  File "C:\Python26\lib\site-packages\win32com\client\dynamic.py", line 84, in _GetGoodDispatch
    IDispatch = pythoncom.CoCreateInstance(IDispatch, None, clsctx, pythoncom.IID_IDispatch)
pywintypes.com_error: (-2147221231, 'ClassFactory ne peut pas fournir la classe demand\xe9e', None, None)

As you can see, I tried hard to make nlconverter working, without any success. But this should not stop you to try. In fact I suspect the Lotus Notes installed on my machine to be crippled or corrupted (can’t really tell). So you may be more lucky than me. In any case, feel free to report any success or failure in the comment section below !

How-to fix Lotus Notes’ disabled TCPIP port error

Today I encountered this error message:

Error logging into server localhost: You must enable the Notes TCPIP port.

This appeared in the fat Lotus Notes client v7.0.3 running on Mac OS X Leopard (10.5).

To fix this issue, first of all, quit Notes. Then locate the Notes’ preference file attached to your current user. Mine was found in my home directory at /Users/kevin/Library/Preferences/Notes Preferences. At the end of this file, add these two directives:

TCPIP=TCP,0,15,0
Ports=TCPIP

Then relaunch Lotus Notes and switch from Island (Disconnected) location to another that will allow your client to listen to the network. In my case, Home (Network Dialup) did the trick:

Problem solved ! :)

How-to export/backup Lotus Notes mails

You are using Lotus Notes as your mail platform. Unfortunately your mailbox has a quota you’ve already reached and you need space. A solution consist in exporting regularly your mails on your local machine to free up your inbox. Here is a little article documenting the export procedure using the fat desktop client.

If screenshots were taken with a french version, instructions given here are for the english one. This will give you enough clues to perform the export whatever the localisation is. The Lotus Notes version I used was the 7.0.2 release.

So first, let’s start Notes and open your mailbox. You should be on a screen similar to this one:

Then, go to the FileDatabaseNew Copy menu:

And you’ll get an export screen that’ll let you choose where to create a local copy of your database:

This will generate a .nsf file containing all your current mail.

Now that you have a backup, you are free to delete all your mails in Lotus Notes. By following this procedure regulary, you can create yearly or monthly archives of you mails without reaching the mailbox quota ! For example, this is how my local archive folder looks like: