Friday, November 14, 2008

Sharing data between processes

Memba Velodoc Outlook Add-In lets you send large files from Outlook. The first component is a VSTO add-in running in the Outlook process, which provides a user interface like WinZip and WinRar to add large files to a transfer package and hyperlinks in the email body to download these files. The second component is an executable called transfer controller which actually uploads this package to a file transfer server in the background. The transfer controller provides a user interface to monitor the progress of transfers and pause, resume and cancel transfers. A plug-in architecture makes the transfer controller compatible with several file transfer protocols and server platforms including FTP. The VSTO add-in and the transfer controller communicate together via .NET remoting which is the standard inter-process communication (IPC) mechanism in .NET.

In all builds including and prior to build #081107 of release 1.0, the TCP port opened for .NET remoting on the transfer controller is hard coded. This has proved to be an issue when using windows fast user switching and terminal services sessions because each user needs his/her own instance of the transfer controller to access the user interface and they are all competing to listen on the same port.

First, ensuring one and only one instance of the transfer controller process per user session is achieved using a local mutex. Note that using a global mutex is a well-known design pattern to ensure one instance of a process per machine.

Second, we needed a way to configure .NET remoting in the transfer controller to listen on the first available port and notify the VSTO add-in to communicate on the same port. Actually the problem is even more complicated as we cannot tell which between the VSTO add-in and the transfer controller is started first because normally the VSTO add-in starts the transfer controller, but then Outlook might be closed and re-opened while the transfer controller is already loaded. Accordingly, we needed a way to scan for an available TCP port, cache it and share it between processes. We could have used the current user registry, but since the user registry is persistent there is a risk to lock the processes into a port which is no more available. So we needed to ensure that even in the case of a crash the port used would not persist once the add-in processes had exited. We have opted for shared memory.

In Windows, shared memory is a special case of memory mapped files, where the file mapping object accesses memory backed by the system paging file. However, the lifetime of this memory ends when the last process connected to the shared memory object closes connection or the application exits, so there is no data persistence. If an application creates shared memory, fills it with data and exits, the data is lost and this is exactly what we need.

Memory mapped files are implemented using the following Win32 kernel APIs: CreateFileMapping, OpenFileMapping, MapViewOfFile, MapViewOfFileEx, UnmapViewOfFile, FlushViewOfFile and CloseHandle. Note that the naming convention used in CreateFileMapping and OpenFileMapping follows the convention of mutexes where a name prefixed with Global\ creates the object in the global address space (machine) and a name prefixed with Local\ or not prefixed at all creates the object in the local address space (user session).

This will feature in the next build of the Memba Velodoc Outlook Add-In due by the end of the year. In the meantime we provide here a prototype in the form of a Visual Studio 2008 console application project. The first instance of the console application which you may launch should scan for an available TCP port, store it in shared memory and display it. All other instances should find that port in shared memory and display it. As a consequence, all instances of the console application should display the same shared TCP port.

No comments: