Getting Velodoc notifications out of junk email folders

Some Velodoc users have complained that they did not receive our email notifications.

We have looked into our application logs and these notifications have definitely been sent. Looking more closely at our SMTP server logs, we have found some entries similar to the following:

2007-08-28 09:07:16 65.55.245.8 OutboundConnectionResponse SMTPSVC1 SV1 - 25 - - 550+Your+e-mail+was+rejected+for+policy+reasons+on+this+gateway.++Reasons+for+rejection+may+be+related+to+content+with+spam-like+characteristics+or+IP/domain+reputation+problems.++If+you+are+not+an+e-mail/network+admin+please+contact+your+E-mail/Internet+Service+Provider+for+help.++For+e-mail+delivery+information,+please+go+to+http://postmaster.live.com 0 0 355 0 578 SMTP - - - -

I have confirmed myself that notifications sent to Live Hotmail addresses were not received, even in the junk e-mail folder.

Although I was pretty sure about the result, I have checked that Velodoc.net was not recorded in any spam database, using http://www.dnsstuff.com/tools/ip4r.ch?ip=velodoc.net.

Going to http://postmaster.live.com as suggested, I have read about SPF and Sender ID. I already knew about it, but our DNS Servers have been hosted by Network Solutions and they do not support SPF records. They know their stuff, don't they? I have even contacted their support team and I have been told that they get little demand for it.

Enquiring further, we have realized that more and more e-mail servers implement SPF/Sender ID. Some are now considering DKIM/DomainKeys to sign emails. So we have taken the decision to move our DNS to DNS Made Easy and to add SPF records to our domains. Note that we could have used UltraDNS or EasyDNS, which offer similar services.

We have used http://www.dnsreport.com to identify other DNS issues. We were only missing a PTR record pointing to mail.velodoc.net. Because each of our web servers has its own SMTP server, we have changed the MX record to www.velodoc.net which already has a PTR record and changed the HELO greeting of the SMTP servers to display the correct fully qualified domain name. Live Hotmail users now receive our email notifications.

In the near future we will be looking at DomainKeys, but contrary to Sender ID, DomainKeys have an impact on the infrastructure because they require that we change our SMTP Servers. If you have some experience with DKIM/DomainKeys and can recommend SMTP servers that implement email authentication, please leave a comment.

Turning the Velodoc Flash applet into a Yahoo widget

Yahoo widget documentation is available at http://widgets.yahoo.com/workshop/.

Yahoo widgets cannot host the browser to display Flash applets. All Yahoo widgets based on Flash require the use of a third party component called WebBrowser4Widgets available at http://www.centraltopic.com. Installing a Yahoo widget which uses this component is a very poor user experience. Several security warnings are displayed, which are too many reasons not to install the widget. Check samples at http://widgets.yahoo.com/gallery/?author=14327

For this reason, we have postponed the packaging of the Velodoc applet as a Yahoo widget and wait until Yahoo widget implements a native mechanism to host Flash applets. Considering the success of Video sites like YouTube, I cannot think of any reason why they should not do it very soon.

Adobe Captivate or Techsmith Camtasia?

Although most of my friends seem to prefer Camtasia, I have been a long-time fan of RoboDemo, now Captivate.

The main difference between the two products is:

• Camtasia records a full-motion video;
  
• Captivate records still images, keyboard strokes and mouse movements which it assembles into a video;
  

The two approaches have their own strengths and weaknesses:

• Camtasia is better at recording and displaying scrolling panes, progress bars and drag-and-drop operations;
  
• Captivate gives a lot more control on the output. In fact, Captivate produces not only demonstrations but also true e-learning content.
  

The new Captivate 3.0 closes the gap and introduces full-motion recording, not in replacement but in addition to still images. Captivate is clever enough to trigger the full-motion recording when it detects a scrolling pane or a drag-and-drop. Anyway you can trigger it manually anytime.

In my opinion, Captivate 3.0 is now far superior to Camtasia on all grounds except price. The Captivate product remains significantly more expansive but it is now justified.

Turning the Velodoc Flash applet into a Vista Sidebar gadget

Documentation

Step-by-step instructions on how to develop a sidebar gadget are available at:

• http://microsoftgadgets.com/Sidebar/DevelopmentOverview.aspx
• http://microsoftgadgets.com/Build/SidebarTutorial.zip
• http://msdn2.microsoft.com/en-us/library/ms723694.aspx
• http://msdn.microsoft.com/msdnmag/issues/07/08/sidebar/default.aspx

Reference documentation is available at http://msdn2.microsoft.com/en-us/library/bb508511.aspx

Implementation

I am always trying to start this type of RAD development from an existing example. The sample that you build in http://microsoftgadgets.com/Sidebar/DevelopmentOverview.aspx is a good starting point.

In our final gadget, the html is simply a div which is filled by Javascript code when the gadget loads:

• The Javascript event handlers are hooked as described in http://msdn.microsoft.com/msdnmag/issues/07/08/sidebar/default.aspx
• On page load we do one of three options:
• If running on Windows 64 bits, we display a message because Flash does not support 64-bit systems.
• If connection is offline, we display a message
• Otherwise, we display the applet

The recommended approach to test a 64-bit platform is to use the System.Machine.processorArchitecture property but this always returns [Object error] on my 64-bit DELL Precision M65. So I have used System.Environment.getEnvironmentVariable("PROCESSOR_ARCHITECTURE") which seems to work consistently over the few Vista computers that I have tested.

There is apparently no function in the framework to check if the network connection is online, so I have used the same approach used in the Google Desktop Gadget.

I have only experienced one hitch in the implementation: the Flash applet catches the mouse events before the Sidebar. As a consequence, the gadget toolbar at the top right of the gadget is never displayed, which prevents from closing or moving the gadget. The workaround I have found is to add an image at the top of the Flash applet, so that when the mouse goes over this image, the gadget toolbar is displayed.

Packaging

You can package your sidebar gadgets as .zip files or .cab files renamed into .gadget files. I have used the following batch file to produce a proper package from sources located in "D:\Documents\Velodoc Sidebar Gadget\Sources". The batch file should be located in the parent directory:

rem ** remove/create a test gadget folder
rd "%LOCALAPPDATA%\Microsoft\Windows Sidebar\Gadgets\Velodoc.gadget\" /s /q
md "%LOCALAPPDATA%\Microsoft\Windows Sidebar\Gadgets\Velodoc.gadget\"
rem ** copy all of the files into test area
xcopy .\Sources "%LOCALAPPDATA%\Microsoft\Windows Sidebar\Gadgets\Velodoc.gadget\" /y /s /q /EXCLUDE:exclude.txt
%SystemDrive%
cd "%LOCALAPPDATA%\Microsoft\Windows Sidebar\Gadgets\Velodoc.gadget\"
"%VS80COMNTOOLS%Bin\cabarc.exe" -r -p n "D:\Documents\Velodoc Sidebar Gadget\Velodoc.gadget" *
pause

Test and debugging

Testing and debugging is relatively easy, considering the logic is in the Flash applet. Simply double click the gadget file to install and test. There is no need to uninstall before a new install, provided the gadget is removed from the Sidebar.

Distribution

Once your gadget is ready, follow the instructions at http://gallery.live.com/submit.aspx to publish it on the Live web site.

Turning the Velodoc Flash applet into a Google Desktop Gadget

Documentation

The Goggle Desktop SDK can be found at http://desktop.google.com/dev/index.html. The documentation is rudimentary, but there are several samples worth exploring. Besides gadgets are packaged in .gg files which can be renamed into .zip files, so that the content can be explored.

A lot of information can also be found in the forums at http://groups.google.com/group/Google-Desktop-Developer?lnk=lr&hl=en-GB.

Implementation

My initial track was to run the Flash applet in the Sidebar as explained in http://groups.google.com/group/Google-Desktop-Developer/browse_thread/thread/2ee23c46237e5408?hl=en-GB. My gadget was very simple: a manifest, a localized string.xml file, the main.xml file represented below and a couple of images.

<view height="240" width="240">
<div id="flashcontainer" style="text-align: center;"></div>
<object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" width="240" height="240">
<param name="movie" value="http://www.velodoc.net/Gadgets/Velodoc.Gadget.swf" />
<param name="wmode" value="transparent" />
<param name="allowScriptAccess" value="sameDomain" />
<param name="quality" value="high" />
<param name="bgcolor" value="#000000" />
</object>
</view>

In particular, there was no Javascript file in this implementation.

This track is a dead end because there is too much interactivity in the Flash applet and Google Desktop seems to get the events before the applet code gets an opportunity to handle them, which results in an unresponsive applet. Accordingly, the only way is to run the applet in a details view.

The best example to start from is HtmlDetailsView which is part of the SDK. Do not forget to change the guid in the manifest, otherwise you won't be able to submit your project to Google at the end. The code is pretty straightforward, so please download the gadget at http://www.velodoc.net/gadgets/GDesktop/Velodoc.GDesktop.gg, rename it into a .zip file and explore.

The only issue I have faced is the use of the system.network.online property to detect the online/offline status. After simply disconnecting my network cable, the value returned was still true. So I have created the isOnline() function which uses a synchronous XMLHttpRequest of type HEAD to confirm access to the Flash applet. I did not feel that an asynchronous request was justified in this case. For more information see http://www.jibbering.com/2002/4/httprequest.html and http://ajaxpatterns.org/XMLHttpRequest_Call.

Test and debugging

Testing is easy and does not require packaging the gadget. Double-click the gadget.manifest file and the gadget is installed in the sidebar.

Packaging and Distribution

To package your applet, simply archive all the files in a zip file. Keep the directory structure where the manifest is at the root of the archive. Then rename the .zip extension into .gg and test. You can install your packaged gadget by double-clicking the gg file.

If you upload your gadget to an IIS 6 server, you will get an error 404 when downloading it. By default IIS blocks files which have no known mime type. See: http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/0f4ac79a-dc2b-4a5f-89c1-d57266aa6ffe.mspx?mfr=true. To be able to download your gadget from an IIS server, you need to create a MIME type for the .gg extension. You can map the .gg extension to "application/octet-stream".

Then the last step is to submit your gadget at http://desktop.google.com/pluginsubmit?hl=en-GB to get some visibility.

Finally, please note that I have also opened a discussion thread regarding this gadget at http://groups.google.com/group/Google-Desktop-Developer/browse_thread/thread/6776e746840080ee?hl=en-GB

Developing a file upload applet for Velodoc with Flash

This article follow a recent article entitled Designing gadgets and widgets for uploading files to Memba Velodoc.

ActionScript 2 and the Flash 8 IDE

This project was my first trial at Flash and in my opinion, Flash 8 presents two main difficulties for .NET developers:

1. The IDE offers rudimentary development and debugging features;
   
2. Scope and addressing of objects (relative or absolute, using _parent or this) is not always clear. Sometimes relative won't work, and this is fixed with absolute addressing. The rule seems to be using relative unless you cannot get around it.
   

Apart from that, Flash is not a great development tool and I would not consider large projects in Flash, but it does a nice job for small applets.

Flash security

As always, sandboxed security is not intuitive and error messages are unfriendly when you get them, so be prepared to spend a significant amount of development time on security. Most issues are explained at:

• http://www.adobe.com/devnet/flash/articles/fplayer_security.html
  
• http://www.adobe.com/devnet/flash/articles/fplayer8_security.html
  

You can also get information about the IE content activation issue at:

• http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=7c29e252
  

The user interface logic

I have opted for a form application with the following forms:

• The application form contains the background elements and the action script.
  
• The outbox form contains a To, Subject and Message text inputs and a "Select file" button. It also contains a label to display the file name and a Send button.
  
• The progress form contains a progress bar and a Cancel button
  
• The settings form allows you to define your settings.
  
• The success form confirms when a use case completed ok.
  
• The error form displays an error message when needed.
  

Except a couple of behaviours, all our script is written in Frame1 of the application form. Navigation between forms is easy to build and does not raise any major issue.

The file upload process

The file upload process using FileReference is simple and fairly well documented in the following documentation:

• http://livedocs.macromedia.com/labs/as3preview/docs/00000142.html
  
• http://www.codeproject.com/aspnet/FlashUpload.asp
  
• http://markshu.ca/imm/flash/tutorial/fileReference.html
  
• http://www.flash-db.com/Tutorials/upload/
  

Nevertheless, two trick needs to be pointed out:

First, FileReference posts a 0 byte request to the designated URL before posting the file. You need to make sure that your server code does not store a 0 byte file in this case.

Second, FileReference emits an invalid multipart request and our server logic got caught. At the end of the request, you will find something like:

------------GI3ae0Ef1GI3cH2KM7gL6ae0GI3Ef1

Content-Disposition: form-data; name="Upload"

Submit Query

------------GI3ae0Ef1GI3cH2KM7gL6ae0GI3Ef1

which should have been:

------------GI3ae0Ef1GI3cH2KM7gL6ae0GI3Ef1

Content-Disposition: form-data; name="Upload"

Submit Query

------------GI3ae0Ef1GI3cH2KM7gL6ae0GI3Ef1--

The differences being an additional \r\n between the form-data and value and a missing double hyphen at the end of the request. Note that this happens when the applet runs in the Flash environment, it does not happen when the applet runs as a gadget within the Vista sidebar, but in this case, there is an additional random character at the end. We have updated our server code to handle these specificities.

Calling web services

There are three ways to call web services in Flash:

1. Remoting is the most flexible, most complicated way. It is the low level stuff.
   
2. The web service component is a data binding control which does not require any programming. You get it to work just by settings properties in dialogs.
   
3. Finally mx.services.* have high level classes that make calling web services from ActionScript really easy.
   

Our requirements are not terribly complex and the mx.services.* classes have been the way to go. There is only one trick, which I have not been able to find either on the web or in the flash documentation, which is how to pass complex objects as parameters.

Styling our Flash gadget

Styling the flash applet certainly represents the largest amount of code and time spent. We needed gradients and bevel effects that would resize. To achieve that, I have used the Flash Drawing APIs described at http://www.adobe.com/devnet/flash/articles/precision_drawing.html and skinning techniques described at http://www.adobe.com/devnet/flash/articles/skinning_3.0.html.

I have decided to postpone changing colours and localization to a future version.

Debugging and testing

For debugging and testing, you will find the following tools helpful:

• Get a debug version of the Flash player at http://download.macromedia.com/pub/flashplayer/updaters/9/flashplayer_9_plugin_debug.exe
  
• Get Flash switcher at https://addons.mozilla.org/fr/firefox/addon/3649
  
• Get Flash tracer at https://addons.mozilla.org/fr/firefox/addon/3469 (With Flash 9, you need to change the path in Flash Tracer Options to C:\Documents and Settings\<user>\Application Data\Macromedia\Flash Player\Logs\flashlog.txt)
  

The applet works fine in Internet Explorer. There is a bug which we have experimented in Firefox: You get a US keyboard if you define wmode="transparent". This is documented at http://blog.headlondon.com/archives/no-wmode-please-were-british/ but seems to have been corrected . Other issues that you may experience are documented at http://www.asserttrue.com/articles/2006/11/17/firefox-and-flash-swf-selection-and-focus-problems.

Result

The result is available at http://www.velodoc.net/Gadgets/Velodoc.Gadget.aspx.

If you want to participate and improve this applet, please contact me. I'll be very happy to share the code with active contributors.

Next, I'll explain how to turn this applet into gadgets and widgets for all major platforms.

The future of Internet TV

Following aerial TV and cable TV, the next revolution is Internet TV or IPTV, which makes the cost of distributing TV programs even lower, allowing for much more targeted and specialized content. There are currently four types of actors that have a chance to play a major role in this exciting revolution:

1. The major TV broadcasters
2. The major telecom providers
3. The online merchants including iTunes
4. The video web sites including YouTube
5. The peer-to-peer newcomers including Joost and Babelgum

The following criteria will make the difference between the platforms and eventually specialize them:

1. Capacity to attract advertising dollars, which is a mix of size and segmentation of the audience
2. Capacity to attract paid subscriptions, which is related to the quality and exclusivity of the content and ultimately to the remuneration and protection of content producers
3. Capacity to display hi-res content, which is mainly a technical issue
4. Ease of use, especially to search for or subscribe to archived content

Let’s now review the various platforms and trends:

	Ad Dollars		Paid Subs.		Hi-Res		Search
Major TV Broadcasters	5	↓	5	↓	4	↑	0	↑
Major telecom providers	0	↑	5	↑	4	↑	2	↑
Online Merchants	0	↑	5	↑	5	↑	5	↑
Video web sites	2	↑	1	↑	1	↑	4	↑
P2P newcomers	1	↑	0	↑	3	↑	2	↑

	Weak point	Strong point	Best fit
Major TV Broadcasters	Transition to Internet	Quality content and revenue stream	News and other short-life content
Major telecom providers	Content	Infrastructure	Mobile phones
Online Merchants	Purchasing process	Hi-Res content	Films
Video web sites	Streaming bandwidth	Navigate and search	Short low-res content
P2P newcomers	Works on PC + Content	Hi-Res content	Documentary

As more platforms will become available, the split between content producers and broadcasters will become more obvious. In this respect, I do not think that the major telecom providers, online merchants, video web sites and P2P newcomers will ever produce content, but will rather enter into agreements to obtain content. In this respect, the major TV broadcasters which both produce and broadcast content will enter in “co-opetition” agreements, for example to broadcast their programs on mobile phones.

Telecom providers are probably the biggest threat to TV broadcasters as many content producers will see in them a new channel for their content. This is probably the main reason for Sky to offer broadband and telephone or for Microsoft to target MediaRoom at service providers.

Considering the purchasing process, online merchants will probably focus on music and films for quite some time and do not represent a significant threat for TV broadcasters and telecom providers.

I cannot imagine video web sites getting enough money from advertising to sustain their activity, they have to remain free for users and they will not be able to compete on high-quality content with TV broadcasters and telecom providers. In my opinion, their only option to survive is to get subscriptions from companies to get their own channel, for example:

• A BMW channel where BMW would present educative content regarding its range of cars;
• A L’OREAL channel where L’OREAL would give advice to women how to make the most of their makeup using its products;
• A DANONE channel where a chef would give recipes using DANONE products;
• A NIKE channel which would give training advice for running the marathon.

The video web site could even provide links to purchase the products demonstrated, competing with video shopping channels.

P2P newcomers will have to review their business model. They are currently in a vicious circle:

• they do not remunerate content producers because they are free and they do not have enough advertising;
• they do not have enough advertising because they do not have enough quality content attracting users;
• finally, they do not have enough quality content because they do not remunerate content producers.

P2P newcomers definitely need some free content to let users evaluate their technology and make the numbers, but they have a vocation at being a video encyclopedia capable of making available not only the blockbusters but more importantly confidential content like documentaries or TV archives and they won’t be able to achieve that without user subscriptions and retributions to content providers.