Introduction
Velodoc is developed in ASP.NET C# 2.0 with Visual Studio .NET 2005.
VSTS comes with unit testing, web testing and load testing, considering load testing is actually an execution environment for unit tests and web tests.
Unfortunately Visual Studio web tests work at the protocol level, recording HTTP traffic in order to replay it. The same applies to Redgate’s ANTSLoad and many other load testing tools. Velodoc is an Ajax application and this does not work.
We have started our search for a solution from the following lists of testing tools:
- http://www.aptest.com/webresources.html
- http://www.softwareqatest.com/qatweb1.html
- http://www.testdriven.com/modules/mylinks/viewcat.php?cid=21&orderby=titleA&PHPSESSID=9f0501e81e532817372ad86a3dec5395
Obviously there are tools like Mercury Winrunner/Loadrunner and the IBM Rational equivalents which certainly cope but they are too expensive.
Our requirements are the following:
- Test ASP.NET applications
- Works on Windows XP with IE
- Compatible with NetAdvantage controls, iFrames, file uploads and Ajax
- Scripts IE including IE dialogs to drive functional UI tests
- Several instances can be executed concurrently to create load/stress tests
- Uses a familiar technology (low learning curve)
- Open source is a plus and a requirement if the supplier has not been around for a long time.
There are three types of solutions which cope more or less with these requirements:
- Macro recorders/players
- Test environments
- Web application scripting frameworks
Macro recorders/players and test environments
Searching on Tucows, Download.com and other shareware web sites reveals loads of macro recorders. Most of them hook the message pump and replay the windows messages. Obviously, the result is not good. We have tried a dozen of them but only iOpus iMacros 5.2 could do a decent job at automating a file upload from https://www.velodoc.net/. The code is reproduced below:
TAB T=1
TAB CLOSEALLOTHERS
URL GOTO=https://www.velodoc.net
SIZE X=876 Y=627
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:aspnetForm ATTR=ID: SenderTextBox CONTENT=test1@acme.com
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:aspnetForm ATTR=ID: RecipientTextBox CONTENT=test2@acme.com
TAG POS=1 TYPE=TEXTAREA FORM=NAME:aspnetForm ATTR=ID: MessageTextBox CONTENT= Test<SP>with<SP>iMacro
FRAME F=1
WINCLICK X=100 Y=429 CONTENT=
WINCLICK X=100 Y=429 CONTENT=C:\test.bin
FRAME F=0
TAG POS=1 TYPE=INPUT:CHECKBOX FORM=NAME:aspnetForm ATTR=ID: SendTermsCheckBox&&VALUE:on CONTENT=YES
WINCLICK X=490 Y=513 CONTENT=
I see at least three drawbacks to using iOpus iMacros as a functional test tool:
- The lack of assertions and reporting capabilities makes it insufficient for functional testing;
- The lack of infrastructure makes it insufficient for load testing;
- The proprietary language reduces the possibilities despite the fact that it can call external scripts or be called via a COM component.
I have also tried several test environments mentioned in the above lists. Generally, recording the file upload test on https://www.velodoc.net/ has always proved difficult but I cannot tell whether it was feasible or not for some of them. I have limited myself to 2 hours per evaluation and the inherent complexity and learning curve was not worth digging passed first impression.
Web application scripting frameworks
I have looked at two open-source frameworks, which both script IE to execute functional tests on the web application:
Both frameworks are very easy to start with due to a familiar language and an intuitive API. Additionally both frameworks provide the necessary assertions to report on the success or failure of a test fixture.
Scripting a file upload on the Velodoc home page with IEUnit entails the following JavaScript code:
_.openWindow(https://www.velodoc.net);
_.setField("SenderTextBox", test1@acme.com);
_.setField("RecipientTextBox", test2@acme.com);
_.setTextArea("MessageTextBox", "Test with IEUnit");
var fname = "C:\\test.bin";
var cmdShell = new ActiveXObject("WScript.Shell");
cmdShell.Run("C:\\EnterFileName.sbk " + fname, 0, false);
_.setFrame(0);
_.clickObjById("FileInput");
_.setFrame(-1);
_.setCheckBox("SendTermsCheckBox",true);
_.clickObjById("SendWebImageButton");
Where EnterFileName.sbk contains:
var fpath = " " + WScript.Arguments(1);
var popupWin = _.waitForWindow("Choose file", 30000);
_.setFrame(0);
_.findWindow(popupWin, "Edit").sendText(fpath);
_.findWinButton(popupWin, "&Open").click();
Scripting a file upload on the Velodoc home page with WatiN entails the following C# code:
using (IE ie = new IE(https://www.velodoc.net/))
{
ie.ShowWindow(NativeMethods.WindowShowStyle.Maximize);
ie.TextField(Find.ById("SenderTextBox")).TypeText("test1@acme.com");
ie.TextField(Find.ById("RecipientTextBox")).TypeText("test2@acme.com");
ie.TextField(Find.ById("MessageTextBox")).TypeText("Test with WatiN");
Frame f = ie.Frame(Find.ById("FileUploadFrame"));
FileUpload fUp = f.FileUpload(Find.ById("FileInput"));
fUp.Set("C:\\test.bin");
ie.CheckBox(Find.ById("SendTermsCheckBox")).Checked = true;
//ie.TextField(Find.ById("SenderTextBox")).FireEvent("onKeyUp");
ie.Button(Find.ById("SendWebImageButton")).ClickNoWait();
SimpleTimer t = new SimpleTimer(60 * 60);
Span s;
do
{
s = ie.Span(Find.ById("DownloadLinkLabel"));
if (!String.IsNullOrEmpty(s.Text))
goto EXIT;
System.Threading.Thread.Sleep(1000);
} while (!t.Elapsed);
throw new WatiN.Core.Exceptions.TimeoutException("...", 60 * 60));
EXIT:
Assert.AreEqual(true, s.Text.Contains(https://www.velodoc.net/));
}
Conclusion
The beauty of WatiN is that it builds on top of C# and .NET framework. Accordingly you get not only a rich language but also a rich environment. Test code built with WatiN can be executed within NUnit and Visual Studio unit testing projects which means that they can also be executed for load tests which I am going to try next although we know already that launching several instances of IE has its own limitation.
