Compiled application exe GUI testing with NUnitForms
NUnitForms is a pretty awsome tool for GUI unit testing.
However it lacks one major thing. You can't test a whole exe or dll.
For example if you have a MyTestApp.exe with various win forms etc., you can't just run it and test it's gui.
And i really wanted to do it because our app sets a lot of other stuff and it's pretty complex so you can't just
call Form.Show() and start playing with it.
So with a lot experimenting and reading and going through NUnitForms source code I've come up with a way to do it.
The first thing to know is how NUnitForms work. There are a lot of examples like this one which was the first to read on my list.
Every one of these examples creates an instance of the form you want to test, calls it's Show() method and then performs tests.
This is because NunitForms have their own application message loop that feeds messages to displayed forms.
So what you have to do is disable your Apps message loop. You can do that by returning the Main() function
(The main Entry point for your application) before it calls it's Application.Run().
This is the original Main function:
[STAThread] static int Main(string[] argsFromMain) { Application.EnableVisualStyles(); Application.SetCompatibleTextRenderingDefault(false); AppRun(); Application.Run(); return 0; }static void AppRun() {
Form1 f = new Form1(); f.Show(); }
Now we want to disable our apps message loop so we add a startup argument "/nomessageloop":
[STAThread] static int Main(string[] argsFromMain) { Application.EnableVisualStyles(); Application.SetCompatibleTextRenderingDefault(false); AppRun(); if (argsFromMain[0] == "nomessageloop") return 0; Application.Run(); return 0; }static void AppRun() {
Form1 f = new Form1(); f.Show(); }
Now if you try to run your app with "/nomessageloop" parameter your app will show the starting form for a moment and then exit.
This is exactly what we want.
I'm assuming you're familiar with NUnit testing.
Now lets see how we can use this in a test. This is a simple test class called Tests.cs:
using System; using System.Text; using NUnit.Extensions.Forms; using NUnit.Framework; using System.Reflection;namespace NUnitFormsTests { [TestFixture] public class Tests : NUnitFormTest { string exeFileName;
<span class="kwrd">public</span> <span class="kwrd">override</span> <span class="kwrd">void</span> Setup() { exeFileName = <span class="str">@"D:\MyTestApp.exe"</span>; <span class="kwrd">string</span>[] argsFromMain = <span class="kwrd">new</span> <span class="kwrd">string</span>[1]; argsFromMain[0] = <span class="str">"/nomessageloop"</span>; AppDomain.CurrentDomain.ExecuteAssembly(exeFileName, AppDomain.CurrentDomain.Evidence, argsFromMain); } [Test] <span class="kwrd">public</span> <span class="kwrd">void</span> Test1() { <span class="rem">// This assumes that MyTestApp.exe has a textBox named textBox1 and </span> <span class="rem">// 2 buttons named button1 and button2</span> <span class="rem">// button1.Click sets the textBox1.Text to "This is a test button1"</span> <span class="rem">// button2.Click sets the textBox1.Text to "This is a test button2"</span> <span class="rem">// So we're testing if that's true.</span> <span class="rem">// this test should pass.</span> TextBoxTester textbox1 = <span class="kwrd">new</span> TextBoxTester(<span class="str">"textBox1"</span>); ButtonTester btn1 = <span class="kwrd">new</span> ButtonTester(<span class="str">"button1"</span>); ButtonTester btn2 = <span class="kwrd">new</span> ButtonTester(<span class="str">"button2"</span>); <span class="kwrd">string</span> expected1 = <span class="str">"This is a test button1"</span>; <span class="kwrd">string</span> expected2 = <span class="str">"This is a test button2"</span>; btn1.Click(); Assert.AreEqual(expected1, textbox1.Text, <span class="str">"Correct button1 click"</span>); btn2.Click(); Assert.AreEqual(expected2, textbox1.Text, <span class="str">"Correct button2 click"</span>); } }
}
Also since now our app is running in the context of the unit test, you have to put anything you have in your apps app.config
into the app.config of the project that holds Tests.cs. It will be read from there.
The only problem i've run into so far is Drag and Drop registration, but this isn't a critical error.
If I encounter any more i'll post them.
UPDATE: I've found a solutions to the Drag and Drop problem and i've posted it here
|
Legacy Comments
Miha Markic
2007-03-01 |
re: Compiled application exe GUI testing with NUnitForms Ah, the joys of forms unit testing. What I miss for the most part is that NUnitForms doesn't deal with non-standard modal dialogs (IOW non-MessageBox stuff). Or at least it didn't... |
Mladen
2007-03-01 |
re: Compiled application exe GUI testing with NUnitForms as far as i know it does. you have the ModalFormTester and it's ExpectModal function that takes a delegate in which you put your test. works preety ok for me. |
Mladen
2007-03-08 |
re: Compiled application exe GUI testing with NUnitForms you're right. it's fixed. thanx |
marco
2007-04-27 |
re: Compiled application exe GUI testing with NUnitForms Many thanks for your article. 1) I have modified this line if (argsFromMain[0] == "nomessageloop") to if (argsFromMain[0] == "/nomessageloop") so that the strings match. 2) I have modified your code because if I run the .exe (not the test) and I close the program the main window disappears but the process does not end: [STAThread] static void Main(string[] argsFromMain) { Application.EnableVisualStyles(); Application.SetCompatibleTextRenderingDefault(false); Form1 f = new Form1(); f.Show(); if (argsFromMain != null && argsFromMain.Length > 0 && argsFromMain[0] == "/nomessageloop") return; Application.Run(f); } Cheers. Marco. |
Andrew
2007-07-02 |
re: Compiled application exe GUI testing with NUnitForms Note: string[] argsFromMain = new string[1]; argsFromMain[1] = "/nomessageloop"; should be string[] argsFromMain = new string[1]; argsFromMain[0] = "/nomessageloop"; |
Mladen
2007-07-02 |
re: Compiled application exe GUI testing with NUnitForms fixed. thanx for reminding me... was meaning to fix it before but i forgot :) |
Duncan Bayne
2007-09-04 |
re: Compiled application exe GUI testing with NUnitForms If you can't test at least the vast majority of your UI with NUnitForms, there's possibly something wrong with the way you're doing TDD. Are you using NMock? |
Mladen
2007-09-04 |
re: Compiled application exe GUI testing with NUnitForms problem was that we didn't do TDD on this project. we put auto-tests at the end of the cycle... so this was the only way to implement it as far as could figure it out. but then i'm no master of TDD :) |
BGaither
2007-10-22 |
re: Compiled application exe GUI testing with NUnitForms I haven't seen any examples on how to test a winforms datagrid. There are methods for this built into NUnitASP. For instance, I would like to test the values returned into a datagrid as well as fire the event "DoubleClick" on a specific row in the datagrid. Does anyone have any experience with this? In the meantime, I am getting around the problem by using the mouse methods to double click the datagrid, but I can only test the results of that action, not the data within the datagrid. Any help is appreciated. |
Mladen
2007-10-23 |
re: Compiled application exe GUI testing with NUnitForms there is no built in datagrid support. you'll have to build your own custom tester for this. you can fire event with the method FireEvent you can test the data in the grid by accessing it's DataSource Property. |
lt
2007-10-24 |
re: Compiled application exe GUI testing with NUnitForms I have a problem when some other objects initialized. I got an error box when ExecuteAssembly: Se produjo una excepción en el inicializador de tipo de 'Spring.Context.Support.ContextRegistry'. Should I have to add all the .dlls of my tested-application as referneces, and initialized it? It isnt enough to set the results directory, and folder references as the same of my tested-app. By the way, I haven't access to the source of the tested-app. Any sugestions?? thanx!! |
Mladen
2007-10-24 |
re: Compiled application exe GUI testing with NUnitForms yes you should add references to all asseblies to your nunit project. |
Sonal Dewle
2007-12-17 |
re: Compiled application exe GUI testing with NUnitForms I am using a 3rd party control, "SPREAD" in vb.net project. I m using NUnitForms for testing the forms. But i m unable to test the Spread Functionality as NUnit is unable to detect the 3rd party control. Can Anybody suggest me how to test a 3rd party control by NUnitForms? |
Mladen
2007-12-17 |
re: Compiled application exe GUI testing with NUnitForms nunit can detect 3rd party controls just fine. just use UserControlTester. or create a new tester derived from usercontrl tester |
Sonal Dewle
2007-12-18 |
re: Compiled application exe GUI testing with NUnitForms I m unable to get a UserControlTester class in NUnitForms 2.0 Aplha 5 version. Ca u plz give me the NUnitForms version where i can access UserControlTester class? |
Mladen
2007-12-18 |
re: Compiled application exe GUI testing with NUnitForms sorry, i meant ControlTester. derive your own tester from ControlTester |
Willi
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms Hello, How can i run my Application, that i want to test, without blocking the NUnitforms tests? I must run my Application lick here: Application.EnableVisualStyles(); Application.SetCompatibleTextRenderingDefault(false); Application.Run(new MyApplicationContext(new string[] {})); //this blocks everything Thanks |
Mladen
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms not quite sure what you mean by blocking Willi. |
Willi
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms How can i back from Application.Run to place where was is called? i must use Application.Run( new MyMainForm() ) to run my application? so i can´t use this way: MyMainForm myForm = new MyMainForm(); myForm.ShowDialog(); Thanks |
Willi
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms I did it like this, but i couldn't debug it: ..... ApplicationTestStart _applicationTestStart = new ApplicationTestStart(); ThreadStart ts = new ThreadStart(_applicationTestStart.Run); Thread t1 = new Thread(ts); t1.Start(); -------------------------------------------------------------------------------- public class ApplicationTestStart { public ApplicationTestStart() { } public void Run() { Application.EnableVisualStyles(); Application.SetCompatibleTextRenderingDefault(false); Application.Run(new MyApplicationContext(new string[] {})); } } |
Mladen
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms so why exactly must u use Application.Run( new MyMainForm() ) ? |
Willi
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms To run my Application, that i want to test with NUnitforms. So with: Application.Run(new MyApplicationContext(new string[] {})); I can not run it with: Form1 f = new Form1(); f.Show();... or something like this... |
Mladen
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms well we do this and it works just fine: protected override void Start( ApplicationContext application ) { // for NUnitForms Testing if( parameterService.Start.ContainsKey( "nomessageloop" ) ) return; // The magic message loop System.Windows.Forms.Application.Run( application ); } |
Willi
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms and my application use Backgroundworker! |
Mladen
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms so? ours is also completly async in nature. |
Willi
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms what do you mean with: parameterService.Start.ContainsKey( "nomessageloop" ).... where can I find it? |
Mladen
2008-07-16 |
re: Compiled application exe GUI testing with NUnitForms parameterService.Start is a string[] that contains input parameters to the Main(string[] args) method. it's just args. |
Willi
2008-07-17 |
re: Compiled application exe GUI testing with NUnitForms but System.Windows.Forms.Application.Run( application ) bolcks also the course of the application. Just for example: public class ProgramTEST { public ProgramTEST() { Application.EnableVisualStyles(); Application.SetCompatibleTextRenderingDefault(false); Application.Run(new MyApplicationContext(new string[] {})); // -->>>This bolcks the course of the application. } public ControlTester buttonTester; public void Test() { buttonTester = new ControlTester("_okButton", "myForm"); buttonTester.FireEvent("Click"); } } } |
Mladen
2008-07-17 |
re: Compiled application exe GUI testing with NUnitForms maybe you haven't read this blog post carefully... if you pass nomessageloop as a startup parameter the Application.Run is not called. |
willi
2008-07-17 |
re: Compiled application exe GUI testing with NUnitForms I have do it like this, but it bolcks by: AppDomain.CurrentDomain.ExecuteAssembly(exeFileName, AppDomain.CurrentDomain.Evidence, argsFromMain); thanks |
Mladen
2008-07-17 |
re: Compiled application exe GUI testing with NUnitForms uff... if you do AppDomain.CurrentDomain.ExecuteAssembly(exeFileName, AppDomain.CurrentDomain.Evidence, argsFromMain); the i have no clue on how you might be able to test this. |
rajuMSN
2009-02-04 |
re: Compiled application exe GUI testing with NUnitForms Hi, My application has following Main() code [STAThread] static void Main() { Application.Run(new CSI_Main()); } How we can use "nomessageloop" for this context. |
Mladen
2009-02-04 |
re: Compiled application exe GUI testing with NUnitForms [STAThread] static void Main() { Form f = new CSI_Main(); if (not nomessageloop in params when your app starts) { Application.Run(f); } } |
Pham Ngoc
2009-04-17 |
re: Compiled application exe GUI testing with NUnitForms Hi Team! My name's Ngoc. Would you mind if I asked you some questions about NUnitForms. 1.Following your example. The form I want to test is displayed; but after that I can't find any controls and corresponding properties. NoSuchControlException occur. 2. Instead of create an instance of the form I want to test or override method Setup to create an instance, I attach to the form. But after that I can't also find any controls and corresponding properties. NoSuchControlException occur.But I really want to do it. Would you please let me know what should I do asap? All of the imformation that you providing and your helping is really useful and important to me. Thanks for your time and consideration. Best regards. Pham Ngoc. |
Mladen
2009-04-17 |
re: Compiled application exe GUI testing with NUnitForms hi, without seeing some acctual code i can't really help you. |
Romka
2009-04-22 |
re: Compiled application exe GUI testing with NUnitForms Hello together, i am trying to test a VB Windows Forms Application with NUnitForms i although have to start the whole application and not only one Form. now i have no main method, where i could disable the message loop i only set the start Form at the project properties... does anybody knows what i could do? thanks |
Mladen
2009-04-22 |
re: Compiled application exe GUI testing with NUnitForms hi, without seeing some acctual code i can't really help you. |
Kuzin
2009-08-18 |
re: Compiled application exe GUI testing with NUnitForms Hi My application can not read the app.config file with nNunitform testing. I have put the app.config near to the test.dll and as test.config but it still could not read the config. Please help, thanks |
Mladen
2009-08-18 |
re: Compiled application exe GUI testing with NUnitForms the correct confing to read from is the Test projects one. if it doesn't read it i don't know what the problem is. |
Kuzin
2009-08-18 |
re: Compiled application exe GUI testing with NUnitForms I had to copy it near to the nunit dll and same name as the test dll. |
Kuzin
2009-08-18 |
re: Compiled application exe GUI testing with NUnitForms the problem is solved. I had to copy it near to the nunit test dll and with the same name. |
Kuzin
2009-08-18 |
re: Compiled application exe GUI testing with NUnitForms Hi I have problems with nUnitForm testing when the application has AsyncCallback. The structure is: FormMain, FormLogin. First the FormMain opens and after it is finished the settings(database etc.) opens the FormLogin as Mdi childer of the FormMain. private void FormMain_Load(object sender, EventArgs e) new MethodInvoker(InitClient).BeginInvoke(new AsyncCallback(InitClientCallBack), null); void InitClientCallBack(IAsyncResult ar) { try { ((MethodInvoker)((AsyncResult)ar).AsyncDelegate).EndInvoke(ar); if (this.InvokeRequired) { this.Invoke(new MethodInvoker(this.UserLogin)); } else { this.UserLogin(); } } catch (Exception ex) { } private void UserLogin() { FormLogin loginForm = new FormLogin(); loginForm.MdiParent = this; loginForm.Show(); } I can not find the elements on the LoginForm when I use this asyncCallback. If I comment out this asyncCallback and call direct the UserLogin, I can reach the element of the LoginForm. How can I use the nUnitForms in this asyncCallback environment? Thans |
Mladen
2009-08-18 |
re: Compiled application exe GUI testing with NUnitForms for any async stuff we had i either subscribed to the command end event or did a while loop for some period of time. async stuff testing is a real pain. |
Kuzin
2009-08-24 |
re: Compiled application exe GUI testing with NUnitForms Hi I have tried to add some sleep but it does not helped. I have made a small sample application to test the asyncron calls. How can I send you the project? Thanks |
Mladen
2009-08-24 |
re: Compiled application exe GUI testing with NUnitForms you can but i doubt i'll be able to help you. i haven't done any work with this in over 2 years. |
Roger
2010-01-12 |
re: Compiled application exe GUI testing with NUnitForms Hi, First off, thanks for your extremely valuable post. I just recently found NUnitForms but our application has the same requirements as yours, e.g. an EXE that uses many forms. The application I'm attempting to test is not developed in a traditional manner. I am currently unable to use NUnitForms for this application and after multiple hours of failed attempts, i basically concluded that the application's main Form does not appear unless the Main() reaches Application.Run(). This likely has to do with how this app is initializes, and I don't know if it's even possible to refactor it in such a way that NUnitForms could be used. This said, I was wondering if you could confirm my assumptions. Whatever it is that is created in Main (in your example, an instance of Form1), its constructor should properly initialize everything required for the form to appear and run correctly, right? Also, in my case, the equivalent of your Form1 class does not inherit directly from Form, but rather inherits from a couple of classes that eventually inherit from Form. Should this be an issue? Thanks for any help you may provide. |
Mladen
2010-01-12 |
re: Compiled application exe GUI testing with NUnitForms @Roger: we actually had a similar setup. you just have to find a reference to Application.run as this starts the main app message pump and exit before that. your assumptions are correct. Form inheritance isn't an issue. |