Mladen Prajdić Blog

Blog about stuff and things and stuff. Mostly about SQL server and .Net

Automated GUI testing - Is it worth it?

In 2007 I've worked a lot with NUnitForms. It contains all major controls testers and you can easily create

your own ones for e.g. a Data Grid or a 3rd party control. I had to create one for Janus GridEx and it wasn't really

that hard. A little reflection went a long way :) Lately, the whole project was at a standstill for some time but now it seems

it's revitalizing nicely with new people on board. So if you haven't yet go take look. This is just one open source GUI testing platform

and I'm sure there are others... I haven't checked them out though.

 

Now to the big question: Is it worth it?  It depends...

I'm not a hardcore TDD practitioner but I like having repeatable unit tests. But having automated GUI unit tests is in my opinion a waste of time.

Why? It simply isn't worth spending time writing them. You should have enough code coverage with non GUI tests to find any bugs long before

hitting the GUI. You might want to be tempted to unit test control positions or visibility or some other property... don't bother.

You'll spend your time better if you write more non GUI unit tests and check your GUI by hand (eye).

So what are they useful for? Well... I've used them for acceptance and integration testing with great success.

 

Integration Tests...

... test how different parts of your application work together. These kinds of GUI tests proved to be VERY useful and they are worth investing time in.

They have a great ROI (return on investment) once they're written.

For example we had a large workflow based application that was completely async in nature. It had a windows forms client that could be deployed on

multiple computers, all those clients connected to a single server that hosted a few webservices for data modifications, retrieval and some other stuff.

The whole app was also connected to Sharepoint server,  Exchange Server, SQL Server, used InfoPath and a 3rd party software to export data from

and into it. With one single GUI integration test we could easily see if everything worked together as it should. We mocked the InfoPath and

3rd party software but everything else ran great.

The point of using GUI tests here is to see how and if your whole application works with all parts integrated together from GUI interaction

to Exchange connectivity. Even the best non GUI integration tests simply don't do that.

 

Acceptance Tests...

... test if your software is up to some specs usually defined by the business process. They're in essence a black box test.

You don't care what goes on inside the software you just want to see if the output is correct for a certain input.

In workflow based processes this can be very useful since you know what the input and output parameters are for your workflow.

For example you start the workflow in your app via NUnitForms, pressing "move in workflow" buttons until the workflow end and then check if

the output parameters have the correct values. You can check for various different GUI parameters in this test along the way but as I said above

don't go overboard. I usually just check for visibility of certain panels or grids... simple things like that.

 

 

When I first started with GUI testing I thought it was a really big waste of time. And to tell you the truth if you don't draw the line somewhere it is.

I even had a moment when I thought it was a good idea to test everything only with GUI tests. That was so far over line I couldn't even see it anymore.

Luckily that moment passed pretty fast. Don't make the same mistake. If used wisely automated GUI tests are a big benefit.

 

kick it on DotNetKicks.com
 

Legacy Comments


Bruno 'Shine' Figueiredo
2008-01-04
re: Automated GUI testing - Is it worth it?
I don't believe that UI testing is a waste of time. Testing if a button is visible is as important as any other business test. I say this from personal experience. Some of the webforms we developed have an extensive UI business rules that must be tested.

Jay Ruuska
2008-01-07
re: Automated GUI testing - Is it worth it?
Well, and don't forget that many organizations require software to work in multiple languages, currencies, etc. If you are not testing the GUI (validating the value of the display) then your risks are very high for serious anomalies that can block user functionality -- and if you are not automating those GUI tests, then your costs are enormous.

Mladen
2008-01-07
re: Automated GUI testing - Is it worth it?
@Bruno:
i agree with you up to a point. I'm not saying you shouldn't test GUI at all. i agree on testing visibility. and try decoupling the business rules from gui as much as possible since this will make your business rules more testable. binding busines rules to a gui clearly violates GUI and BI layer separation. or am i misunderstanding you?

@jay:
have to disagree here. the app i was working on is in english, german and french.
we never tested the texts them selves. the proper way to do this is to have a localized resource file in which you hold translations. then you can simply test those. and not wasting time doing it through GUI.

Bruno 'Shine' Figueiredo
2008-01-07
re: Automated GUI testing - Is it worth it?
In my case what happens is that, for instance, in a webform, a button can only be visible if two textboxes are filled and the values on those textboxes are valid values. These validations are UI business rules. This UI must be tested. Today I was talking to a collegue and we here trying to figure out if this automating tests where uses cases or unit tests.... what is our opinion by the way?

Mladen
2008-01-07
re: Automated GUI testing - Is it worth it?
i'd say use cases. we do test common validation rules, but not nearly all combinations.

magiser
2008-06-27
re: Automated GUI testing - Is it worth it?
How did you implement the control tester for a janus control. Can you explain me, or give some link to read? thanks!!!

Mladen
2008-06-30
re: Automated GUI testing - Is it worth it?
well if you would have left an email i'd send it to you... but since you haven't.

magiser
2008-07-01
re: Automated GUI testing - Is it worth it?
Thanks, I was expecting for an answer first. my email is magiser@gmail.com

Jhon
2008-07-08
re: Automated GUI testing - Is it worth it?
Hi, I´m add some Janus Controls to NUnitForm ( EditBox, UIButton, UIRadioButton and CheckedComboBox ) but it doesn´t show the properties when I write the test, for example: editBox1.Properties.
Do you Know Why?
Tanks

































Jerel
2009-04-09
re: Automated GUI testing - Is it worth it?
Hi, I'm interested in using nUnitForms with janus controls as well. Can you let me know how you did it too?

Thanks

mikail
2009-04-13
re: Automated GUI testing - Is it worth it?
Performing GUI tests is an important part in creating high quality products.
Is not enough to have big coverage on unit testing, you need also to check the logic of user interface.
I can enumerate here several cases when having automated GUI tests helps:
- check the tab index order of controls
- check changes in form state (e.x. enabling/disabling/hiding graphical objects)
- check dialogs open
- check message box and its text when some errors are catched
These and many others are vital for a QA acceptance.
There are several good frameworks which can be used to perform such tests. I can enumerate here 2 of most used: NUnitForms or GUI Automated Tests from crom-osec.
Using a good framework to perform automated tests helped me many times, especially on maintenance phase, when some changes in the code were producing unwanted changes in user interface.

Mladen
2009-04-13
re: Automated GUI testing - Is it worth it?
@mikail:
you're talking about acceptance and integration testing. i'm all for that.
unit tests are a different thing.