Testing performance while testing UI: AKA = The “one time to rull them all” pattern

I know I have some pending stories for you, but I keep seeing something that I think is worthy to talk about. It is a very simple idea, but people usually don’t use it for some reason.

For the newbie lynx: Performance testing is a type of testing that we do in order to measure how well a website responds to the user input and load back an output. Sometime ago, users have to wait minutes for an image to load in their screens, there was no choice, nothing better available. Now a days, there is plenty of competition in services so one of the reasons someone can pick a service instead of another one is the speed of response. Everybody is busy after all, right? Better performance = better quality.

There are many tools available to measure performance and different parts of the application to measure it (server vs client). The developer tools in the different browsers provide ways of checking how long an object has taken to load.

Many companies have specific teams that are in charge of performance testing. I’m seeing this quite often, even if the developers are doing their own tests, someone else is in charge of the performance. However, it is usually forgotten that you can validate and debug some of the performance problems during your UI automation (or even in your unit tests).

I believe this to be a very good practice. As a testing principle, the earliest you catch the defect in the development process the cheaper it is to fix (in terms of resources and time used in regression). How can you do this?

Imagine you have the following code to test that an element exists in a website (this is only pseudo-code for the newbies to understand the concept):

wait(time)

element = findElement(X)

Now imagine that you are doing the same for another object:

wait(time2)

element2 = findeElement(Y)

Ok. If we keep doing this all over our code, we are going to end up with a lot of different times set up for each of the objects. On top of that, if there are problems with the connection or with the servers, we would have to modify one by one the waiting methods or set up really high times. Our code has suddenly become difficult to maintain and we have no idea of how well the application is generally performing.

How can we improve this in an easy way that could gives us an idea of the performance of the site? Very easy, we just use the same waiting time for all of the methods, which we set up in a constant. That way, if we have problems because of our test environments or anything on the likes, we can increase the waiting time for all the objects at once. We can also play around decreasing it so we get an idea of the minimum time a user is going to be waiting until all the objects are loaded in all the pages.

constant myTime = time

wait(myTime)

element=findElement(x)

….

wait(myTime)

element=findElement(Y)

We can also have a different one to load the pages or specific groups of elements (for example iframes), so we have different group of restrictions. Then it is just a matter of defining what is an acceptable time for a user to wait. This will ensure that at the firsts testing phases, all objects load faster than that time. After that, it would be good that you measure the performance of your server regularly and have some communication system if it goes bellow certain threshold. You can still have performance issues because of the server resources, threading issues or management of a high amount of users, but you will catch and easily identify some basic issues by just adding this constant across your code.

Disclaimer: I haven’t found this in any books or anywhere (yet, I am sure I am not the first one in thinking about this). The pattern name of the title is totally made up by me.

Note: usually performance tests run across systems/browsers/devices and so it is considered part of ‘system testing’. But that is…well… another story.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s