Examples of AI applications and how possibly test them

Posted on by noemielis

Recently I attended an online crowdchat hosted by ministry of testing about testing AI applications.

The questions were very interesting, but it was hard to think of a right answer for all AI applications, as this is a very broad field. Explaining it over twitter would be confusing, so I thought I may as well create a post giving some examples.

Kudos to someone on Twitter that mentioned supervised and unsupervised learning at the end of the chat. I was very sleepy at the time (the chat started at 4am my time) so I was not able to find his tweet in the morning to vote for it. I think that we could understand better the types of AI applications that we could have if we divide them in supervised vs unsupervised. More information here.

Supervised learning examples

The idea behind it is easy to understand: These applications have a phase of learning in which we keep feeding them with data and rewarding them if they produce a correct result, while punishing them when they don’t until the produced results match the expected results within a threshold. (In other words, until we are happy with the results).

Let’s ignore for now the exact ways in which we can punish or reward a machine and just focus on the general idea.

After this learning phase, we generally just use the application and no more learning takes place. We “turn off” the learning. This is called inference phase. Not all the applications have inference phase; sometimes we want them to keep learning from the users, but this can turn out to be problematic, as we will see further on.

I think these are the easiest AI applications to test functionally speaking, as we just need to pass new data and check the results obtained against the expected. Apart from this, they behave just like any other application and we can also go through the other types of testing without many changes (performance, security, system..)

NPR / OCR:

Imagine for example a number plate recognition system – once the system learns how to recognize the numbers in the license plate, you don’t have to keep training it. The application can use the learned patterns to verify the new number plates.

There are many tests we could think of here, without caring for how the application gets the results: try with characters that have strange typography (if allowed in the country), tilt the number plate, check the boundary in distance from the vehicle…

An OCR (optical character recognition) application could also be done with this technique. In fact, the number plate recognition system could be considered as a specific type of OCR.

Digital personal assistance (Cortana, Siri, Alexa…):

Quite common nowadays, they help you find out information using voice commands. They could also use supervised learning (although, I believe the right classification for them would be “semi-supervised learning”, but let’s think of them as just supervised for the sake of the example). However, in this case the application keeps learning from the users. It stays in the learning phase.

The reason they can ‘safely’ do this it is because they collect data from the users but not their direct input in whether the result was to be penalized or rewarded. An example of application getting direct input from the user to keep learning would be a chatbot that guesses something and asks if that guess was correct. This could be easily tricked by dishonest users.

Applications that keep learning are much trickier to test, even functionally, as if we pass wrong inputs to test, they will learn wrong. If I had to test one of these, I would use a copy of the state of each iteration we would like to test in an isolated environment, so we don’t break the acquired good learning. For performance testing it would be best to use valid data, to ensure the learning process continues well.

If anybody is concerned about AI gaining consciousness, this type of applications would be the problematic one, as they could be learning things we are not aware of depending on the power that the programmer and the user gave them and the data they are able to collect. This brings up the question: Should testers be responsible to test consciousness?

Unsupervised learning examples

The key of these applications is to discover relationships on the data without direct penalization or reward. They are very useful when we are not sure of what the output should be, and to discover things that we would not naturally think of being related.

There are two types: Clustering (when the system discovers groupings in data) and association (for discovering rules describing data). I won’t go deep on them in this post, as it is a lot of information as it is.

Tailored content-advertising (Amazon, Netflix, Google…)

These apps want to be able to predict what the customers that bought something would be interested on next. In fact, digital personal assistance tools could also use this data to help you find what you want (that’s why I mentioned before they should be classified as ‘semi-supervised’ learning). I cannot think of any ways of testing this except checking on the impact on the sales after the application is in place, but this could potentially be subjective to chance or other factors not related with the application itself.

Apart from that, the test of the application should be the same as we already do with non-AI applications (not just the results, but how the user inputs the data and how the application responds and shows back the data…) Imagine this as a feature of a bigger product, all the other features would also need to be tested as well.

The moral impact of these applications, in my opinion, is that at some point they might be telling you (as a user) what you want, even before you know you wanted it.

What could possibly go wrong?

What should we be careful about in AI that might not need so much attention in other  apps?

Things could go very wrong if we leave apps learning constantly and we leave the users to provide the penalization or rewards. You probably have heard of applications such as image recognition systems and chatbots becoming racists and sexists. Sometimes this is because the test data given to the application is biased, but it could also be because of trolls playing around with the application in unexpected ways and giving rewards when the application is wrong.

Also leaving apps learning on their own is not the best idea, as we do not control what they are actually learning, as mentioned before.

If you are interested, I found an article with some more examples of issues with AI applications here.

What else have you got?

Below is a list of readings that I found very interesting while researching for this post (a couple of the links are about video games and AI):

How “hello neighbor” game’s AI works

AI predicting coding mistakes before developers make them

Examples of AI

Game examples of AI

How would you test these applications?

What do you think about the moral connotations?

If used well, AI could be harmless and powerful. In fact, it could also be a good tool that we could use for automating our testing, but that’s…well…another story.

Interviewing your interviewer (17 tips)

Posted on by noemielis

This topic is probably not only for testers, but, since I have been dealing with it recently, I thought some people might find it interesting. Also, I know, I owe you a couple of other posts (I’m working on them, but I am also moving houses and I have limited time to write right now)

I have been told that I ask good questions at the end of the interviews, so I’d like to share my system in case someone else finds it useful. Tip: the important thing is not to ask questions to impress your interviewer, but to use this time frame to find out things about the job and the company.

I usually get very nervous when I do interviews, even though I have interviewed people myself as well. I find, that a good trick for me not to get so nervous, is to think of the process as a two-ways interview, in which you are also interviewing the company to verify if you really want to work in there.

I know this might be hard to take, especially for beginners: “How would I be interviewing a company that I want to work for? I just want the job, if I didn’t like the company I would not have applied for it.” However, it is important to know as much as possible about the job you are about to be doing for a good while. That is why it is crucial that you ask as many questions as you can in order to understand how they work in that particular team and what is expected of you.

Another point, if you can, is to double check the answers with the interviewer’s reaction to the question. I mean, they are supposed to say good things about the company… imagine how bad they would look if the candidate says something like “well, I need to drop my application because the interviewer told me this is an awful place to work for”. But, if you are paying enough attention, you can see some reactions such as long pauses or struggles, that could lead you to believe that they did not feel comfortable with the question or they are trying to sugar coat their answers.

By now, you are probably thinking that this could be a good advise, but you would like to see actual examples of these questions. I owe you a couple of code samples by now, so I won’t let you down on this one. Be careful, some of them might be in the job description, and asking them might show little research and be so annoying for the interviewer as when they ask you to walk them through your CV (which usually means they have not bothered to read it fully). Below some examples of things you might need to know before joining a company (note, when I do an interview these come naturally to me, it depends on the specific job, these are general examples that I can think of right now):

  1. What process are they following? (Agile, waterfall…)
  2. Would you be joining the team of the interviewer or a different one?
  3. How would you relate to the interviewer in the company?
  4. What technology are they using? (The description usually would mention one or two, but you might ask what would be the biggest used, or for a technology that is not in the description, for example, what they use for source control)
  5. Do they do code reviews?
  6. What’s the relationship between the developers and testers? Do they sit together? Share code? Do common code reviews?
  7. How often do the interviewer…? (meet clients, have team meetings, create new features, spend in paperwork such as performance review…)
  8. How long are the springs? (if using agile)
  9. How many times did the interviewer use *insert benefit* this month?
  10. How do they do the performance reviews? How do they measure performance?
  11. What are they expecting of the candidate?
  12. Is there a possibility of *insert benefit*? (getting bonus, stock, gym membership, learning expenses… this depends on what you are looking for in the company)

Some extra tips:

  1. Try to ask the right questions for the right interview: technical interviewers might not know the answer to an hr questions (for example benefits) and hr people might not have an answer for technical questions (for example technologies they use). You might be wasting their time and not getting your answer anyways, so it is better to save the questions for the right round. (Be sure to learn about the rounds to know when to ask what)
  2. If you can, try to say them in a way that sounds a bit more personal for the interviewer, they are more likely to give you honest answers if you are asking for their opinions than for the company’s protocols. For example, the style of number 9 is more personal than number 12 for knowing about benefits. While number 9 gives you information about the actual behaviour of your co-workers and the non-spoken politics in the company, number 12 can give you room to negotiate a particular benefit that is not usually given (you can use this style with hr).
  3. I have said this already, but: don’t ask them for the sake of asking questions. Think about what it is not clear to you ahead of time and take a pen and paper next to you when you ask them to write down the answers. This can help you not to repeat yourself and to remember everything at the end of the process. It might be something you can negotiate or it might be something for you to discard the company (or the specific team). This does not only give you valuable information but puts you in a more powerful and confident position when doing an interview rather than feeling under test.
  4. Don’t get angry or depressed if you don’t get the job: sometimes it’s just a matter of being lucky and getting an interviewer that connects better with you. Sometimes a company might offer different positions or to work for teams that use different technologies that might be more aligned with your experience. And above it all: if your interviewer is pushing you back for not having the exact same expertise as him/her, you are probably better off not working with that person anyways. I think the trick for a company to work well is to have people with different set of expertise: they might not know something with the same depth as you do, but they might know a lot about something else and you both can learn from each other.
  5. Keep trying and practising: doing interviews might be extenuating but, with enough practice and asking the right questions, sooner or later, you’ll get just the job you want.

Let me know in the comments if you can think of other good questions to ask and let me know if you like these sort of posts about the interview process. I could tell more about it, but that’s… well… another story.

 

 

Automating the automation: Dealing with dynamic objects IDs

Posted on by noemielis

As testers, we sometimes find that developers don’t take testing into account while writing the system and design things that are very difficult for us to test. I’ve some experiences about it that I would like to share in case they help or inspire you.

Localised ID’s

A developer on my team was once tired of redeploying many times to rename objects because of business decisions. He decided to put into place a system to store the objects so people from localisation and internationalisation could change the text themselves without need of redeployment from his side.

It sounds like a great idea, but the issue was that the id’s were also on that table, and the people that were on charge of translating the text, would also translate those ID’s without understanding what they were for.

What was before me, was a set of localised pages that “most of the times” would work.

In this case, the solution I did was to check the developer’s code to understand how he was retrieving those objects and do the same from my code.

Automatic object creation

Another case of dynamic objects Id’s that I’ve seen is when the developers would create an automatic number of items dynamically on the page. In this case, the objects were created always with id’s that didn’t really identify what object they were, but would follow an structure.

For example, if they populate a list of users based on an input, each of them could have an id on the sort of “id_1, id_2, id_3…” (at least they had id’s).

Before me, they were doing manual tests because “it was not something we could automate”.

For this case, what I did was searching for id’s in the page. Something like this:

boolean caseExit = false;

int i=0;

while (!caseExit)

{

if (getObject("ID_" + i++).exits)

// do something

else

caseExit = true;

}

The “do something” part could be many things. For example, we could check that all the users had a certain property.

If your “exists” throws an exception you might need a try-catch instead of if statement in here.

An important note: do this if you are not able to retrieve the objects somehow else. For example, if what you have is a table you can easily access the different rows as long as the table has an id (you don’t need ID per row). It could also be similar if this is an html list.

The problem is when you have a serie of new unrelated objects. Moreover, if this is the case for all the objects in the page (they auto populate them all somehow), maybe auto-creating the page would help better for the case (explained on my previous post).

Angular/react dynamism

With introduction of JIT (a great article explaining here: buff.ly/2qFL28g) the browsers could start handling more dynamism in their websites. AngularJS (insert increasing version number here), ReactJS or VueJS are examples of frameworks that allow for this to happen.

But, as these frameworks started to get popular, some other tools were also created that would allow us to deal with this new dynamism. For example, AngularJS team created Karma, which is a NodeJS application that allows you to input your tests in the command line and aligns well with Jasmine and others tools for testing.

For end to end, you could check tools like protractor, nightwatchjs and testcafe.

There are many frameworks, extensions and customisations in the open source community. They are starting to move almost as fast as the front end tools (The frustration on front end is very nicely explained in this post: https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f )

Each of them would be tailored for a particular case scenario, so if you are building a framework you need to do a good research first, and ask many questions to your team.

POM for dynamic objects

Many people start forgetting about page object design pattern when they start automating on dynamic objects. However I would recommend you still incorporate it as much as you can because it really helps maintaining a clean code and reducing the time to write it.

Even if you have a lot of tests, you usually don’t have the time resources to execute every single one of them. For this, you need to decide what test to run, and this could also be a difficult, repetitive and automate-able process. But that’s…well…another story.

Automating the automation

Posted on by noemielis

Have you ever find yourself writing the exact same basic tests for certain objects over and over again? With the only difference on the id or path or even the element retrieval?

Wouldn’t it be nice to have some way of doing this bit faster? Automation is about making your computer do repetitive tasks for you, and this can be applied to everything, including, yes, writing code.

The process to automation is as follows:

  1. Identify repetitive tasks
  2. Write code that would do these tasks for you
  3. Identify when the code needs to be executed
  4. Execute the code or automate the execution

In order to give an example or something being repetitive during automation, I would like to introduce the page object model. If you already know about this, maybe you should skip the next section.

Page object model

The page object, in short, it’s a model for organising functional testing classes and objects so the code is cleaner and easier to modify. These sort of models are also known as design patterns (note to any developer reading this: yes, there are design patterns for testing and you should be using them!).

There is plenty of information about POM (page object model), starting on selenium website (http://www.seleniumhq.org/docs/06_test_design_considerations.jsp#page-object-design-pattern), so I am just going to summarise it here.

In POM, we have (at least, depending on the implementation) two different types of classes:

  1. A class (page class) that would gather all the information of the objects that we need from a website’s page (for example, a button or a text box) and micro-methods to preform common tasks (such as click, type, select, verify existence…). If you are very newbie, just open your browser’s developer mode (usually f12 does it) and using the selector hover over the website to see the objects that it has and their associated values.
  2. Another class (model class) would implement the more sophisticated methods (test cases) using the objects and methods exposed by our page class.

In this way, if the website changes layouts it would be easier to retrieve the objects again without the need of finding every instance of that object throughout the code.

1. Identify the repetitive tasks

So, you need to extract some of the elements on a website page, and add small known methods… We have found our repetitive tasks!

However, our job is not done here. There are still many things to take into account for automation:

  1. The tools: What tools are the developers using? What programming language? And how are we going to design the system in respect to this? We should, within possible, try to align with the development tools, but sometimes this is not the best choice. For example, imagine that developers use both javascript and ruby to build a website, but that you have a team full of knowledgeable test engineers experienced in Java, shall we train them and align with the developers or take advantage of their current skills? This would have to be taken case by case.
  2. Dynamic programming: Would we need to extend our framework to support elements that would be refreshed in the screen without fully re-loading the website? (Nowadays, you most likely would!)
  3. Number of elements/iframes: If we have a lot of iframes, or elements in the website (such as many nested divs), but we only need to access certain elements, we might prefer to write an automated solution that allows us to input the elements that we want. However, it could be possible that we want to define everything on a page automatically, because that takes away one manual step while keeping a reasonable load.

2. Write code that does the tasks for you

To give an example and in order to simplify things, let’s say that we have decided to create a solution in Java, that there is no dynamic loading and that we are going to say which elements and properties we want rather than getting them all.

The idea of the code is as follows:

Allow input of a list of elements, the type, the selector type and the selector for each of them. We could use a csv file for this, for example:

name= loginbox, type=input, selector=id, id=loginbox;
name= password, type=input, selector=xpath, id=…;
name= loginButton, type=button, selector=name, id=loginButton;
name= textSuccess, type=textBox, selector=text, id=success;

  1. For each of the lines on the input file, create an object with the name and selector given.
  2. For each of the lines on the input file, create a method for the object Type:
    1. For input: type method
    2. For button/href: click method
    3. For textBox: method to verify test
    4. For all: exist method

It would look something like this:

// retrieval of the fileLines left out for exemplification
for (Object o : fileLines) {

switch(o.type) {

case "input":

toPrintInPageClass += "WebElement " + o.name + " = driver.findElement(By." + o.selector+ "(\"" + o.id + "\"));\n";

toPrintInPageClass += "void " + o.name + "_type(string toType) { \n\t" + o.name + ".sendKeys(toType); \n}";

break;

case "button":

...... (you should have gotten the idea now)

This should take less than an hour to build, but after that, every page would be built in no time and the code would be less prone to errors as opposed to copy-pasting the same code over and over again. (You write one class as opposed to one per page)

An additional benefit is that if you have a manual tester in the team, now you can assign him or her the definition of the inputs and execution of the code. And you just made that person able to “program” with this “very high level programming language”.

Bonus: After this is built, we can create another class that would extract the items from the website into the input file for this class. That way we can retrieve all the elements in one go without human intervention.

3. Identify when the code needs to be executed

What happens if a developer changes the website and adds an element? Shall we execute the whole thing again or just add manually the missing element?

Even if it is tempting to add the missing element manually, I would suggest to add it to the input file, otherwise if someone decides to execute this code, your element would be missed. It is likely to still be faster to execute the code than to add an element manually.

But what if it is not? What if we have so many elements at the moment than executing the code can take longer than adding just one more?

I would still run the code rather than add elements manually, because it could as well be that some of those elements do not exist anymore. But if it is only one quick change, please remember to change the input file too.

As an addition, we could add into the code functionality to modify the class already created rather than create a new one, but I’d say this could be a lot of overhead for the benefit that you can get out of it.

4. Execute the code or automate the execution

Lastly, especially if you have the object retrieval automated too, you might want to automate the execution. For example, you can say to the computer (using a cron file could be a way): run this code every evening at 8pm. Or once a week… once a month… That way you could “completely” forget about the page definition side of the POM and just focus on the functionality.

Alternatively, if you have a way of retrieving the objects from the website, you could check that the original input is the same one as the newly generated and only execute the page class creation if they are different. That should be faster to run and it would allow you to change the actual code only when required.

However, be careful with this approaches, because you might miss when you need to add new functionality.

This is quite common in automation: you need to keep an eye on it. After it is built, it usually needs maintenance and refactoring. The better built, the less maintenance, but that does not mean 0 maintenance.

Conclusions:

Many sources would recommend to make short and simple tests, with the idea of identifying the failing parts easily and clearly. Nonetheless, sometimes you can spend longer in creating these simple tests than in actually testing the parts that are likely to fail, so the right balance is important.

What we’ve looked at today would help creating many simple tests, but you still should be careful not to over-test things as sometimes it could be too expensive or not really needed.

On the other hand, this could be a difficult task to do if we find with dynamically created objects or/and dynamically assigned id’s/properties. There are ways of dealing with these as well, but that’s…well…another story.

My views in the future of testing

Posted on by noemielis

As you might know by now from my previous posts,  I’ve always been between testing and developing.

Every company is different and every person in the team is different. Some developers highly respect testing, write tests or at least like to learn to think like testers. There are developers that use testers as a shield against management’s disappointment when something goes wrong in production.

Some testers know how to code and are interested in new languages, technologies, systems… others might know some programming but just enough so they can stay in a comfortable job (this could apply to the devs too). Others are great testers although they do not know how to program (and that should not be an issue or be taken lightly by the rest of the team).

I have noticed a lot of confusion and open interpretations on the testing side of development and I have seen many movements on that side: I have seen some companies moving from QA to SDETs, and others moving from SDETs to Software engineers. It seems like this is the new fashion, the process to follow, developers to do both dev and test tasks (and operations as well, if possible…).

I have been concerned, to be honest, that I might end up in pure development and I would be fully extincted.. I felt as endangered as my lynx specie is.

The introduction of TDD and then of BDD, makes companies ignore other types of testing (maybe with exception of security) in favour of deploying earlier and faster, thus leaving the end to end ‘testing’ directly to the users.

However, I think this is not the right approach. Don’t get me wrong, I was already developing and testing, I would have ‘no issues’ moving with this current. But at the end of the day, if nobody is in charge of the quality, why would anybody care about it?

If previous testers are either gone or converted to developing first and thinking about quality second.. at some point, after hiring more and move developers and promoting thinking about speed of development first, who is going to think about quality? Who will have the knowledge to make a point or call out missing scenarios?

This might as well be a cyclic fashion, and at some point companies would be needing a test expert. Someone that cares about quality and creates integration and system testing, checks that the unit tests are right, verifies which tests are going to be run in the different builds, creates other types of testing, researches about testing, architect solutions for automation… Or at minimum someone that can train the developers to check for the right things and ensures the right testing happens.

How can you get test experts if there are no testers anymore? At the moment, the answer is: from other companies that are still using testers… but what if there are no such companies anymore?

I’ve heard complaints about how hard is to find testers that also program or have development qualities. Or developers that are OK being called testers and undertaking tasks in the test system. Maybe this post helps someone understand the paradox.

So, if we don’t find testers.. is it ethic/safe to deploy without comprehensive end to end and allow the users to report the issues in order to speed up development? To me, it is not, although I can understand where this comes from and that in some environments it’s very hard and costly to find alternatives.

Will this be the new quality assurance culture? I don’t think so, but that doesn’t mean that the things will go back to the way they were.

Should I be scared of losing my job? Well, that highly depends on your company, so I cannot answer that in an easy way.

What can I do to make sure I will stand out? I’d say, try to make yourself valuable, just as with any job. And I do not mean writing spaghetti code, (although high respect to the pastafarians, arr!) I mean knowledge wise.

I think there is much more to come. Research to be done. New ways of automating, new processes, development, test expertise …

And that’s why I think the future of testing is uncertain, but it will be different to what we have known until now. It has started to change and I think we better take on that boat and participate in the change that let us be driven by it.

Where to start? We should start by automating everything, including the automation. But that would be… well… a different story!

Between two rivers

Posted on by noemielis

As a developer, I often find myself stuck in a problem and I rarely get the freedom to switch my mind to a different task until I solved the first issue. But I need to be able to switch tasks, that’s how my mind works.

I read somewhere that this is the trick to creativity: just do something else that is more automatic and then go back to the hard problem. While only developing, I find it hard to work on this ‘all or nothing’ way, it does not feel like I am creating, it feels like I’m forced into making the features.

Testing, for most companies I worked for, was at the exact same technical + programming level as the developer role. By this I mean: you needed to be a developer if you wanted to get the position and you get the same interviews.

At first, as tester, I felt unchallenged, as if I was just repeating tasks…and I do not mean manual testing, writing some tests automation could be repetitive as well. But that was until I learned that I could not only automate the tests, but also the processes. So I get more job done faster! I could do things in parallel, running tests that will show my results while writing code that will clear up my to do list… That’s one of the reasons that I love it, besides the fact that you get to think out of the box and create new ways of solving and proving issues.

If you are curious about different roles across the two disciplines, here are the names of the roles I’ve done and what type of work they involved:

  1. Software developer: This is used as a generic term for someone that can write code (programs) and can also design technically the new features (sometimes with the help/agreement of other team members). There are many different types of developers and the number of positions are increasing and evolving with the years: front end, back end, full stack, devops, mobile, VR… And now you can find people really specialized in a particular language, so if you are starting on a position, make sure you like that language.
  2. System verification tester: In my case, it was similar to software developer, only that instead of working with databases, back end, front end or applications, I was working on automation (and so with a bit of it all as well). You also need to be able to think out of the box and come up with tests that would help demonstrating quality. In this position, I, in particular, did not get to talk a lot with the developers of the system we were verifying, but it was common to check with the customers to make sure their quality criteria was met. There was also a degree of performance testing there. Basically, we were making sure the system was working across different platforms and specifications by writing code to check it.
  3. SDET (software developer engineer in test). I really like this role, even thought it sounds as if I you are a ‘provisional’ developer, rather than a developer that works in the test side. I like that it has the word ‘engineer’ on it, which I deserve (I have two engineering degrees). In here, SDETs work in the same sort of things as the SDEs.  The only difference would be that while the SDEs work in making the code for the actual features, SDETs work in making the tests for those features and verifying they worked. Code reviews were done across disciplines making everybody familiar with all the code.
  4. Software engineer: This is a recent enough title. If you see this, it means that in the company the developers also do the testing, usually switching between the two roles, or as part of BDD or TDD and with faster releases. I like this role as well, although quality tend to get lower priority in favour of faster releases and I particularly believe it is better to have someone in charge of quality so it does not get forgotten.
  5. Software QA lead: ‘Developer’ and ‘engineer’ words are out of this name, but, I still got to do some degree of developing when I had this title. However, not as much as I wanted to get in this particular position, but I suppose that depends on your career goals and on the opportunities on the particular company to meet them. I did enjoy the leading part though, more below.
  6. Software engineer lead: Same as number 4 but also leading. I am mentioning it because I had the wrong assumption, when I was a younger lynx, that the leaders did not get to do coding anymore. I am expected to do as much as the others, but I also have other leading tasks and have to keep an eye on the team. It is hard, but also quite rewarding because you get to help others and inspire them.

I want to get the ‘developer’ label as little as the ‘tester’ one. I’m neither. I’m both.

Especially when you start labeling developers by their expertise. Could I be a front end developer? I could, but I’m not really. A back end? Even less… I’m sort of a ‘all in one’ and ‘none of the above’ at the same time.

I find it interesting that the developer types are becoming new positions, while some companies are moving away from the SDET title (which it’s a developer type).

Dev and test positions keep evolving and changing across companies. How would testing be done in the future? That would be… well…a different story.

The beginning of the test lynx experience

Posted on by noemielis

I maintain my belief that anybody can become a tester (maybe all they need to do is to change their hats). I also think some people are born testers. This was my case, only I did not know it when I started my career.

I think you could easily identify future testers when they are kids if you observe their behaviours. For example, at the beach, is the kid building a castle? Possibly..dev. Is the kid destroying someone else’s castle (hopefully when that kid is away) or pointing the defects of the other kid’s construction? Possibly.. test. What if the kid first build the castle and then proceeds to destroy it? Well, that would be my case, you would understand it better later on.

When I was about 9 years old, I had an old computer, one of those that you plug to the TV and run on basic. It happened to come with a book for basic (or my parents bought it?). The thing is that I wanted to go further than the games I was playing with and I started to learn how to write some code.

Being able to write games myself opened a new door to wonderland for me. I was still curious to see how the games I was playing with were written, so I learned a very useful command: ‘LIST’. That was when the magic really started: lines and lines of beautiful code that I would have not been able to put together so far, but, yet,  I could understand.

Some of those games took very long in loading and sometimes they would fail (‘Syntax error’ were the worst two words I could read back at those days.)

It turned out that the games were loaded by sound, and sometimes the cassette could jump and eliminate bits of code. Sometimes I was able to fix it, which would mean not to wait again for 2 hours (hurray!).

Years later, I did learn some concepts about testing during my computer science degrees, but it is usually something only described briefly as part of other courses (maybe the testing community could do something about this?).

When I started working, I first joined as a developer (I did not even know there were many other choices). It was fun to build those sand castles, but something was wrong… I was not satisfied, I needed something else. But that is…well… another story.

Lynx under recovery

Posted on by noemielis

Hopefully you have noticed I have been away for a while now… that would mean you were following the posts and found them somehow interesting or helpful and I am grateful for that.

I have still many, many stories to tell and projects to explore with you…but sometimes a lynx gets too focused on hunting (projects at work, personal commitments, courses, last minute new year’s resolutions…) When this happens, little time is left to tell stories to other lynx, but, at the same time, many other potential stories are being born.

So the goal of this post is just to let you know that the test lynx is still out there and soon to publish some more stories.

However, there would be a slight change on the way the stories are told… as you know, I always hint about my next story on the previous one. This has been proven to be not quite sustainable… So, I will eventually tell the hinted story, but, I might tell an unrelated story next with the goal of speeding up the posting and not to block potential ‘out of the oven’ stories.

At the end of the day, even though I have a great passion on automation and testing, I can only stretch so much… I am only a lynx being.

Now that we know each other for some time, you might be interested in knowing more about my own story…but, that is, definitely, well…a different story.

 

 

 

 

Testing performance while testing UI: AKA = The “one time to rull them all” pattern

Posted on by noemielis

I know I have some pending stories for you, but I keep seeing something that I think is worthy to talk about. It is a very simple idea, but people usually don’t use it for some reason.

For the newbie lynx: Performance testing is a type of testing that we do in order to measure how well a website responds to the user input and load back an output. Sometime ago, users have to wait minutes for an image to load in their screens, there was no choice, nothing better available. Now a days, there is plenty of competition in services so one of the reasons someone can pick a service instead of another one is the speed of response. Everybody is busy after all, right? Better performance = better quality.

There are many tools available to measure performance and different parts of the application to measure it (server vs client). The developer tools in the different browsers provide ways of checking how long an object has taken to load.

Many companies have specific teams that are in charge of performance testing. I’m seeing this quite often, even if the developers are doing their own tests, someone else is in charge of the performance. However, it is usually forgotten that you can validate and debug some of the performance problems during your UI automation (or even in your unit tests).

I believe this to be a very good practice. As a testing principle, the earliest you catch the defect in the development process the cheaper it is to fix (in terms of resources and time used in regression). How can you do this?

Imagine you have the following code to test that an element exists in a website (this is only pseudo-code for the newbies to understand the concept):

wait(time)

element = findElement(X)

Now imagine that you are doing the same for another object:

wait(time2)

element2 = findeElement(Y)

Ok. If we keep doing this all over our code, we are going to end up with a lot of different times set up for each of the objects. On top of that, if there are problems with the connection or with the servers, we would have to modify one by one the waiting methods or set up really high times. Our code has suddenly become difficult to maintain and we have no idea of how well the application is generally performing.

How can we improve this in an easy way that could gives us an idea of the performance of the site? Very easy, we just use the same waiting time for all of the methods, which we set up in a constant. That way, if we have problems because of our test environments or anything on the likes, we can increase the waiting time for all the objects at once. We can also play around decreasing it so we get an idea of the minimum time a user is going to be waiting until all the objects are loaded in all the pages.

constant myTime = time

wait(myTime)

element=findElement(x)

….

wait(myTime)

element=findElement(Y)

We can also have a different one to load the pages or specific groups of elements (for example iframes), so we have different group of restrictions. Then it is just a matter of defining what is an acceptable time for a user to wait. This will ensure that at the firsts testing phases, all objects load faster than that time. After that, it would be good that you measure the performance of your server regularly and have some communication system if it goes bellow certain threshold. You can still have performance issues because of the server resources, threading issues or management of a high amount of users, but you will catch and easily identify some basic issues by just adding this constant across your code.

Disclaimer: I haven’t found this in any books or anywhere (yet, I am sure I am not the first one in thinking about this). The pattern name of the title is totally made up by me.

Note: usually performance tests run across systems/browsers/devices and so it is considered part of ‘system testing’. But that is…well… another story.

 

An unexpected story

Posted on by noemielis

This is not the story you were waiting for. I was supposed to continue talking about testing methods with basic examples. However, I need to tell a different story. I want to tell you the story of the “Test Bash”, my first test conference.

I heard that attending conferences could do good to you and your career. I thought if they exist, they would be specific and far away.

One day I read about the test bash. The content looked interesting and on my field, but I thought it was a lot of money to spend to attend, even thought it was happening in the uk and that is a short trip from Ireland, where I live.

I am contracting now, so everything needed to come from my own expenses. However, I was extremely lucky and they decided to do some discounts for people like me. I was still not sure whether to go, it was the last ticket that I purchased.

And there I was: suddenly on my way to Brighton, no idea where to go from the train station, and knowing nobody there… Finally, an adventure!

First of all, I started looking at all the meetups relating to test happening in Brighton in the time I was there. There was one just after I arrived. So I went to this pub, directly after checking in the hotel, introducing myself to total strangers, expecting to be in a corner trying to eardrop some interesting conversation … But to my surprise, I was immediately taken in and involved in many of those, just as the regulars.

The three questions:

There were three topics I brought up often in the conversations:

1) Do you know of others test conferences worth it to go?

2) What do you think of certifications?

3) Should a tester know programming?

The first one I asked because I started reading about other test conferences and they are usually even more costly than the test bash. I wanted to know other’s experiences to discard or consider further conferences.

What I learned? First of all, my lynx, I was attending to the probably better value/money balanced conferenced in Europe. EuroSTAR is the most known conference in Europe. Ireland have a small community of testers and last year the EuroSTAR was hosted in Ireland! (introduce feline facepaw here)

I also heard about other conferences worth the time and the effort: expoqa (Spain), let’s test (Stockholm), Copenhagen context, Pipeline conference…

The second question I was interested in, had to do with certifications. I do have a good working experience at this point, but some things you cannot get on some jobs and I like learning new things constantly. Unfortunately, most of the people who I spoke to agreed in that if you have good experience, test certifications are irrelevant. It is good to have the istqb if you are just starting in test but other than that I couldn’t find out a certification that everybody was happy about. Interesting. Also, for developers there are plenty of certifications and courses available but the options for testers are quite limited. Interesting again, I’m setting as goal right here and now to dig deeper into this.

The third question, I liked to bring it up because I come from a developer and engineering background and I should be able to go back to it. I wanted to know people’s opinion about testers that know programming or come from an engineer career. To my surprise (and a little of let down for a short while) most of the people I met told me, in different words, that I was overqualified. Even others that came from my same background seemed to have gave up all relationship with code. On the other hand, some people really got interested on this fact about me and most agreed that it could be of advantage to at least know programming. There is still hope.

A handful of people agreed with me in that we need more people with coding capabilities and background and they keep up to date with coding. It is not my intention to scare off anybody without a career in IT, but to bring the attention of people like me that worked hard to get one and doesn’t go into the  testing field because they think they might be overqualified. Not the case, you can choose to do whatever you want to do. Therefore, if your passion is testing, why not to work towards a career on it? Is it peer pressure? Is it about money?

I learned something from my experience: if you like what you are doing, you will be doing a brilliant work. If you are doing a brilliant work, you will be naturally well compensated economically and you will probably not even care about that!

The workshops:

In relation to the workshops and conferences: I wish I could have done them all, but I had to choose between them. They all looked very interesting. Some of them brought more value to me than others, but all of them brought even more interesting conversations.

I learned many things and got many concepts to think about. Some key concepts and tips below:

  • Talk always with all the parties in the development and understand their definition of quality and testing to alienate it with ours.
  • Quality is a team commitment.
  • Translate technical things for the none technical people.
  • Translate non-technical things to the technical people.
  • Setting expectations before a meeting keeps people focused.
  • Job titles are important. I’m going to call myself “Lynx trainer”!
  • There are many places to practice testing, paid projects and live bug bashes. I promise to go through them and talk about them in a separate post.
  • Ask questions.
  • Ask many questions.
  • Ask more questions.
  • Be impeccable with your words.
  • The shorter the releases, the more chances to success and easier rollbacks.
  • Companies change but if you are passionate about what you do, you will always find ways of keep doing it.

There were also many books and topics I should look deeper into. I have been working on all that but I didn’t want to slow down the release of this post even longer and forget about the experience and the feeling.

I could be talking about all of the workshops with detail, but there are many good experienced bloggers out there that are also faster than me and you could learn more about it through them. Besides, those are, well… other stories.