We need to talk about quaternions…

I know, I know, this doesn’t seem like anything that has to do with testing. However, I found this concept to be very challenging and I think some people might be interested in knowing about it, especially if they are considering automating objects on VR.

The problem:

You want to be able to rotate the main camera with Google VR. Google design has reached the conclusion that moving the camera in virtual reality should be something that the user does, not the developer (I think it was possible in earlier versions of Unity and Google VR sdk).

So I decided to create an empty object and assign the camera as a child object of it. To move the camera, I move this object. To undo the camera movement done by the user, I move the object the opposite way. Should be easy, shouldn’t it?

Well, sometimes… but other times it does not work. I spent a very long time searching online for other people that had this issue and it was either working for them or not, nobody would explain why this was the case. I believe the reason to be the effect of gimbal lock.

I know, that sounds like a made up word, but this is actually an important concept that you should be aware of, let me explain:

Gimbal lock is an effect in which the object loses one degree of rotation, which is common when you use three dimensional vectors. At some point, two of the rotation coordinates get lined up in a way that every time you move one, the other one moves as well. If you are a visual person, this video from GuerrillaCG (a Youtube channel with information about 3D modelling and animation) explains it quite clearly.

How did I understand what this mean and how did I figure this was the issue? I decided to encapsulate the camera object in another object, and then that object in another one. Then I assigned them some colored spheres and pointers and I run some tests. After a while, it became clear that movements were not coordinated and that the rotation variables were not working as expected.

Attempts to understand 3D issues

Introducing quaternions:

The difference between Euler vectors (the usual 3 components vector that indicates a point in the 3D space) and quaternions (a 4 components vector) could help us avoid the Gimbal lock effect.

Quoting wikipedia: ” In mathematics, the quaternions are a number system that extends the complex numbers. […] Quaternions find uses in both pure and applied mathematics, in particular for calculations involving three-dimensional rotations such as in three-dimensional computer graphics, computer vision,[…] “

I know, it still sounds a bit gibberish… Basically (very basically), quaternions are a mathematical way of representing 3D objects by using 4 coordinates (representing them in 4D space instead), that an Irish mathematician (called William Rowan Hamilton) came up with and decided to literally set in stone, on a bridge.

https://upload.wikimedia.org/wikipedia/commons/a/a2/William_Rowan_Hamilton_Plaque_-_geograph.org.uk_-_347941.jpg
Commemorative Hamilton plaque on Broome Bridge. From wikipedia

They are difficult to understand and to visualise (as we don’t intuitively know how a 4D space should be visualised), but the maths applied to them are easier than to the 3D vectors. For example: if you want to rotate an object on an x plane, on the y plane and on the z plane, the object will not be in the same state if you perform the entire rotation on x and then on y and then on z, that if you do it in a different order (because each coordinate rotates based on the others too). However, with quaternions, you will always get the same result, as you can rotate two planes at once (which is particularly convenient when you are trying to undo a movement or automate a 3D object)

I am linking to a lot of articles below for you to deep dive, but two main points to remember are:

  1. You should keep one of the values of the quaternions fixed (or set to 1) as this is the way of telling the quaternions that there has been a rotation.
  2. Rotation angles in quaternions are half the expected in 3D rotation (so be careful when calculating the rotations in 4D). This is known as double-cover and it’s actually quite useful once you get the grasp on it.

If you are interested in knowing more about quaternions, you can check this and this videos from 3Blue1Brown (a Youtube channel full of interesting mathematical concepts with a very easy explanation, I really recommend it) Also, I enjoyed the explanations on this article. If you are considering working with 3D objects movement at some point, you should definitely watch these videos and play around with the simulators to understand everything better.

Conclusion

It is not my intention to go against the design of a system such as Google VR, and you should listen to the platform owners as, in general, it is wise not to tinker with the camera. However, sometimes I find it to be useful to undo a user’s movement, for example, for automation purposes or if the gyroscope is drifting on its own (more about phone’s sensors in this interesting article).
In these cases, the use of quaternions is so fundamental, that it is definitely worth it to spend sometime learning about them.
The next step after the automation of the camera and other 3D objects would be to automate the hands movements, but that’s…well…another story

How do I get into…? guide

One of the most common question I get when people reach out to me about virtual reality (VR) is: how could I get started? Even thought I already have written an article about this, but maybe I should talk about my own experience instead so you can get more specific examples about it. If you are not into VR, please bear with me, as what I’m about to tell you can help you getting into any topics you wish to get into.

My Story

My first experience with a VR device was during a hackathon in Microsoft, when one of the interns brought his Oculus Rift. Back then it was a very expensive device, so it was very interesting to be able to play around with one. But I found that it would still have some issues to solve, starting for adding hands gestures.

As life goes, sometimes you get stuck in what you are doing at work and don’t get the time to investigate interesting new things. In my case, I bought a house and there was a lot of stress related to this. It was not until years later that I actually got the chance to try again another device, this time it was over mobile on a meetup called “tech for good” in Dublin. In this meetup, they were using VR mobile devices to provide social impact. It was my first experience with phone VR and I thought: Ok, now this is something that anybody can use and get, therefore it is something that is going to need testing.

After that, another hackathon (this time an open Nasa hackathon) got my interest in VR and AR back. I highly recommend this hackathon as I made really good friends there and we had so much fun building a AR/VR experience to navigate a satellite. My team (who won the local People’s choice award in Dublin) created an application that simulate a satellite around the orbit (on AR) and translate to see the view from that satellite (VR). If you are interested, here is our project

When I found myself having more time, I started looking for information about VR. I found a Udacity course on VR and decided to take it on. Back when I started it, the course covered many topics, although they made the decision of separating the courses in different specialties, which makes much more sense. If you are interested in seeing some of the projects I made during this course, check my Github account.

After that, I got interested in open source projects on AR and wanted to start doing testing there… However, life got in the way again when I moved to China. It’s still on my to-do list.

I was lucky enough to start working for Netease Games in China right after, so I had then enough flexibility and hardware access to do some more research in VR including some automated testing with Google Cardboard, which it should be now integrated in Airtest project (I know, not many people are using Google Cardboard anymore but, hey, you need to start somewhere.. the other research is still ongoing)

I also was lucky to have the opportunity to attend to the second Sonar in Hong Kong, which is a music and technology festival, and it showcased some cool new technologies and devices in VR (including aroma experiences and snow-surfing)

Besides that, I started to think of plans and ways of testing VR applications too (as Netease was working in some projects like nostos, which I had the opportunity to try myself and really enjoyed it).

Around that time, I gave a talk in Selenium conference in India gathering all this gained knowledge (which I talked about on this post). In order to prepare for this talk I played around and created my own ‘conference simulator’ just to get prepared for it.

Another thing I do frequently to gain knowledge in VR is to watch playthroughs and online reviews, as you can learn a lot from watching others play and it could be very good to understand your potential users if you are working on a game. I also have read some books on the matter (shout out to packtpub which gives away free IT books everyday!)

Have you found a pattern?

I know you have, if you are a reader of this blog you surely are a clever Lynx by now, but just in case, I have highlighted it in bold letters: Attending to (and after a while starting) hackathons, meetups, talks and festivals, watching or reading online related content and books, and playing around in courses, open source projects, at work and on your own projects will get you into anything you are interested to get into.

It sure sounds like a lot of things to do, but the tools are already around you, and I’m talking about years worth of experience here. Just take one thing at a time and you will too become an expert of that thing you are into. The biggest difficulty is to pick which one to take at any given time and to be honest about yourself and how much time you can spend on that. (I regret a ton not having put more efforts on the AR open source project when I had the chance)

Of course, if you are not really into it, then it would sound like a lot of work, in which case it’s probably better to save yourself time and pick something else. I like to think of it as a test of passion, or on the words of ‘Randy Pausch’ from his talk ‘Achieving Your Childhood Dreams’: “brick walls”. (By the way, this is one of the best motivational talks I’ve ever watch, and I actually re-watch it at least once a year to keep me motivated. Also, it mentions VR too ūüôā )

As you would imagine, this is not the only subject I spent time with or I gave my attention to, another big one for me is artificial intelligence, but that’s…well…another story.



Hacking social media

I know, I still owe you some stories, but I am now inspired to talk about something else. Besides, today’s issue is easier to put it into words. I don’t need to sit down and think carefully on a way of explaining some technical concept such as artificial intelligence while not sounding boring. But, just so you know, I am still working on the other stories.

I would like to show you how dangerous social media could become and on one hand highlight the need of asking the right questions when a new technology comes along in order to set proper tests and barriers (lynx are curious animals, aren’t we?).

On the other hand, highlight the importance to take breaks from it and think about yourself and things that would make YOU happy instead of thinking of things that ‘would make other people think that you are happy’ and therefore approve of you. I hope you enjoy reading this article.

Let’s think about it: the most viewed YouTube channels from independent creators are from people below their 30’s (or just on them), many started them 5-9 years ago. They have been getting a lot of pressure from fans and companies that would like a piece of their influence. Celebrities with less direct exposure to their fans have done crazy things in the past because of social pressure. Yet these influencers are not invited (that I know of) to Davos or famous lists of most influential people, besides demonstrating incredible marketing strategies, knowledge of new technologies, having charisma and being very intelligent (more than they let to be seen in some cases)

Social media affects society, not only these influencers, but many people are actually feeling depress or harm themselves because of social media. It is also a potential source for propaganda of all types and an a source for advertisement of all sorts by using the platforms ‘algorithms‘ in their favor.

There is a very important point to consider, which is its the potential for hacking (if you are interested on this, there is more information here and here). So, imagine that someone could actually go there and decide what you are going to see… how could this affect you?

Techniques and prevention:

Let’s imagine a platform that accepts comments and likes (let’s forget about dislikes). How could someone socially hack it?

1) Removing the likes: We would need to intercede the information that this platform is showing to the user and eliminate the likes the user will see. Maybe we should only eliminate them partially, so the user is not suspicious of not seeing any likes at all. For this, we could have a pondered random variable that would eliminate or not eliminate each like. How would you feel if all of the sudden nothing you write gets any likes? Prevention: From test side, make sure the like system works properly and cannot be done by anonymous sources. Make sure accounts are real. Make sure the user sees only real data. From user perspective, when you see something that you like, mention it in person, start a conversation about it instead of just clicking a button.

2) Liking specific posts: This is a bit fancier. Based on above, we could have some sort of AI algorithm that could classify the posts. Then we can decide which comments are going to show as liked for the user. How would you feel if only some type of your posts would get liked? Would that change your way of writing? Prevention: From test side, make sure all information is shown to the user. From user side, find your audience and focus on them. Also, consider talking with this people directly too (or in conferences). Try to keep honest to your goals for writing and who you want to reach.

3) Filtering comments: This would require some form of classification as with the previous point. Instead of targeting the likes, we would target the comments but the idea would be the same. Eliminate from view those that are not ‘interesting’. What would you think if you only receive a certain types of comments? Prevention: From test side, make sure all information is shown to the user. Maybe have a conversation about the feature itself and allow users to hide all comments. From user side, as above.

4) Creating comments: We could create new comments with AI. You might think the user would realize about this, but if done carefully they might not even notice this or confront the person making that comment. Besides, the social media platform might allow for not-logged in comments. This adds to the feeling of the previous one. Prevention: Have a conversation about blocking anonymous comments or disabling them. From user side, if you see a strange comment from someone that does not add up, clarify this with the person. It can also help with misunderstandings. Option 2, disable or stop reading comments.

5) Changing the advertisement around the website to a convenient one for propaganda or for harm (only for some types of social media). Prevention: Most of sites have a way of deciding what advertisements you are more interested of. Also, try cleaning cookies regularly or use private browsers and VPN services.

6) Extracting automatically interesting information for malicious purposes. Prevention: Be careful with what you post, don’t use information that is available for anybody as your passwords or security questions or pictures with personal data (such passports or train tickets). If you really want to share pictures of a trip, try uploading them after the trip and enjoy while in there!

7) Connecting certain types of people. I am not 100% sure how this could be used for malicious purposes but surely someone would find a way. Making sure you can block people is also very important.

8) Taking things out of context. Prevention: It’s very hard to delete something from the internet once it is out there, but some platforms allow it. Have you ever read your old posts? It is a good idea to do some clean-up every so often. Also, if this happens to you, keep a track of the entire context. Maybe have a system in which you can remove what you have written before it goes online, take some hours before posting to make sure you want to post that.

Why am I talking about social hacking social media in a test blog? Well, because, if you happen to be working on developing a social media project, you should make sure that these attacks are not possible and think about how the user could feel for other features to come.

(Please take some time to go throughout the articles I liked above to know more)

Thoughts about being addicted to being connected:

When was the last time you did something good for someone but did not tell anybody about it? When you do something good for someone and post about it, how do you know you are doing it because of the other person and not to have a better image of yourself in front of others?

When was the last time that you went for a trip and didn’t share the pictures with anybody? What about not taking any pictures at all? There is something truly special about having a memory that is just yours to have.

Conclusion

Social media has evolved quickly in very short time, and we need to consider a lot of new things, more so if we are in the development team of one of these platforms. We should really stop and agree about what is ethical and not with the particular platforms and maybe even list the set of contraindications as if we do with addictive substances. For example, would it be ethical for some platforms (maybe for young people?) to change what the users see in order to protect them from the bad critic? Consider this could, in theory, save some lives, but, in contrast, it would take away some potential good feedback disguised as bad comments. Maybe this is a feature you want people to turn on and off? If not, maybe you should list in your contraindications that it could be an issue. And I don’t mean terms and conditions, it’s not about saving your behind if anything happens, it’s about actually alerting the user about what could be experienced. Terms and conditions are…well.. another story.

Sources:

Some sources if you are interested in this area that helped me control my internet usage:

Book: “How to break up with your phone” by Catherine Price.

Watch: Crash courses on navigating digital information

Talking in selenium conference

Recently I’ve had the amazing opportunity to present in none other that Selenium conference in India. Even though I have done some other talks in the past, this was a big challenge.

I would like to share my experience, for those lynx curious about talking in or attending to conferences.

1) Submitting proposals:

The power of a deadline can sometimes be very impressive…at least for me.

I was keeping a couple of proposals for things I wanted to share for quite a while, always postponing to actually sit down and properly write them.

However, when I received a message stating that the last call for proposals for the selenium conference was in a couple of days, I told to myself: “hey, nothing to lose, just write them down and send them on before is too late. Let’s just get out of our comfort zone!”

2) Getting accepted and preparing for the conference:

After submitting the ideas, I’ve gotten contacted for one of my proposal asking for more details. There was some exchange of communications and after that…silence… nothing. I’ve gotten no idea if they were happy with my replies at all.

A few days later, I received an email. Bad news: my proposal was not accepted. I was shocked, after so many messages, I thought they would have been interested on it. Luckily they provided a rejection reason: “We’ve already accepted another proposals from the same speaker”.

Really?? I suddenly remembered: I did send two proposals. And right after that, I realised I was accepted to speak in THE selenium conference… and I started to freak out. What now?

seconf

I took a deep breath.

I reviewed the published list of speakers and then I realised that I had the pleasure of having met one of them before: Maaret Pyhajarvi. I brace myself, took all my courage and decided that if I was going to speak in the conference, I should be able to reach out for help.

I was very lucky that she was very nice and helpful. We arranged some online meetings (which was not easy because of the time difference) and she gave me plenty of valuable advice. I’m so happy that I asked Maaret for help and so grateful for her advice. I highly recommend that you watch her presentation and lightning talks, they were absolutely brilliant and inspiring.

OK, what else can I do to prepare for the day? (Besides planning and creating the slides and content, which is easier said than done)

As I was talking about virtual reality, I thought it was only fair to create an app for practising. So I asked the organisers for some pictures of the room I would present on and I created it on VR, even adding some original murmuring sounds. If you are interested, it’s uploaded here.

3) Travelling to India. Getting around. Impressions:

One of the reasons for me to apply for the selenium conference in India was that I was, at the time, located in China, so it was very close. I didn’t need many days off, the trip is shorter and it also was fully covered by the conference. Also, I’ve never been in India, great excuse to visit it!

I was told to first book the trip and then request reimbursement. They actually helped me to pick a better flight that was also a bit cheaper, and they dealt with the currency exchange. I am thankful for the organisers to help so much with this.

They also booked me in the hotel where the conference was taking place and arrange a car to collect me from and to the airport. I didn’t know how useful this would be until I arrived there. I would have had no idea about how to get to the hotel once there, so once again: so grateful for this.

I didn’t have time to travel a lot, just meet with the other speakers for food. That said, it seemed that Bangalore didn’t have much to see within the city but if you go a bit outside you can visit places. Unfortunately I didn’t have time for this , so my opinion about India can’t really count much from this trip. I’m going to have to go back for a better impression ūüėä

4) Impressions on the selenium conference:

I feel it was well organised, food was delicious (although I did get sick coming back home, but apparently this could happen during your first days in India for what I’ve heard).

I didn’t have much time to see all the talks, but I’ve watched some of the videos afterwards. I feel there was a good balance of speakers and that people should take into account the levels explained on the descriptions.

It was a pity that I was not able to attend to all the talks, but it is OK because of the recordings. I also felt pressured because I was presenting, so I’m happy I presented early enough and I could relax afterwards.

I really enjoyed it overall, it was a great experience and amazing opportunity for networking. All the speakers were very approachable, which I think it’s one of the biggest values of attending a conference.

5) The presentation. Self-feedback. Lighting talks.

What you are about to read now is a series of self-improvement tips, because, as a good Lynx, I am always learning. I am writing them more for myself than for you, but maybe this could help you too, if you ever consider presenting anything.

Looking back, I wish I reviewed much more and I trusted my own experience of needing more content than what it seems enough when practising. I felt it ended up being a bit short. Learning: prepare extra content and practice a lot.

For some reason (probably the fact that it was THE SELENIUM conference) I was as nervous as if it was my first conference ever. Learning: breathe slow, try relaxation techniques before the talk.

I wish I would have kept it more natural, after the matter I came up with more ideas to break the tension with the audience, but before that I didn’t know how to do it. Learning: get information about audience and think of how to break tension with them and make the content more interesting.

I wasn’t so sure about the content: it was a beginner talk but because it was very generic I could not go deep in any of the testing types… I wish now I would have spent longer in some of them or done some demo. However, I was relying in the questions to go a bit deeper…shockingly, there were not many questions. I think it could be a cultural thing because other presenters told me that the same happened to them as well. Learning: keep only 5 minutes for questions and show more technical bits. Talk about what you think it is important instead of waiting for them to ask.

I did blank out and forgot many many things I was meant to say. I took my safety notes with me but yet I lost my way throughout. This was what I was most scared of. Learning: slow down, move the notes as I go along. Have notes on the laptop, not only paper, it looks more natural to look in there.

Performance: muy accent didn’t come up clean. I should pursue to speak slower. I also looked and sounded tired, I was. I should try to get better sleep before the presentation day. Learning: just before the presentation, get as much sleep as possible. Travel earlier to avoid jet-lag. Record yourself to check if the pronunciation¬†is correct or you need to change some words.

The lights were not the best – neither my shirt nor my slides matched with the background and projector. Learning: bring spare clothes and presentation with two background colours and try them before the meeting.

Video: it started late, my introduction was not recorded. I didn’t know I wasn’t supposed to move and I disappear every so often. Learning: Ask video person to record a short video and let you see it. Ask if you can move around or better stay in one place.

Luckily I could make up a bit during the lighting talks, in which I seem more like myself. But, unfortunately, not many people attended and it’s not part of the video of the talk.

6) Conclusion

Even though I feel I could have done much better, I enjoyed the entire experience a lot. I learned plenty from it and I will be much more prepared for the next one.

If you attended or have watched the video, you might be interested on knowing more about a project that I mentioned I am working on…but that’s, well, another story.

Hands on: automating scratch with AirtestIDE

On one of my firsts posts in this blog, I went through a hands on about how to automate scratch. The goal was to be able to teach kids how to test, hand by hand with how to develop programs.

Back at the time, the only program that I could find that would allow me to do this automation was not open source. The experience was very nice but since the goal was to use it on coderdojo , the conclusion was that it would not work out to use a paid solution.

Currently, I’m part of the development team on an open source project called ‘Airtest‘. Please note the intention on this post is not to advertise this product, but to provide a different solution that could actually be used in coderdojo. However, I want to disclaim it clearly: I work on this project, also, I am very fond of it.

Part 1: scratch development

Scratch is a simple tool for getting started with developing just it by dragging and dropping easy sentences. I’ve already explained how to create an app with scratch in my previous post, so I am going to assume you know how to do this.

Let’s just use the project that I created as example: http://scratch.mit.edu/projects/48422496/

This program moves a cat (called sprite) 10 pixels every time the user clicks on it. If it reaches a corner it supposes to bounce. The goal is to verify that the cat bounces when reached the corner.

Part 2: Using Airtest project to automate scratch

    1. Go to the official website:¬†http://airtest.netease.com/ and download Airtest IDE. (Note: it is possible that you see some Chinese around as we are based in China, but don’t get too scared, everything should also be written in English)
    2. Click on “File” -> “New” -> “.air Airtest Project”,¬† to create a new airtest test file1
    3. Make sure you have “Selenium window” options active by clicking “Window”->”Selenium Window” (you can close Airtest assistance, Poco Assistance and devices).¬† ¬† ¬† ¬† ¬† ¬† ¬† ¬† 2
    4. You also want to make sure you have your Chrome path configured in “Options”->”Settings”. If you have Chrome installed and are using Windows, this is generally under C:/Program Files (x86)/Google/Chrome/Application/chrome.exe 3
    5. Make sure your cursor is at the bottom of the script editor.¬† Then, click on the globe icon at the top left of the “Selenium Window” to open a browser for test.4
    6. Click “Yes” on the yellow message that appears on the top. 5Now you could continue with the recording method (right hand side top button on the Selenium Window). However, since scratch is done in flash, we won’t be able to retrieve a selenium element. we would proceed with Visual testing.
    7. Make sure your cursor is at the bottom of the script editor. Click the “start_web” button and input the url in the quoted yellow text. 6 It would look like this:
      driver.get("https://scratch.mit.edu/projects/48422496/")
    8. Make sure your cursor is at the bottom of the script editor. Click the “airtest_touch” button and select the flag so it is within the highlighted rectangle. Double click inside it to confirm the selection¬† 7
    9. Insert a this piece of code (this is a loop to repeat the next sentence 30 times, which would be the cat click):
       for x in range(0, 30): 
    10. Make sure your cursor is at the bottom of the script editor. Click tab once. Click the “airtest_touch” button and select the cat. Double click to confirm the selection. Your code should look like this (I have added a sleep command to wait for the elements but it should not be needed): 8
    11. Click play to see how the program works (make sure to activate flash on the browser for the program to load the first time). Feel free of getting some snack while your computer does your job for you ūüėČ
    12. Bonus step: verify that the cats bounces. You can do this by inserting an “assert_template” and selecting the cat (same as with airtest_touch). However, the program would pass if it finds a cat, in any position, so even if it is upside down. Alternatively, you can create a snapshot to check manually this part.

Conclusion

Since Airtest project’s back-end (poco) is based on Python (a programming language) you can insert loops and other instructions easily.

Because of its powerful IDE which allows for record/playback or the use of simple buttons, it is easy for beginners.

Since visual testing is possible, even within a Selenium test, it works with programs or websites in which object hierarchy (DOM) is not easy/possible to retrieve (such as flash applications).

Because it is open source and the IDE is free, it can be used in teaching.

There is much, much more that this one tool can do that what is shown here.¬†However, that is… well, another story.

Examples of AI applications and how possibly test them

Recently I attended an online crowdchat hosted by ministry of testing about testing AI applications.

The questions were very interesting, but it was hard to think of a right answer for all AI applications, as this is a very broad field. Explaining it over twitter would be confusing, so I thought I may as well create a post giving some examples.

Kudos to someone on Twitter that mentioned supervised and unsupervised learning at the end of the chat. I was very sleepy at the time (the chat started at 4am my time) so I was not able to find his tweet in the morning to vote for it. I think that we could understand better the types of AI applications that we could have if we divide them in supervised vs unsupervised. More information here.

Supervised learning examples

The idea behind it is easy to understand: These applications have a phase of learning in which we keep feeding them with data and rewarding them if they produce a correct result, while punishing them when they don’t until the produced results match the expected results within a threshold. (In other words, until we are happy with the results).

Let’s ignore for now the exact ways in which we can punish or reward a machine and just focus on the general idea.

After this learning phase, we generally just use the application and no more learning takes place. We “turn off” the learning. This is called inference phase. Not all the applications have inference phase; sometimes we want them to keep learning from the users, but this can turn out to be problematic, as we will see further on.

I think these are the easiest AI applications to test functionally speaking, as we just need to pass new data and check the results obtained against the expected. Apart from this, they behave just like any other application and we can also go through the other types of testing without many changes (performance, security, system..)

NPR / OCR:

Imagine for example a¬†number plate recognition system – once the system learns how to recognize the numbers in the license plate, you don’t have to keep training it. The application can use the learned patterns to verify the new number plates.

There are many tests we could think of here, without caring for how the application gets the results: try with characters that have strange typography (if allowed in the country), tilt the number plate, check the boundary in distance from the vehicle…

An OCR (optical character recognition) application could also be done with this technique. In fact, the number plate recognition system could be considered as a specific type of OCR.

Digital personal assistance (Cortana, Siri, Alexa…):

Quite common nowadays, they help you find out information using voice commands. They could also use supervised learning (although, I believe the right classification for them would be “semi-supervised learning”, but let’s think of them as just supervised for the sake of the example). However, in this case the application keeps learning from the users. It stays in the learning phase.

The reason they can ‘safely’ do this it is because they collect data from the users but not their direct input in whether the result was to be penalized or rewarded. An example of application getting direct input from the user to keep learning would be a chatbot that guesses something and asks if that guess was correct. This could be easily tricked by dishonest users.

Applications that keep learning are much trickier to test, even functionally, as if we pass wrong inputs to test, they will learn wrong. If I had to test one of these, I would use a copy of the state of each iteration we would like to test in an isolated environment, so we don’t break the acquired good learning. For performance testing it would be best to use valid data, to ensure the learning process continues well.

If anybody is concerned about AI gaining consciousness, this type of applications would be the problematic one, as they could be learning things we are not aware of depending on the power that the programmer and the user gave them and the data they are able to collect. This brings up the question: Should testers be responsible to test consciousness?

Unsupervised learning examples

The key of these applications is to discover relationships on the data without direct penalization or reward. They are very useful when we are not sure of what the output should be, and to discover things that we would not naturally think of being related.

There are two types: Clustering (when the system discovers groupings in data) and association (for discovering rules describing data). I won’t go deep on them in this post, as it is a lot of information as it is.

Tailored content-advertising (Amazon, Netflix, Google…)

These apps want to be able to predict what the customers that bought something would be interested on next. In fact, digital personal assistance tools could also use this data to help you find what you want (that’s why I mentioned before they should be classified as ‘semi-supervised’ learning). I cannot think of any ways of testing this except checking on the impact on the sales after the application is in place, but this could potentially be subjective to chance or other factors not related with the application itself.

Apart from that, the test of the application should be the same as we already do with non-AI applications (not just the results, but how the user inputs the data and how the application responds and shows back the data…) Imagine this as a feature of a bigger product, all the other features would also need to be tested as well.

The moral impact of these applications, in my opinion, is that at some point they might be telling you (as a user) what you want, even before you know you wanted it.

What could possibly go wrong?

What should we be careful about in AI that might not need so much attention in other  apps?

Things could go very wrong if we leave apps learning constantly and we leave the users to provide the penalization or rewards. You probably have heard of applications such as image recognition systems and chatbots becoming racists and sexists. Sometimes this is because the test data given to the application is biased, but it could also be because of trolls playing around with the application in unexpected ways and giving rewards when the application is wrong.

Also leaving apps learning on their own is not the best idea, as we do not control what they are actually learning, as mentioned before.

If you are interested, I found an article with some more examples of issues with AI applications here.

What else have you got?

Below is a list of readings that I found very interesting while researching for this post (a couple of the links are about video games and AI):

How “hello neighbor” game’s AI works

AI predicting coding mistakes before developers make them

Examples of AI

Game examples of AI

How would you test these applications?

What do you think about the moral connotations?

If used well, AI could be harmless and powerful. In fact, it could also be a good tool that we could use for automating our testing, but that’s…well…another story.

Interviewing your interviewer (17 tips)

This topic is probably not only for testers, but, since I have been dealing with it recently, I thought some people might find it interesting. Also, I know, I owe you a couple of other posts¬†(I’m working on them, but I am also moving houses and I have limited time to write right now)

I have been told that I ask good questions at the end of the interviews, so I’d like to share my system in case someone else finds it useful. Tip: the important thing is not to ask questions to impress your interviewer, but to use this time frame to find out things about the job and the company.

I usually get very nervous when I do interviews, even though I have interviewed people myself as well. I find, that a good trick for me not to get so nervous, is to think of the process as a two-ways interview, in which you are also interviewing the company to verify if you really want to work in there.

I know this might be hard to take, especially for beginners: “How would I be interviewing a company that I want to work for? I just want the job, if I didn’t like the company I would not have applied for it.” However, it is important to know as much as possible about the job you are about to be doing for a good while. That is why it is crucial that you ask as many questions as you can in order to understand how they work in that particular team and what is expected of you.

Another point, if you can, is to double check the answers with the interviewer’s reaction to the question. I mean, they are supposed to say good things about the company… imagine how bad they would look if the candidate says something like “well, I need to drop my application because the interviewer told me this is an awful place to work for”. But, if you are paying enough attention, you can see some reactions such as long pauses or struggles, that could lead you to believe that they did not feel comfortable with the question or they are trying to sugar coat their answers.

By now, you are probably thinking that this could be a good advise, but you would like to see actual examples of these questions. I owe you a couple of code samples by now, so I won’t let you down on this one. Be careful, some of them might be in the job description, and asking them might show little research and be so annoying for the interviewer as when they ask you to walk them through your CV (which usually means they have not bothered to read it fully). Below some examples of things you might need to know before joining a company (note, when I do an interview these come naturally to me, it depends on the specific job, these are general examples that I can think of right now):

  1. What process are they following? (Agile, waterfall…)
  2. Would you be joining the team of the interviewer or a different one?
  3. How would you relate to the interviewer in the company?
  4. What technology are they using? (The description usually would mention one or two, but you might ask what would be the biggest used, or for a technology that is not in the description, for example, what they use for source control)
  5. Do they do code reviews?
  6. What’s the relationship between the developers and testers? Do they sit together? Share code? Do common code reviews?
  7. How often do the interviewer…? (meet clients, have team meetings, create new features, spend in paperwork such as performance review…)
  8. How long are the springs? (if using agile)
  9. How many times did the interviewer use *insert benefit* this month?
  10. How do they do the performance reviews? How do they measure performance?
  11. What are they expecting of the candidate?
  12. Is there a possibility of *insert benefit*? (getting bonus, stock, gym membership, learning expenses… this depends on what you are looking for in the company)

Some extra tips:

  1. Try to ask the right questions for the right interview: technical interviewers might not know the answer to an hr questions (for example benefits) and hr people might not have an answer for technical questions (for example technologies they use). You might be wasting their time and not getting your answer anyways, so it is better to save the questions for the right round. (Be sure to learn about the rounds to know when to ask what)
  2. If you can, try to say them in a way that sounds a bit more personal for the interviewer, they are more likely to give you honest answers if you are asking for their opinions than for the company’s protocols. For example, the style of number 9 is more personal than number 12 for knowing about benefits. While number 9 gives you information about the actual behaviour of your co-workers and the non-spoken politics in the company, number 12 can give you room to negotiate a particular benefit that is not usually given (you can use this style with hr).
  3. I have said this already, but: don’t ask them for the sake of asking questions. Think about what it is not clear to you ahead of time and take a pen and paper next to you when you ask them to write down the answers. This can help you not to repeat yourself and to remember everything at the end of the process. It might be something you can negotiate or it might be something for you to discard the company (or the specific team). This does not only give you valuable information but puts you in a more powerful and confident position when doing an interview rather than feeling under test.
  4. Don’t get angry or depressed if you don’t get the job: sometimes it’s just a matter of being lucky and getting an interviewer that connects better with you. Sometimes a company might offer different positions or to work for teams that use different technologies that might be more aligned with your experience. And above it all: if your interviewer is pushing you back for not having the exact same expertise as him/her, you are probably better off not working with that person anyways. I think the trick for a company to work well is to have people with different set of expertise: they might not know something with the same depth as you do, but they might know a lot about something else and you both can learn from each other.
  5. Keep trying and practising: doing interviews might be extenuating but, with enough practice and asking the right questions, sooner or later, you’ll get just the job you want.

Let me know in the comments if you can think of other good questions to ask and let me know if you like these sort of posts about the interview process. I could tell more about it, but that’s… well… another story.

 

 

My views in the future of testing

As you might know by now from my previous posts, ¬†I’ve always been between testing and developing.

Every company is different and every person in the team is different. Some developers highly respect testing, write tests or at least like to learn to think like testers. There are developers that use testers as a shield against management’s disappointment when something goes wrong in production.

Some testers know how to code and are interested in new languages, technologies, systems… others might know some programming but just enough¬†so they can stay in a comfortable job (this could apply to the devs too). Others are great testers¬†although they do not know how to program (and that should not be an issue or be taken lightly by the rest of the team).

I have noticed a lot of confusion and open interpretations on the testing side of development and I have seen many movements on that side: I have seen some companies moving from QA to SDETs, and others moving from SDETs to Software engineers. It seems like this is the new fashion, the process to follow, developers to do both dev and test tasks (and operations as well, if possible…).

I have been concerned, to be honest, that I might end up in pure development and I would be fully extincted.. I felt as endangered as my lynx specie is.

The introduction of TDD and then of BDD, makes companies ignore other types of¬†testing (maybe with¬†exception of security) in favour of deploying earlier and faster, thus leaving the end to end ‘testing’ directly to the users.

However, I think this is not the right approach. Don’t get me wrong, I was already developing and testing, I would have ‘no issues’ moving with this current. But at the end of the day, if nobody is in charge of the quality, why would anybody care about it?

If previous testers are either gone or converted to developing first and thinking about quality second.. at some point, after hiring more and move developers and promoting thinking about speed of development first, who is going to think about quality? Who will have the knowledge to make a point or call out missing scenarios?

This might as well be a cyclic fashion, and at some point companies would be needing a test expert. Someone that cares about quality and creates integration and system testing, checks that the unit tests are right, verifies which tests are going to be run in the different builds, creates other types of testing, researches about testing, architect solutions for automation… Or at minimum someone that can train the developers to check for the right things and ensures the right testing happens.

How can you get test experts if there are no testers anymore? At the moment, the answer is: from other companies that are still using testers… but what if there are no such companies anymore?

I’ve heard complaints about how hard is to find testers that also program or have development qualities. Or developers that are OK being called testers and undertaking tasks in the test system. Maybe this¬†post¬†helps someone¬†understand¬†the paradox.

So, if we don’t find testers.. is it ethic/safe to deploy¬†without comprehensive end to end and allow the¬†users¬†to report the issues in order to speed up development? To me, it is not, although I can understand where this comes from and that¬†in¬†some environments it’s very hard and costly to find alternatives.

Will this be the new quality assurance culture? I don’t think so, but that doesn’t mean that the things will go back to the way they were.

Should I be scared of losing my job? Well, that highly depends on your company, so I cannot answer that in an easy way.

What can I do to make sure I will stand out? I’d say, try to make yourself valuable, just as with any job. And I do not mean writing spaghetti code, (although high respect to the pastafarians, arr!) I mean knowledge wise.

I think there is much more to come. Research to be done. New ways of automating, new processes, development,¬†test expertise …

And that’s why I think the future of testing is uncertain, but it will be different to what we have known until now. It has started to change and I think we better take on that boat and participate in the change that let us be driven by it.

Where to start? We should start by automating everything, including the automation. But that would be… well… a different story!

Between two rivers

As a developer, I often find myself stuck in a problem and I rarely¬†get the freedom to switch my mind to a different¬†task¬†until I¬†solved the first issue. But I need to be able to switch tasks, that’s how my mind works.

I read somewhere that this is the trick to creativity: just do something else that is more automatic and then go back to the hard problem. While only developing, I find it¬†hard to work on this ‘all or nothing’ way, it does not feel like I am creating, it feels like I’m forced into making the features.

Testing, for most companies I worked for, was at the exact same technical + programming level as the developer role. By this I mean: you needed to be a developer if you wanted to get the position and you get the same interviews.

At first, as tester, I felt unchallenged, as if I was just repeating tasks…and I do not mean manual testing, writing some tests automation could be repetitive as well. But that was until I learned that I could not only automate the tests, but also the processes. So I get more job done faster! I could do things in parallel, running tests that will show my results while writing code that will clear up my to do list…¬†That’s one of the reasons that I love it, besides the fact that you get to think out of the box and create new ways of solving and proving issues.

If you are curious about different roles across the two disciplines, here are the names of the roles I’ve done and what type of work they involved:

  1. Software developer: This is used as a generic term for someone that can write code (programs) and can also design technically the new features (sometimes with the help/agreement of other team members). There are many different types of developers and the number of positions are increasing and evolving with the years: front end, back end, full stack, devops, mobile, VR… And now you can find people really specialized in a particular language, so if you are starting on a position, make sure you like that language.
  2. System verification tester: In my case, it was similar to software developer, only that instead of working with databases, back end, front end or applications, I was working on automation (and so with a bit of it all as well). You also need to be able to think out of the box and come up with tests that would help demonstrating quality. In this position, I, in particular, did not get to talk a lot with the developers of the system we were verifying, but it was common to check with the customers to make sure their quality criteria was met. There was also a degree of performance testing there. Basically, we were making sure the system was working across different platforms and specifications by writing code to check it.
  3. SDET (software developer engineer in test). I really like this role, even thought it sounds as if¬†I you are a ‘provisional’ developer, rather than a developer that works in the test side. I like that it has the word ‘engineer’ on it, which I deserve (I have two engineering degrees). In here, SDETs work in the same sort of things as the SDEs. ¬†The only difference would be that while the SDEs work in making the code for the actual features, SDETs work in making the tests for those features and verifying they worked. Code reviews were done across disciplines making everybody familiar with all the code.
  4. Software engineer: This is a recent enough title. If you see this, it means that in the company the developers also do the testing, usually switching between the two roles, or as part of BDD or TDD and with faster releases. I like this role as well, although quality tend to get lower priority in favour of faster releases and I particularly believe it is better to have someone in charge of quality so it does not get forgotten.
  5. Software QA lead: ‘Developer’ and ‘engineer’ words are¬†out of this name, but, I still got to do some degree of developing when I had this title. However, not as much as I wanted to get in this particular position, but I suppose that depends on your career goals and on the opportunities on the particular company to meet them. I did enjoy the leading part though, more below.
  6. Software engineer lead: Same as number 4 but also leading. I am mentioning it because I had the wrong assumption, when I was a younger lynx, that the leaders did not get to do coding anymore. I am expected to do as much as the others, but I also have other leading tasks and have to keep an eye on the team. It is hard, but also quite rewarding because you get to help others and inspire them.

I want to get the ‘developer’ label as little as¬†the ‘tester’ one. I’m neither. I’m both.

Especially when you start labeling developers by their expertise. Could I be a front end developer? I could, but I’m not really. A back end? Even less… I’m sort of a ‘all in one’ and ‘none of the above’ at the same time.

I find it interesting that the developer types are becoming new positions, while some companies are moving away from the SDET title (which¬†it’s a developer type).

Dev and test positions keep evolving and changing across companies. How would testing be done in the future?¬†That would be… well…a different story.

The beginning of the test lynx experience

I maintain my belief that anybody can become a tester (maybe all they need to do is to change their hats). I also think some people are born testers. This was my case, only I did not know it when I started my career.

I think you could easily identify future testers when they are kids if you observe their behaviours. For example, at the beach, is the kid building a castle? Possibly..dev. Is the kid destroying someone else’s castle (hopefully when that kid is away) or pointing the defects of the other kid’s construction? Possibly.. test. What if the kid first build the castle and then proceeds to destroy it? Well, that would be my case, you would understand it better later on.

When I was about 9 years old, I had an old computer, one of those that you plug to the TV and run on basic. It happened to come with a book for basic (or my parents bought it?). The thing is that I wanted to go further than the games I was playing with and I started to learn how to write some code.

Being able to write games myself opened a new door to wonderland for me. I was still¬†curious to see how the games I was playing with were written, so I learned a very useful command: ‘LIST’. That was¬†when the magic really started: lines and lines of beautiful code that I would have not been able to put together so far, but, yet, ¬†I could understand.

Some of those games took¬†very long in¬†loading and sometimes they would fail (‘Syntax error’ were the worst two words I could read back at those days.)

It turned out that the games were loaded by sound, and sometimes the cassette could jump and eliminate bits of code. Sometimes I was able to fix it, which would mean not to wait again for 2 hours (hurray!).

Years later, I did learn some concepts about testing during my computer science degrees, but it is usually something only described briefly as part of other courses (maybe the testing community could do something about this?).

When I started working, I first joined as a developer (I did not even know there were many other¬†choices). It was fun to build those sand castles, but something was wrong… I was not satisfied, I needed something else. But that¬†is…well… another story.