In a previous post, we shared our experience that while most software testers are not very good at finding bugs, about 1 out of 8 of them are superstars who are 3 to 5 times as productive as their average peers. We also shared in that article how you can collect the right metrics to identify these superstars once they’re on your staff.
But how do you find these superstars in the first place? In other words, is there any way you can know ahead of time what to look for on a resume and what questions to ask in an interview that will help you identify and hire these superstars?
The answer is a qualified yes. While no screening or interviewing technique is foolproof, you can certainly increase your odds of finding and hiring superstars by keeping the notable differences between them and their not-so-impressive peers at the top of your mind.
Rather than try to anticipate all the kinds of things a QA applicant’s resume might say or to create a list of questions for you to ask in an interview, let’s simply make sure we’re clear on the differences in the way they think and the way they approach their work.
Below, you will find the attributes of good—and bad—testers. Keep these points in mind the next time you need to hire an internal tester or a contractor for a special project.
Superstar testers…
- Focus on finding bugs, not on documentation (or lack thereof). They will gladly write the necessary documentation (i.e., documentation that will be needed later for a specific, important reason) but they never waste time writing documentation for documentation’s sake and don’t expect anyone else to do so.
- Always find bugs in any new software and get very nervous when they don’t.
- Have high attention to detail, noticing bugs and typos that others miss.
- Can stay focused and attentive under conditions of extreme tedium.
- Do enough research to figure out how to reliably reproduce the bugs that they find and write bug reports so that developers can easily reproduce them. They may bring a non-reproducible bug to a developer’s attention to brainstorm on its cause or on how to reproduce it, but they will never simply write up a bug report on a non-reproducible bug and “let it be development’s problem.”
- Approach testing with a “try to break it” attitude, emphasizing negative and edge test cases and seeing themselves as the last line of defense for the end user.
- Are creative and sneaky when it comes to finding bugs, anticipating both what novice users might do as well as the mistakes that tired, rushed developers might make.
- Automate when and only when it actually saves time and increases their productivity, never just to learn a new skill or boost their marketability and never because they’re bored with manual testing. A QA professional does what is best for the product, whether or not it is “fun.”
- Are creative with ideas about how to test functionality that others are willing to give up on as “untestable.”
- Report all of the bugs that they find, allowing the team to decide which ones should be fixed.
Testers to avoid…
- Complain about lack of documentation from marketing or development and cannot be effective without it (or claim that they can’t).
- Write a lot of documentation (plans and/or scripts) that has no specific downstream consumer and that cannot be justified with even a qualitative explanation of the ROI.
- Submit reports on bugs they can’t reliably reproduce.
- Write cryptic or confusing or incomplete bug reports that require developers to come ask for help reproducing the bugs. [We should note here that many software QA people are foreign nationals who do not speak English as a first language. There is nothing at all wrong with this as long as they can communicate effectively in written English. But too many organizations hire foreign national QA people as a “cost saving” measure, not realizing the hidden cost of wasting hours of developer time on in-person conversations about unreproducible bugs. Make sure that you don’t make this mistake.]
- Argue for automation for reasons other than the good of the product or project or with worn-out and disproven promises of cost savings. Automation can shorten release cycle times, but it almost never saves money. If a tester can’t clearly delineate when automation makes sense versus when it doesn’t as well as demonstrate clearly how it is justified on your project, you can bet that you are talking either to an amateur or a charlatan.
- Neglect or underemphasize negative and edge case tests, focusing instead on “use cases.”
- Get bored easily and miss subtle but simple bugs like misspellings, misalignments, differences between platforms or browsers, etc.
- Simply miss a lot of bugs, no matter how much they “test,” defending their sloppy work by focusing on the number of test cases they ran and/or the documentation they produced rather than on the number of bugs they found.
- Have a “prove that it works” attitude that’s a little too cozy with the developers rather than seeing themselves as an advocate for the end user.
- Conversely, have an adversarial or combative relationship with development, arguing about bugs or feature design after decisions have been made.
- Filter the bugs that they report (or at least claim that they do), deciding unilaterally what is important enough to report and what isn’t.
We’ve all seen the horrendous stats about how many apps get downloaded versus the number that actually get used more than once or twice. The odds are against the success of any app, so leaving bugs in a product gives users an excuse to delete it. Don’t let your product die for reasons you could easily have prevented with a little more attention to your QA team.
Oh, and one more thing… Please don’t think that we’re recommending that you hire only experienced testers. That is not at all the case. We must all start our careers with no experience. Your test team can benefit from fresh, excited young testers. But remember that inexperienced testers with the potential to become superstars will have the attitude, temperament, and creativity to set them apart from their peers. And, of course, teachability and a healthy dose of humility are great foundational traits in any career.