Quality Assurance (QA)

QA is the place in the project that so many people falter.

Entrepreneurs getting to this point often don’t realize what goes into it and thus underestimate the time it takes to do this, and both developers and project managers get to this point around the time they are getting tired on the project MVP or release. So understanding the amount of work and time it takes to get a project ready for release is critical to creating a solid user-ready system.

Digging deep into the project from multiple perspectives is key to a good release, whether it is your MVP or a new feature in a mature system. When doing QA, especially for a new system, it is critical to have multiple people looking at the system from different perspectives. Designers will see issues with the user experience or flow, final system users will see different aspects of use that could be improved upon or sometimes need to be fixed before the system can be used, and of course developers will see development issues.

QA Lessons:

1. Realize that QA is an ongoing process, not a one time project

Entrepreneurs I’ve met so often think their QA will be ‘done’ at some point, despite the fact that they have more development coming. It doesn’t end. QA is a part of development and as long as you’re building complex systems that interact with each other, you’re going to have to do more QA to get a good product. In fact, there is a very clear correlation between the amount of QA time spent on a project and how well the launch goes.

2. Have a separate QA team or developer

Having been a developer, a project manager, a tester, and a SaaS entrepreneur, I can tell you that when you’re building something, you just don’t see the bugs sometimes. You can go through the whole thing and test it yourself and not find any issues only to have one outsider look for five minutes and a see major issues right away. Even the best developers don’t see the issues in front of them when their heads are in the project. So, always, always, always have someone other than the lead dev or dev team doing QA on your project or you are going to have things slip by.

3. QA from different perspectives

On my projects, I like to have a QA engineer, UX designer, developer, and project manager run through a set of tests before the project is ready to launch. It is time consuming and often very frustrating, but it is worth every second. Each user type sees things that the others don’t, and having them review the system can give you feedback that you would otherwise never have had. That being said, have the lead QA person do the review first and fix the first round of bugs before having everyone else go through the system or a lot of time will be wasted.

4. Writing descriptions of bugs technically and thoroughly saves a lot of time

Everyone on the development team, entrepreneurs, designers, and other stakeholders especially should have at least some amount of training on how to technically explain bugs in the system.

PROTIP: A lot of time can be wasted by developers searching for bugs that either don’t exist, aren’t bugs, or are not relevant. Well written bugs can probably save 7% - 20% of time in debugging.

5. Write specs and tests

Specs and tests are a part of Agile Methodology. When writing a specification for work, if you write out how to test that specification it will streamline how QA team members test the system. This can be more challenging on an MVP, but as the system progresses, it becomes critical.

6. Use an industrial strength PM system when doing technical QA

Using a sub-par system to track and manage bugs is can be a point of failure for the development of a system. My personal preference is JIRA with their integration into BitBucket and Confluence, but there are of course other systems out there. If you don’t have an industry standard system in place for this, you and your team will start losing bugs, or prioritizing issues incorrectly. So get the right tools and know how to use them!

7. Spend the time now or triple the time later

Technically speaking, you don’t have to do a round of QA. If you don’t though, you’re going to have a product that doesn’t work and people get upset about. If you do a halfway job of QA prior to launch, you’re going to have half-way or less happy users. This is invariably going to lead to you or your team fixing the issues later and now having to deal with a bunch of upset people and a failing product.

8. Assessing QA time

Quality Assurance in a SaaS business generally takes anywhere between 10% and 25% of the time to build the system. It never, ever takes less than 10%, so make sure to account for that in your timelines!

9. Padding your Expectations

When building SaaS systems, as an entrepreneur it is important to pad your expectations when it comes to the QA process. Really, you should pad your expectations throughout the entire project and you’ll be much happier, but, at the very least, do it here. If the team tells you it is going to take 2 weeks to QA the product, just plan on 4 weeks of work. It may sound outrageous, but that’s almost exactly the way it goes almost every time. If you’re working with an agency that is already taking this padding into account, then I would suggestion you still pad this timeline, just not by as much.