A Conversation with Noah Sussman -- QA Manager, etsy.com

I have been having these questions about when testing happens in a continuous integration context and there have been a couple of suggestions that I talk to etsy.com's QA Manager, Noah Sussman and pick his brain about it.

I met Noah once at a selenium tech talk featuring Jason Huggins (more name dropping) a couple months back. One of his team members, Michelle D'Netto, was presenting how Etsy builds their regression tests. I remember Noah as someone who was very stoic during that tech talk, always taking down notes. When I met with him last week and I guess 2nd impressions are better since he was very personable, very easy to talk to and was very open to sharing his thoughts about testing in general.

Here are some key takeaways from that conversation.

when your development culture shifts, adapt

Etsy has a very interesting development culture because they don't have a merging strategy and they have a very exhaustive use of continuous integration. For a company that has 240+ employees and of which, only four have official "tester" designation, it really does beg the question, when and where does testing happen? The short answer is that "It happens". Because everyone commits to trunk, everyone "tests" and not just the testers. They also test in production, which is made possible through switches via conditionals in the code. The developers spend up to 20% of their time testing their work and the 4 "testers" are focused on finding out the unknown unknowns. 

resourcefulness

His context also brings about using a set of tools in more ways than most people use them. From the lowly curl command, splunk, selenium, a gazillion unit tests, and nagios to name a few. These tools give them valuable information. Graphs of real-time metrics that would give everyone in the team sense of what is going on with the site so they can act accordingly and timely when an undesirable event happens, or is about to happen.

testing advocacy

Most test managers in Noah's shoes would probably lobby for more testers or feel completely dejected at the sheer weight of the responsibility of testing for a company who has 60 developers per tester (don't  hate, I know my math is not quite exact). Instead, he lobbied for more testing. His testing advocacy resulted to hiring more testing conscious developers and gave the development team a much needed testing feedback which in turn gave his team a much envied status in the testing community, exploratory testers. 

Thank you Noah for taking the time out to meet with me and I look forward to that Etsy show and tell.