Born to Test – Conference Highlights from StarWest 2015

This week the Software Testing World is gathering at StarWest 2015 in Anaheim, CA (yeah – that’s where Disneyland is) to share the latest best practices when it comes to Software Quality, Testing, Continuous Testing, DevOps, … and all the other cool words floating around.

I am a big fan of testers – that’s actually how I started my career: as a tester testing a testing tool 🙂 -> back then it was SilkPerformer from Segue (later Borland & Microfocus). 15 years later I am here and was invited to share my thoughts about DevOps and why Testers play such a big role in it: Got my session today at noon in case you are here and willing to listen to my thoughts: DevOps: Find Solutions, Not more Defects.

I also try to make it to as many sessions as possible and post my highlights throughout the conference.

Puzzles @ Test Lab

If you happen to be at StarWest check out the Test Lab – especially the puzzles they put up. Here is my highlight of last night:

Whats the next sequence?
Whats the next sequence?

Wed, 8:30: Things that really matter in Testing – Today and Tomorrow

Looking forward to the Keynote from Bj Rollinson – Test Mentor with 25 years experience

Sharing his experiences when working for Microsoft. Some interesting thoughts on following topics – more on this on his blog:

  • Tools: “Use the tools that help you solve your problem” – BUT – “Sometimes you have people having solutions/tools to problems that you don’t have!” (side-kick to tool vendors :-))
  • Certifications: Not a thought in companies such as Microsoft, Google or Amazon.
    A degree doesn’t mean you really learned something if you don’t have the personal drive to really WANT it. Don’t just do it because you have to!
  • Standards: Quote he cited: “Standards are a direct threat to the creative work of testing teams”. His words (in my language): “I don’t think large testing companies religiously adhere to standards!”
  • Lifecycle: Back in his early MSFT days they did weekly test builds, reviews, retrospectives. Key is that whatever you call it – whatever process you follow – you have to continuously improve the process – otherwise you end up running into the same issues over and over again and never get better
  • Exploratory Testing: “I think all testing is exploratory in testing – even my baseline regression test is!”. At MSFT all of their team members (PM, Devs, …) used the latest builds on a daily basis in a real-life setting.
  • Automation: It solves the problem of doing a lot of things in a short time. But it is not creative or exploratory.
    Its great if you deal with a lot of data – testing different permutations – load testing – integration tests. BE CREATIVE!! Don’t just test “bob/password/a” 🙂
    Key is that automated tests also automatically verify the result! Its “Start to Finish”!

Wed, 10:00: Testing the Internet of Everything

Another Keynote to come – Paul Gerrard on a hot topic. IoE Testing. – more information on his blog:

What is the IoE (Internet of Everything)? It allows us to build something “really magical” (Not Disney-Magical) when combining all these cool little devices 🙂

  • It is a 19 Trillion $$ Market. Expecting between 50-700bn devices over the next 5-20 years
  • I suggest you check out his blogs and the videos he showed us. Really enlightening on whats possible, whats upcoming, which companies invest billions of $$ in IoT and IoE.
  • The test strategy for the IoE is the test strategy for everything -> “Understand what it means to be connected!”
  • Digital Transformation: “I don’t know what it is exactly – but it is what is happening right now”
  • Internet of Sheep: Knowing when Daisy is ready for the Bull 🙂
  • Big Data: IoE brings it to life!
  • Prototyping Barcelona: have a real-life test bed for IoT apps in Smart Cities
  • New Model Testing:
  • Shift-Left: Devs responsible for testing, testers embedded in dev teams, testers morphing to business analystics

Wed, 11:30: DevOps – Finding Solutions, not more Defects

This was my session – which I based off several of my blog posts. I think we as testers have to Level-Up our game and help pushing Software Quality further Left. That means we need to not only increase our test coverage through automated tests. I think we need to find the more challenging problems but then help and educate developers to avoid these problems right from the start. Check my blog Functional Test (R)evolution for some of my ideas.

Wed, 15:00: Putting Quality First through Continuous Integration

I listened to Adam Auerbach from Capital One earlier this year at Velocity. He inspired me back then to include some of his stories from Velocity into my presentations – such as “Every Manual Tester Does Automation”.

Adam is the “last test manager” after CapitalOne moved through the Agile/DevOps transformation. Certain jobs got optimized/shifted. Amazing insights – here my other highlights:

  • We started with Agile and ended up with Sprints and Hardening Sprints -> I call this “Agile Fail!
  • Real Time Automation starts with BDD -> Only build the minimum viable product/feature -> no gold-plating needed!
  • Everything tied into the Pipeline -> thats our assembly line -> and everything can stop the line if there is a problem
  • Test Design: Most Tests have to be Unit Tests, followed by Service Layer and UI
  • Service Virtualization: No need to test all the backend services all the time, e.g: his mainframe -> just virtualize it and focus on testing of the service-users
  • Real Time DevOps Dashboard: Open Sourced Project Hygieia -> check out (on GitHub)
  • Internal ATDD Community: offer internal education, mentoring, …
  • Lessons Learned for Adopting Continuous Testing: Reuse proven Frameworks, be prepared for extra costs, evangelize at every level and discipline
  • Testers: Must learn a programming language, understand how to use CI Tools
  • Developers: Accountable for writing “automatable” code, responsible for passing tests (all tests)
  • Everyone: Take accountability for “Done” being working code and passing tests; Work in pairs!

Amazing Quote: “We don’t log bugs -> we fix it!” -> “We talk with each other – what an amazing concept!”

Thu, 8:30 – I don’t want to talk about bugs: Lets Change the Conversation

Janet Gregory kicking it off with the question: “Remember your last release to production – how do you know it was successful?” -> she was looking for “Real User Feedback Loops” as answers from the audience such as “I tested it so I know” – or “No news (complaining users) are good news” – is not what we are looking for!

  • Don’t just measure yourself against numbers such as % Code Coverage, % of Bugs, …
  • If you do measure make sure you provide measures that are meaningful and actionable for the receivers.
    • Example: “Don’t just compile pretty charts and reports that nobody understands and needs”
    • Example: “Sit down with your manager and figure out what they really need – even if it is just GREEN, YELLOW or RED” -> that’s actionable

Summary Quote from Lee: “Stop Counting Bugs – Level-Up and Contribute More to the Business”

Thu, 9:45 – Selenium: Practical Tips and Tricks

Selenium Test Guru Andrew Krug enlightening us about his best practices on using Selenium.

GREAT SESSION – but as Andrew pointed out. No need to start taking notes as he went through a lot of good stuff in a short period of time. Check out his slides posted to the conference page

Andreas Grabner has 20+ years of experience as a software developer, tester and architect and is an advocate for high-performing cloud scale applications. He is a regular contributor to the DevOps community, a frequent speaker at technology conferences and regularly publishes articles on You can follow him on Twitter: @grabnerandi