New Course! RST: Testing, Automation, and AI
Organized by Nine and Qalibrate
“Test automation” is a hot topic. If you’re a programmer, you’re probably thinking in terms of testing as a straightforward programming task. If you are in the testing field, then you are probably being pressured to automate your work. And now along comes AI, with its own set of promises, problems, and pressures. To deploy or apply tools responsibly, you must know how to look at them critically and avoid the traps. This class is designed to help you learn how to do that.
What About Tools?
Creating, applying, and maintaining powerful tools to support testing can be immensely valuable. If you want to do those things well, they can also present huge challenges. This three-day class is designed to help you help you meet the challenges, looking through the lenses of the Rapid Software Testing approach, starting with this foundational idea: testing is evaluating a product by learning about it through experiencing, exploring, and experimenting.
So how should you go about applying tools in testing? What about “automated testing”? Only certain specific aspects of testing can be automated, so which aspects should you focus on? What comes first? There are lots of expensive tools you could use, but there are also free tools, and maybe you can create some tools yourself. What should you NOT focus on? What traps must you try to avoid? These questions have always been important, but they’re now more critical than ever, because…
What About AI?
Over the last few years, “AI” — “artificial intelligence” — has gone from a set of emerging technologies to an industry obsession. Testers are being pressed to test products with AI features, and to use AI in their work. Meanwhile, there is uncertainty and controversy about what “AI” even means, since AI is fundamentally a marketing term, rather than an engineering term.
Whether designed for classification or prediction, or for generating text or code, all forms of AI have this in common: they are black boxes, imbued with magical behaviours and properties, whose behaviour is neither controlled, nor understood, nor otherwise known to be safe. A responsible approach requires us to know enough about them to be aware of the risks—which means we must know how to test them.
Much testing has traditionally been framed as formalized, procedurally structured test cases that check output against specific, prescribed results. That won’t work for GPT-based technologies whose output is non-deterministic by design. The Rapid Software Testing approach—and its approach to tools and automation—is designed to address that problem by helping you learn how to test for real.
In this three-day class, we’ll help you—and through you, your organization—to expand notions of testing and automation beyond GUI or API output checking to focus on business risk. We will show you some creative uses of inexpensive tools to probe data. We’ll present tools that can help you visualize and report on test coverage. We’ll help you to analyze dimensions of cost and value, and to evaluate how different kinds of tools can help or hurt your testing. We’ll talk about how to recognize and learn from things that happen in the secret life of testing and automation. And finally, we’ll look critically and pragmatically at AI — both in terms of how to test it, and how we might apply it in testing work.
Maybe you’re a coder; maybe you’re not. This class is designed to help you either way. If don’t write code, we won’t teach you how, but we will help you learn to work immediately and productively with people who do.
This class is not affiliated with any tool vendor. We’ll use tools in class, but we don’t teach you to operate any specific tool. We may demonstrate or mention particular tools during the class, but we have not accepted and will not accept any payment or benefit from commercial interests. (We may be biased, but we have not been bribed.) We will show you how to analyze vendor claims critically.
In short, this class is about the essence of tool strategy: vision and setting goals that make sense for you and your organization.
Check out comments from recent classes here:
Endorsements
Course Description
Read the RST: Testing, Automation, and AI full course description