One on One with Automated Software Testing Expert Phil Japikse

The upcoming Visual Studio Live 2-Day Hands-On Training Seminar (June 5-6, online), organized by the hard-working folks at 1105 Media (my boss) promises to be a killer opportunity for developers to update their skills and knowledge on an increasingly important topic with the potential to make their lives much easier: automated software testing.

I don't usually promote events in this space, but after talking (via email) with the seminar/workshop's presenter, Phil Japikse, I couldn't help myself. Japikse is an impressive guy. He's been developing software for more than 35 years, and he's been working with .NET since the first betas! He's got an alphabet soup of certifications (Microsoft MVP, ASPInsider, MCSD, PSM II, PSD, PSPO, PST), and he's the Lead Director of the Cincinnati .NET User’s Group and the Cincinnati Software Architect Group, and he founded and runs the CincyDeliver conference. (Guess which city he's from.)

Japikse's day job is Chief Technology Officer at the Pintas & Mullins Law Firm, and he's co-author of the latest editions in the best-selling "Pro C#" book series (Apress Publishing), including Pro C# 10 with .NET 6: Foundational Principles, (with Andrew Troelsen.) You can follow him on twitter. (He volunteers for the National Ski Patrol, which is very cool and explains his handle.)

JKW: What will you be covering in this workshop?

Japikse: We begin with an open discussion of the benefits (and possible friction points) of incorporating automated testing into your standard development practices. We also cover many of the tools used for automated testing in C# and .NET. This includes xUnit, MOQ, Machine Specifications (MSpec), and Microsoft PlayWright. Testing topics include unit testing (TDD and TED), mocking, integration testing, and UI testing.

JKW: Just to be clear, please define "automated software testing."

Japikse: Wikipedia defines it like this: "In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes." That’s not a bad definition, but it might be simpler to just say automated testing is the ability to execute tests and report the results without human intervention.

JKW: What are the different types of automated software tests?

Japikse: There are three main types we will be discussing in the workshop. Unit tests are tests that are very focused and test one particular unit of work, which in C# is a single function. Integration tests test an end-to-end process, like testing a repository for data access. User Interface tests are designed to test functionality by exercising the UI, which in our examples will be a browser.

JKW: What about TDD and TED?

Japikse: TDD stands for Test-Driven Development or Test-Driven Design. It’s the art of creating a test that confirms the behavior you need from the application, but before you develop that behavior. Since you haven’t written the code yet, the test fails. You then write the code to make the test pass. We will spend a good deal of time in the workshop on TDD.

TED is a term that a friend of mine started using years ago, and it stands for Test Eventual Development. It's used to describe the act of going back to already written software and adding tests. This tends to be more difficult and time-consuming than TDD. But in reality, there’s a lot more software that isn’t covered by tests than software that is. We will also cover how to add tests to existing software and the tricks that help this effort, such as mocking out dependencies and creating seams.

JKW: Software testing is an evolving practice affected by emerging development methodologies, tools, and trends (agile, shift left, AI). Where are we right now in that evolution related to tooling?

Japikse: For years developer testing meant running your app and banging on the keyboard, or creating a console app to test different parts of the application—or, more commonly, testing was left to the QA department. Because of the manual nature and lack of tooling, most developers didn’t do much testing besides simple app execution and some very rudimentary tests.

I remember when nUnit was first introduced into the .NET ecosystem. I was extremely excited because we (C# and VB.NET developers) finally had a tool to create automated tests. Since nUnit was introduced, the amount and quality of the available tools have grown exponentially. We now have tools to run a wide variety of automated tests, including unit, integration, and user interface. In addition, we can include automated testing in build and deploy pipelines.

JKW: Where are we right now in that evolution in terms of the involvement of developers in the process? In other words, what is their role today?

Japikse: I remember giving a lunch-and-learn on nUnit for a client shortly after it was released. I was teaching how to leverage nUnit to run and report the results of automated tests. Several of the developers were getting visibly upset with the topic. I stopped my lecture to ask them what was wrong. They all said that this was going to drastically increase the time it would take them to develop. I was confused since nUnit would drastically shorten the time to execute tests. They answered that they weren’t testing now. It wasn’t the tooling that was upsetting them, it was the thought of writing tests for their code that was the problem!

A common misconception is that creating automated testing increases the delivery time. There was a study done at Microsoft some years ago that looked at different teams. Some were using a test-first strategy, some were using a test-eventual strategy, and some groups were using traditional QA departments for their testing. Although the cycle time (the time to deploy a given feature) was slightly higher for those doing automated testing, the throughput (the number of features deployed) was much higher. This was because the quality of their work was much higher, and they had much less rework.

We all know it’s more interesting to work on new features and tedious and boring to fix bugs. If you aren’t including at least some automated testing in your development process, you are going to spend more time fixing bugs and less time building new features.

JKW: What are the chief benefits of incorporating automated testing into your development process?

Japikse: In addition to increasing the throughput and getting to work on new features, a direct benefit of automated testing is ensuring that the code we write works as we expect it to. Another (and more important) benefit is that you make sure the code we write doesn’t break any other code in the system.

JKW: Which projects and tests benefit the most from automated testing?

Japikse: All of them. The more complex or important the system is, the more testing it needs. Software that controls airplanes, for example, must be extremely well tested. One could argue that game software doesn’t need as much testing. It all depends on the business requirements for the application.

JKW: What are the top automated testing tools and why are they at the top of your list? (xUnit, MOQ, etc.)

Japikse: We now have many choices when it comes to automated test tools. In the .NET development space, there’s a host of free and open-source testing frameworks, including xUnit, NUnit, MSTest, MSpec, MOQ, Microsoft Fakes, Selenium, PlayWright, and plenty more.

My tools of choice are xUnit and MSpec for creating and executing unit and integration tests, MOQ for mocking dependencies and verifying behaviors, and PlayWright for UI testing. xUnit was the first framework to fully support .NET Core and is the next evolution of the unit testing frameworks. MSpec is for context specification style testing and was a major tool in my testing arsenal in the .NET framework space. It was recently updated to be compatible with .NET Core. MOQ has a long history in the C# space with rich mocking capabilities and behavior verification and has continued this trend with .NET Core. PlayWright is a new tool, at least for me, and I like the integration capabilities with xUnit.

JKW: How important is it to mock out dependencies when it comes to automated software testing?

Japikse: For unit tests, it’s vital. Unit tests must isolate a single unit of work and eliminate anything that can skew the results. For example, assume you have a method that calculates the sales tax on an item based on the country/state/county that they live in. If the method uses a repository to get the tax tables, and that repository call fails (e.g., the database is down), then the test will fail, but not because the logic for calculating the tax is wrong. It fails because it couldn’t access the database. It’s much better to mock out the dependency so the test is isolated to just the tax calculation. This also allows for testing different scenarios without having to modify the database records. Finally, it’s much more efficient to eliminate database calls when running automated tests.

JKW: Are organizations embracing automated testing with support for developers? How do developers feel about it?

Japikse: I don’t have any hard facts for you, but I feel like more companies are embracing automated tests. More and more developers I see at conferences and attending my trainings are at least familiar with the terms and tools but might be in different stages of adoption.

JKW: Finally, I'm wondering how important is automated testing expertise to the careers of developers today?

Japikse: When hiring, it’s a key factor I look for when someone says they are a senior developer.

Posted by John K. Waters on May 15, 2023