What is Testing Code?

And how does it cost you less than nothing?

Cody Towstik

Published in

Dev Genius

13 min readMay 28, 2020

Photo by National Cancer Institute on Unsplash

TL;DR

Testing code saves time and effort for both you, and developers that come after.

This article will touch on:

What testing is, using some trivial examples to elucidate this matter
Why you should (neigh, need!) to test your code
Sample testing approaches, and how they benefit you over options

If you’d like to jump ahead to any sections, or need to quickly access them later, here’s the low down:

1. So… what is testing?
2. Types of Tests
3. Why is Testing Useful?
4. Keeping Tests Clean
5. Powerful Testing Tools
6. Overview

Foreword 💃

Any Computer Science program worth its salt will have introduced you to the concept of testing your code. No, I don’t mean littering your code with console.log(‘asdf’) or running your program over and over again at 5 o’ clock in the morning before your assignment.

When I was in school, the course that finally mentioned what it meant to test code didn’t come until my second year there. That’s an entire year of crying over forgotten edge cases or pressing up on the terminal to find previous runs.

If you’re learning to write code online, you’ll probably hear about testing your code day zero. You will likely find many tutorials implementing a popular testing framework. Maybe you’ll find long monologues with words you barely understand about why you should test code. All way before understanding really what testing is, and what’s going on behind the scenes.

Whether you’re learning to code through a university or just risking it for the biscuit and learning completely online, there is SO much information to process. Why waste time on testing??? Why waste time on something that doesn’t feel like progress, that doesn’t generate sick hax0r terminal output, nor sexy pixels on the ‘front end’?

It’s sort of like learning to use an IDE properly, there are already thousands
of things to learn, why should you take the time to do things efficiently?

Efficient

(especially of a system or machine) achieving maximum productivity with minimum wasted effort or expense.

So… what is testing?

Testing code means to run your code with a controlled input, and checking
for an expected output. The example provided in the foreword demonstrates a very basic example of ‘testing code’, whereby the user is running their python file divide.py with various arguments.

This is generally a flawed approach to test any program that you expect to ever make changes to, or that scales beyond just a few lines of code.

What’s the issue, dear?

You’ll need to remember all of the commands that you have already used to test your code.
Results of each execution are only being verified by your fallible human eyeballs.
After every change, you need to manually re-run each test.
Manually inputting each command means you may input incorrect parameters.
…
I’ll leave the rest up to your imagination.

Before we jump into potential full-fledged approaches to testing, let’s take a look at a couple of increasingly better examples.

Approach 0

See this in a gist.

Some areas where we might improve from our terminal command testing approach:

Running tests should be easily repeatable.
We want to be able to run all tests at once.

S’pose you have a program that loads a database with the name of each Pokemon you have caught, and how many you have. This is your Pokedex. So you write a function that, given the Pokedex state and a list of Pokemon, returns how many of those specified Pokemon you own in total.

Of course, you lack experience with the debugger and just want some quick results, so you log the result before returning.

Being the pragmatic programmer you are, you realize that you should (neigh, need!) to test this function to make sure it works given various inputs you expect it to be called with.

You don’t want to start up your database every time, and quickly realize you can mock some sample test data in a map.

Now you can call your function with a few various expected inputs, and watch for the right outputs.

Doh! It seems like we haven’t handled the case where a Pokemon that we don’t have yet is requested. Since there are only four ‘tests’, it wasn’t too hard to match up the output to each test. But, we have to visually verify each result and remember what is expected for each test. For a small sample program, this isn’t a bad approach. But, even after only adding a few more tests, it would get tedious real quick. Even if we just change some of the sample data, remembering the expected result for each test is untenable and incredibly error prone.

We want to save brain cycles, and consider an automagic and repeatable approach. Let’s do one better.

Approach 1

See this in a gist.

Let’s remember some areas of improvement for our next approach:

Test results should be verified programmatically, rather than visually.
It should be easy to see the results and match them to the corresponding test.

Instead of just a bare console.log(), let’s add some flavor with a new function that will help you, the developer, to identify which tests went wrong and what the actual output versus the expected output was.

Now we can programmatically check the results of each function, and easily
identify which test function it belongs to.

Cool, the first two tests passed so we know something is going right. Using the testID, we effortlessly know our function works with empty data and when we call it with Pokemon we already have. The two failed outputs inform us that we don’t even get numbers — NaN — when we use Pokemon that we don’t have. We didn’t even have to look back at the test function calls, nice!

Now we have a clue as to what is going wrong. Going back to our calculateTotalSpecifiedPokemonCaught function, we can use the debugger to find out that the pokedexState[ currentPokemon ] expression returns undefined when the Pokemon doesn’t exist in our Pokedex (see line 16 in the gist).

Let’s split getting our Pokemon data from the Pokedex and incrementing the count into separate lines. If the Pokemon doesn’t exist and we get NaN, we’ll just count it as zero.

See the completed example in a gist.

It is often a good idea to split complex logic into separate lines, and is a good
practice towards creating clean code.
Imagine debugging this nightmare of a line:
let currentPokemonCount = isNaN( pokedexState[ currentPokemon ] ) ? 0 : pokedexState[ currentPokemon ]

If you aren’t building a complex program, this approach may be enough for you. Additionally, separating tests into different files for different sets of related code is a good way to keep things organized. Tests should be easy to understand at first glance, so don’t be afraid to put comments. At minimum, include a blurb about what each test is testing for, and possibly why its a necessary test if the reason isn’t obvious.

Tests should be easy to understand at first glance.

Types of Tests

The different types of test you should have is often represented as a pyramid.

A strong pyramid needs a strong foundation. The lower on the pyramid, the smaller unit of code that is being tested, and the more of those tests you should have. One UI (E2E) test might test the flow of creating, editing, and then saving a document. That flow is composed of many different functions, each of which should have their own unit tests.

The examples given earlier in So… what is testing? describe a type of test called unit tests. They are the most simple test and, as you might assume, tests a single unit of code. Typically, this means an individual function, but you’ll often find them as a collection of related methods. When a test uses multiple units to perform a single operational flow, it is typically referred to as a Component test.

For example, if we had an API call that accepts as parameters a userID and the requestedPokemon to check the count of, we could mock those variables and test the entire flow of the internals of the API.

Perhaps something like:

These are considered White Box tests because we know the internals of the code and can directly pass functions test arguments and
programmatically test each individual output.

Unit Tests are written from a *programmers* perspective. They are made to ensure that a particular method (or a unit) of a class performs a set of specific tasks.

Functional Tests are written from the *user’s* perspective. They ensure that the system is functioning as users are expecting it to.

StackOverflow, Anthony Forlorney

Functional tests, and integration tests, are considered Black Box testing because the internals are a ‘black box’. The tester doesn’t consider the internal program structure. This is a sort of Quality Assurance (QA) process. Using our python divide.py file as an example: a QA tester only cares if they input one as the numerator and ten as the denominator — 1/10 — they get back 0.01. Maybe it is important for your program to keep leading zeros on decimals, too. So good job! The user doesn’t care if you are using Numpy to do this calculation, nor if you decided to use bitwise math. Only getting the expected outputs, in the correct form, for given inputs.

A type of functional testing is API testing. On the server side, this might mean using a test client that supplies your API with expected mock input, verifying their output. Potentially chaining multiple API calls to mock a flow of interactions.

On the client side, this might mean running the server and making API calls, verifying the response of each one. However, this kind of testing only tests the data layer of your application.

You want to check the view layer as well. There are frameworks that let you test the visual elements that the user actually sees rendered to the screen, covered later in Powerful Testing Tools. This is called UI testing, where your chosen framework will be programmatically ‘clicking’ like an actual user would, not directly interacting with the code at all.

The lines between each testing paradigm on the pyramid can be blurred, and they serve as more of a guideline rather than a hard set rule. If you are using a test client on the server to test your API with mock input, it is technically Black Box testing because your test client doesn’t have direct access to the code. Even though you as a developer can read and understand the code behind each API call, from the perspective of the test itself, there is a degree of separation. If you ask 20 developers to describe each kind of test in the pyramid, you’ll probably get 20 different answers.

This only scratches the surface on how you can test your programs, so please take some time to explore other resources that are more directed towards more in-depth explanations and examples. Use the bolded keywords above as a starting point for your searches.

Why is Testing Useful?

We have seen that testing helps us verify that our code works for a set of specified inputs. This saves developers valuable time whilst debugging, QA people can now verify our code does what we say it does.

The benefit of testing goes well beyond just the initial verification of code.

Testing helps future proof your code by:

Protecting you from code or framework changes.

If another developer comes along in the future and modifies the code, they’ll know for certain if their changes broke the spec. They might try to optimize parts of the code that breaks some important edge cases, which would suck to catch way down the road. Again, think about this in the context of working on a full scale app with many devs making changes every day, multiple times a day.

Even for non-local changes, this can save you. If you upgrade one of your libraries that changes THEIR code and is suddenly returning a different result to your program, you’ll know immediately where the issue is.

For non-code changes, like if you are switching to a new compiler that unwraps your for loop backwards (haha, wut?), you’ll know right away.

Providing documentation for future devs (that could be you, too!).

Ever look at a new codebase (especially one with a dynamically typed language, *cough* Javascript, *cough cough* Python), and it takes you a while to figure out how certain functions are expected to be used? Or “why is this button here and what it is it for?”

Well, if the code base has a healthy set of tests accompanying the code, all you need to do is find the corresponding test and see what some expected inputs and outputs are. It is a way for the developer who implemented the code to say, “Yes I expected the results to be rounded!” or, “We should throw an expection when the Pokemon doesn’t exist in our Pokedex, not count it as zero.”

You can practice using your library or framework, and verify it works how you expect.

You aren’t writing production code, so tests are the perfect place to check out tools and methods in your shiny new framework that you haven’t used before. This is also an easy way to document some example usages with the framework in the context of your project, and describe some best practices.
Future devs will thank you for it, especially if they aren’t familiar with the library themselves.

Plus, you can make sure that your external libraries are accepting data in the form you expect, and returning it in the expected form. Even if you think you know that it does, now it is programmatically verified. This follows points 1 and 2, as it protects you from changes due to future updates and acts as further documentation.

You should always add a layer of abstraction around the libaries you use. Checkout this talk given at DinosaurJS by Daria Caraway about
How to Have an Amicable Breakup With a JavaScript Library

Say it with me: ”We love good documentation, whoo!”

Keeping Tests Clean

Testing code is a powerful and necessary tool. With great power, comes great responsibility. Please don’t treat your tests as second-class citizens. I’m not saying you should necessarily hold your test code to the same rigorous standards as production code, but it should be close.

Tests should be easy to understand at first glance.

Keeping your tests commented so it is clear what is being tested, and why it is being tested. Extract common patterns used multiple times within or across tests into utility functions with clear and documentation.

Just like in production code, variables should have long descriptive names, and the code should closely adhere to your general code style guidelines.

Letting proper code style fall by the wayside for your tests is the first descent into chaos that you should seek to avoid at all cost. Bad code style detracts from readability, which may lead to confusion about what is actually being tested and is it being tested right.

Tests should be organized and easy to find and track.

Related tests should be grouped together in individual files. Consider a directory structure for all of your test files that makes sense to you, and you can easily describe to a team. Almost worse than not having tests is not knowing what is being tested, or where it is tested. Think about strategies you can use to make sure it as easy as possible to find a test, and to verify that a feature is being tested in all the ways you expect.

Maybe you always put new tests at the bottom of a file so things don’t move around too much, or you put a comment on the test of a particular feature.
As all things programming, this is more of an art than a science.

Powerful Testing Tools

The earlier you start testing in your project lifecycle, the more potential for time saved and the less risk of breaking something trying to fix something else.

You will thank yourself later that you did, and pick up a valuable skillset along the way! Even for very small one off projects, like a school assignment or a small tool, I implore you to implement at least a very basic testing system. Use some examples or ideas presented in this article, or build out an entire testing suite. Get in the habit of including this in your workflow.

For any project that evolves beyond a simple tool, it is likely you will want a powerful framework to help you out. Testing frameworks already have the boring details figured out and are usually documented with a few best practices.

The defacto test framework for Java — JUnit— has dozens of assert methods that are implemented similarly to our checkResults(..) method used in the earlier examples. Anything from assertEquals(..) to assertNotNull(..)

Javascript projects might use Jest or Mocha for unit, integration, or other data layer tests.

For visual tests, you might rely on Cypress or ye ol’ Selenium.

Regardless of the language you are using, choosing a framework is a personal choice. Many testing frameworks behave in similar ways and offer similar tools. For example, all of these frameworks have their own implementation of a @BeforeEach annotation, which runs a block of code before each and every test, to remove boilerplate like logging in each time.

Don’t worry about getting caught up in “which tool is the best”, because if you write good tests with any framework, you’re a step ahead of the game.

Overview

Testing costs you less than nothing to implement, saving you time and giving everyone peace of mind.You know what it means to test code, so stop shooting yourself in the foot and start testing!

Tests help…

protect you from code or framework changes.
provide documentation for future devs (that could be you, too!)
you practice using your library or framework, and verify it works how you expect.

Tests should…

be easy to read and understand.
closely adhere to your general code style guidelines.
be organized and easy to find and track.

If you want to go back: