There are many ways that you can test a React application. Early on in a project, when you are still defining an application’s essential purpose and function, you might choose to create tests in some very high-level form, such as Cucumber tests. If you are looking at some isolated piece of the system (such as creating and maintaining a data item), you might want to create functional tests using a tool like Cypress.

But if you are deep into the detail of creating a single component, then you will probably want to create unit tests. Unit tests are so-called because they attempt to test a single piece of code as an isolated unit. While it’s debatable whether unit test is the correct term for testing components (which often contain subcomponents and so are not isolated), it’s the name usually applied to tests of components that you can test outside of a browser.

But how do you unit test React components? There have historically been several approaches. Early unit tests relied on rendering the component into an HTML string, which required minimal testing infrastructure, but there were multiple downsides:

However, tests created in this way lacked the reality of tests created in browsers. The subtleties of the interaction between the virtual Document Object Model (DOM) and the browser DOM were lost. Often subcomponents were not rendered to reduce the complexity of the tests.

The Testing Library by Kent C. Dodds attempts to avoid the issues with previous unit testing libraries by providing a standalone implementation of the DOM. As a result, tests can render a React component to a virtual DOM, which can then be synchronized with the Testing Library’s DOM and create a tree of HTML elements that behave like they would in a real browser.

If you created your application with create-react-app, you should already have the Testing Library installed. If not, you can install it from the command line:

$ npm install --save-dev "@testing-library/react"
$ npm install --save-dev "@testing-library/jest-dom"
$ npm install --save-dev "@testing-library/user-event"

The Testing Library allows us to render components using the DOM implementation in @testing-library/jest-dom. The User Event library (@testing-library/user-event) simplifies interacting with the generated DOM elements. This User Event library allows us to click the buttons and type into the fields of our components.

To show how to unit test components, we will need an application to test. We’ll be using the same application through much of this chapter. When the application opens, it asks the user to perform a simple calculation. The application will say if the user’s answer is right or wrong (see Figure 8-1).

Figure 8-1. The application under test

The main component of the application is called App. We can create a unit test for this component by writing a new file called App.test.js:

describe('App', () => {
  it('should tell you when you win', () => {
    // Given we've rendered the app
    // When we enter the correct answer
    // Then we are told that we've won
  })
})

The preceding code is a Jest test, with a single scenario that tests that the App component will tell us we’ve won if we enter the correct answer. We’ve put placeholder comments for the structure of the test.

We will begin by rendering the App component. We can do this by importing the component and passing it to the Testing Library’s render function:

import { render } from '@testing-library/react'
import App from './App'

describe('App', () => {
  it('should tell you when you win', () => {
    // Given we've rendered the app
    render(<App />)

    // When we enter the correct answer
    // Then we are told that we've won
  })
})

Notice that we pass actual JSX to the render function, which means that we could, if we wanted, test the component’s behavior when passed different sets of properties.

For the next part of the test, we’ll need to enter the correct answer. To do that, we must first know what the correct answer is. The puzzle is always a randomly generated multiplication, so we can capture the numbers from the page and then type the product into the Guess field.1

We will need to look at the elements generated by the App component. The render function returns an object that contains the elements and a set of functions for filtering them. Instead of using this returned value, we’ll instead use the Testing Library’s screen object.

const input = screen.getByLabelText(/guess:/i)

The filter methods in the screen object typically begin with:

const { container } = render(<App />)
const theInput = container.querySelector('#guess')

There is one small concession to this approach made by the filter functions: the ...ByTestId functions. If you have no practical way of finding an element by its content, you can always add a data-testid attribute to the relevant tag. That is useful for the test we are currently writing because we need to find two numbers displayed on the page. And these numbers are randomly generated, so we don’t know their content (Figure 8-2).

Figure 8-2. We cannot find the numbers by content because we won’t know what they are

So, we make a small amendment to the code and add test IDs:

<div className="Question-detail">
  <div data-testid="number1" className="number1">
    {pair && pair[0]}
  </div>
  &times;
  <div data-testid="number2" className="number2">
    {pair && pair[1]}
  </div>
  ?
</div>

This means we can start to implement the next part of our test:

import { render, screen } from '@testing-library/react'
import App from './App'

describe('App', () => {
  it('should tell you when you win', () => {
    // Given we've rendered the app
    render(<App />)

    // When we enter the correct answer
    const number1 = screen.getByTestId('number1').textContent
    const number2 = screen.getByTestId('number2').textContent
    const input = screen.getByLabelText(/guess:/i)
    const submitButton = screen.getByText('Submit')
    // Err...

    // Then we are told that we've won
  })
})

We have the text for each of the numbers, and we have the input element. We now need to type the correct number into the field and then submit the answer. We’ll do this with the @testing-library/user-event library. The User Event library simplifies the process of generating JavaScript events for HTML elements. You will often see the User Event library imported with the alias user, which is because you can think of the calls to the User Event library as the actions a user is making:

import { render, screen } from '@testing-library/react'
import user from '@testing-library/user-event'
import App from './App'

describe('App', () => {
  it('should tell you when you win', () => {
    // Given we've rendered the app
    render(<App />)

    // When we enter the correct answer
    const number1 = screen.getByTestId('number1').textContent
    const number2 = screen.getByTestId('number2').textContent
    const input = screen.getByLabelText(/guess:/i)
    const submitButton = screen.getByText('Submit')
    user.type(input, '' + parseFloat(number1) * parseFloat(number2))
    user.click(submitButton)

    // Then we are told that we've won
  })
})

Finally, we need to assert that we have won. We can write this simply by looking for some element containing the word won:2

// Then we are told that we've won
screen.getByText(/won/i)

This assertion will work because getByText throws an exception if it does not find precisely one matching element.

However, the test is liable to break if your application is running slowly. This is because the get... and query... functions look at the existing state of the DOM. If the result takes a couple of seconds to appear, the assertion will fail. For this reason, it’s a good idea to make some assertions asynchronous. It makes the code a little more complex, but the test will be more stable when running against slow-moving code.

The find... methods are asynchronous versions of the get... methods, and the Testing Library’s waitFor will allow you to rerun code for a period of time. By combining the two functions, we can create the final part of our test:

import { render, screen, waitFor } from '@testing-library/react'
import user from '@testing-library/user-event'
import App from './App'

describe('App', () => {
  it('should tell you when you win', async () => {
    // Given we've rendered the app
    render(<App />)

    // When we enter the correct answer
    const number1 = screen.getByTestId('number1').textContent
    const number2 = screen.getByTestId('number2').textContent
    const input = screen.getByLabelText(/guess:/i)
    const submitButton = screen.getByText('Submit')
    user.type(input, '' + parseFloat(number1) * parseFloat(number2))
    user.click(submitButton)

    // Then we are told that we've won
    await waitFor(() => screen.findByText(/won/i), { timeout: 4000 })
  })
})

You can download the source for this recipe from the GitHub site.

Tests are simply examples that you can execute. Consequently, tests have a lot in common with component gallery systems like Storybook. Both tests and galleries are examples of components running in particular circumstances. Whereas a test will make assertions with code, a developer will make an assertion of a library example by looking at it and checking that it appears as expected. In both galleries and tests, exceptions will be easily visible.

We’re going to look at how you can reuse your Storybook stories inside tests. You can install Storybook into your application with this command:

$ npx sb init

The example application in this chapter is a simple mathematical game in which the user needs to calculate the answer to a multiplication problem (see Figure 8-3).

Figure 8-3. The example application

One of the components in the game is called Question, and it displays a randomly generated multiplication question (Figure 8-4).

Figure 8-4. The Question component

Let’s say we don’t worry too much about tests for this component. Let’s just build it by creating some Storybook stories. We’ll write a new Question.stories.js file:

import Question from './Question'

const Info = {
  title: 'Question',
}

export default Info

export const Basic = () => <Question />

And then we’ll create an initial version of the component that we can look at in Storybook and be happy with:

import { useEffect, useState } from 'react'
import './Question.css'

const RANGE = 10

function rand() {
  return Math.floor(Math.random() * RANGE + 1)
}

const Question = ({ refreshTime }) => {
  const [pair, setPair] = useState()

  const refresh = () => {
    setPair((pair) => {
      return [rand(), rand()]
    })
  }

  useEffect(refresh, [refreshTime])

  return (
    <div className="Question">
      <div className="Question-detail">
        <div data-testid="number1" className="number1">
          {pair && pair[0]}
        </div>
        &times;
        <div data-testid="number2" className="number2">
          {pair && pair[1]}
        </div>
        ?
      </div>
      <button onClick={refresh}>Refresh</button>
    </div>
  )
}

export default Question

This component displays a randomly generated question if the user clicks the Refresh button or if a parent component passes in a new refreshTime value.

We display the component in Storybook, and it looks like it works fine. We can click the Refresh button, and it refreshes. So at that point, we start to use the component in the main application. After a while, we add a few extra features, but none of them are visual changes, so we don’t look at the Storybook stories for it again. After all, it will still look the same. Right?

This is a modified version of the component, after we’ve wired it into the rest of the application:

import { useEffect, useState } from 'react'
import './Question.css'

const RANGE = 10

function rand() {
  return Math.floor(Math.random() * RANGE + 1)
}

const Question = ({ onAnswer, refreshTime }) => {
  const [pair, setPair] = useState()
  const result = pair && pair[0] * pair[1]

  useEffect(() => {
    onAnswer(result)
  }, [onAnswer, result])

  const refresh = () => {
    setPair((pair) => {
      return [rand(), rand()]
    })
  }

  useEffect(refresh, [refreshTime])

  return (
    <div className="Question">
      <div className="Question-detail">
        <div data-testid="number1" className="number1">
          {pair && pair[0]}
        </div>
        &times;
        <div data-testid="number2" className="number2">
          {pair && pair[1]}
        </div>
        ?
      </div>
      <button onClick={refresh}>Refresh</button>
    </div>
  )
}

export default Question

This version is only slightly longer than before. We’ve added an onAnswer callback function that will return the correct answer to the parent component each time the application generates a new question.

The new component appears to work well in the application, but then an odd thing occurs. The next time someone looks at Storybook, they notice an error, as shown in Figure 8-5.

Figure 8-5. An error occurs when we look at the new version of the component

What happened? We’ve added an implicit assumption into the code that the parent component will always pass an onAnswer callback into the component. Because the Storybook stories rendered Basic story without an onAnswer, we got the error:

<Question/>

Does this matter? Not for a simple component like this. After all, the application itself still worked. But failure to cope with missing properties, such as the missing callback here or, more frequently, missing data, is one of the most typical causes of errors in React.

In the test file, we can then create a Jest test that loads each of the stories and then passes them to the Testing Library’s render function:4

import { render } from '@testing-library/react'
import Question from './Question'

const stories = require('./Question.stories')

describe('Question', () => {
  it('should render all storybook stories without error', () => {
    for (let story in stories) {
      if (story !== 'default') {
        let C = stories[story]
        render(<C />)
      }
    }
  })
})

onAnswer is not a function
TypeError: onAnswer is not a function

useEffect(() => {
  // We need to check to avoid an error
  if (onAnswer && result) {
    onAnswer(result)
  }
}, [onAnswer, result])

import { render, screen } from '@testing-library/react'
import user from '@testing-library/user-event'
import Question from './Question'
import { Basic, WithDisabled } from './Question.stories'
...
it('should disable the button when asked', () => {
  render(<WithDisabled />)
  const refreshButton = screen.getByRole('button')
  expect(refreshButton.disabled).toEqual(true)
})

You can download the source for this recipe from the GitHub site.

One of the principal features of high-quality code is the way it responds to errors. The first of Peter Deutsch’s Eight Fallacies of Distributed Computing is: the network is reliable. Not only is the network not reliable, but neither are the servers or databases that connect to it. At some point, your application is going to have to deal with some network failure. It might be that the phone loses its connection, or the server goes down, or the database crashes, or someone else has deleted the data you are trying to update. Whatever the causes, you will need to decide what your application will do when terrible things happen.

For this recipe, we are going to use Cypress. We mentioned the Cypress testing system in Chapter 1. It’s a genuinely remarkable testing system that is rapidly becoming our go-to tool in many development projects.

To install Cypress into your project, type the following:

$ npm install --save-dev cypress

Cypress works by automating a web browser. In that sense, it is similar to other systems like Selenium. Still, the difference is that Cypress does not require you to install a separate driver, and it can both control the browser remotely and inject itself into the browser’s JavaScript engine.

Cypress can therefore actively replace core parts of the JavaScript infrastructure with faked versions that it can control. For example, Cypress can replace the JavaScript fetch function used to make network calls to the server.⁵ Cypress tests can therefore spoof the behavior of a network server and allow a client-side developer to artificially craft responses from the server.

We will use the example game application that we use for other recipes in this chapter. We will add a network call to store the result each time a user answers a question. We can do this without creating the actual server code by faking the responses in Cypress.

To show how this works, we will first create a test that simulates the server responding correctly. Then we will create a test to simulate a server failure.

Once Cypress is installed, create a file in cypress/integration/ called 0001-basic-game-functions.js:⁶

describe('Basic game functions', () => {
  it('should notify the server if I lose', () => {
    // Given I started the application
    // When I enter an incorrect answer
    // Then the server will be told that I have lost
  })
})

We’ve put placeholder comments for each of the steps we will need to write.

Each command and assertion in Cypress begins with cy. If we want to open the browser at location http://localhost:3000, we can do it with the following:

describe('Basic game functions', () => {
  it('should notify the server if I lose', () => {
    // Given I started the application
    cy.visit('http://localhost:3000')

    // When I enter an incorrect answer
    // Then the server will be told that I have lost
  })
})

To run the test, we can type:

$ npx cypress run

That command will run all tests without showing the browser.7 We can also type the following:

$ npx cypress open

This command will open the Cypress application window (as you can see in Figure 8-6). If we double-click the test file, the test will open in a browser (as you can see in Figure 8-7).

Figure 8-6. The test will appear in the Cypress window when you type npx cypress open

Figure 8-7. A Cypress test running in a browser

The example application asks the user to multiply two random numbers (see Figure 8-8). The numbers will be in the range 1–10, so if we enter the value 101, we can be sure that the answer will be incorrect.

We will see elsewhere in this chapter how we can remove randomness in test scenarios and make this test deterministic, which will remove the need to capture data from the page.

Figure 8-8. The application asks the user to calculate the product of two random numbers

We can use the cy.get command to find the input field by a CSS selector. We can also use the cy.contains command to find the Submit button:

describe('Basic game functions', () => {
  it('should notify the server if I lose', () => {
    // Given I started the application
    cy.visit('http://localhost:3000')

    // When I enter an incorrect answer
    cy.get('input').type('101')
    cy.contains('Submit').click()

    // Then the server will be told that I have lost
  })
})

Now we just need to test that the application contacts the server with the result of the game.

We will use the cy.intercept() command to do this. The cy.intercept() command will change the behavior of network requests in the application so that we can fake responses for a given request. If the result is going to be POSTed to the endpoint /api/result, we generate a faked response like this:

cy.intercept('POST', '/api/result', {
  statusCode: 200,
  body: '',
})

describe('Basic game functions', () => {
  it('should notify the server if I lose', () => {
    // Given I started the application
    cy.intercept('POST', '/api/result', {
      statusCode: 200,
      body: '',
    })
    cy.visit('http://localhost:3000')

    // When I enter an incorrect answer
    cy.get('input').type('101')
    cy.contains('Submit').click()

    // Then the server will be told that I have lost
  })
})

We will need to give the network request an alias. This will allow us to refer to the request later in the test:9

cy.intercept('POST', '/api/result', {
  statusCode: 200,
  body: '',
}).as('postResult')

We can then make an assertion at the end of the test, which will wait for the network call to be made and will check the contents of the data sent in the request body:

describe('Basic game functions', () => {
  it('should notify the server if I lose', () => {
    // Given I started the application
    cy.intercept('POST', '/api/result', {
      statusCode: 200,
      body: '',
    }).as('postResult')
    cy.visit('http://localhost:3000')

    // When I enter an incorrect answer
    cy.get('input').type('101')
    cy.contains('Submit').click()

    // Then the server will be told that I have lost
    cy.wait('@postResult').then((xhr) => {
      expect(xhr.request.body.guess).equal(101)
      expect(xhr.request.body.result).equal('LOSE')
    })
  })
})

This assertion is checking two of the attributes of the request body for the expected values.

If we run the test now, it will pass (as you can see in Figure 8-9).

Figure 8-9. The completed test passes

Now that we’ve created a test for the successful case, we can write a test for the failure case. The application should display a message on-screen if the network call fails. We don’t actually care what details are sent to the server in this test, but we still need to wait for the network request to complete before checking for the existence of the error message:

it('should display a message if I cannot post the result', () => {
  // Given I started the application
  cy.intercept('POST', '/api/result', {
    statusCode: 500,
    body: { message: 'Bad thing happened!' },
  }).as('postResult')
  cy.visit('http://localhost:3000')

  // When I enter an answer
  cy.get('input').type('16')
  cy.contains('We are unable to save the result').should('not.exist')
  cy.contains('Submit').click()

  // Then I will see an error message
  cy.wait('@postResult')
  cy.contains('We are unable to save the result')
})

In addition to generating stubbed responses and status codes, cy.intercept can perform other tricks, such as slowing response times, throttling network speed, or generating responses from test functions. For further details, see the cy.intercept documentation.

Discussion

Cypress testing can transform how a development team works, specifically in its ability to mock network calls. Teams frequently develop APIs at a different cadence than frontend code. Also, some teams have developers who specialize in frontend or server code. Cypress can help in these situations because it allows frontend developers to write code against endpoints that don’t currently exist. Cypress can also simulate all of the pathological failure cases.

Network performance can introduce intermittent bugs. Development environments use local servers with little or no data, which means that API performance is far better at development time than in a production environment. It is straightforward to write code that assumes that data is immediately available, but this code will break in a production environment where the data may take a second or so to arrive.

cy.intercept('GET', '/api/widgets', {
  statusCode: 200,
  body: [{ id: 1, name: 'Flange' }],
  delay: 1000,
}).as('getWidgets')

You can download the source for this recipe from the GitHub site.

This recipe uses a custom Cypress command invented by Etienne Bruines.

Applications need to cope with being disconnected from the network. We’ve seen elsewhere how to create a hook to detect if we are currently offline.¹⁰ But how are we test for offline behavior?

Solution

We can simulate offline working using Cypress. Cypress tests can inject code that modifies the internal behavior of the browser under test. We should therefore be able to modify the network code to simulate offline conditions.

$ npm install --save-dev cypress

describe('Offline working', () => {
  it(
    'should tell us when we are offline',
    { browser: '!firefox' },
    () => {
      // Given we have started the application
      // When the application is offline
      // Then we will see a warning
      // When the application is back online
      // Then we will not see a warning
    }
  )
})

describe('Offline working', () => {
  it(
    'should tell us when we are offline',
    { browser: '!firefox' },
    () => {
      // Given we have started the application
      cy.visit('http://localhost:3000')

      // When the application is offline
      // Then we will see a warning
      // When the application is back online
      // Then we will not see a warning
    }
  )
})

cy.network({ offline: true })
cy.network({ offline: false })

Cypress.Commands.add('network', (options = {}) => {
  Cypress.automation('remote:debugger:protocol', {
    command: 'Network.enable',
  })

  Cypress.automation('remote:debugger:protocol', {
    command: 'Network.emulateNetworkConditions',
    params: {
      offline: options.offline,
      latency: 0,
      downloadThroughput: 0,
      uploadThroughput: 0,
      connectionType: 'none',
    },
  })
})

This command uses the remote debugging protocol in DevTools to emulate offline network conditions. Once you have saved this file, you can then implement the rest of the test:

describe('Offline working', () => {
  it(
    'should tell us when we are offline',
    { browser: '!firefox' },
    () => {
      // Given we have started the application
      cy.visit('http://localhost:3000')
      cy.contains(/you are currently offline/i).should('not.exist')

      // When the application is offline
      cy.network({ offline: true })

      // Then we will see a warning
      cy.contains(/you are currently offline/i).should('be.visible')

      // When the application is back online
      cy.network({ offline: false })

      // Then we will not see a warning
      cy.contains(/you are currently offline/i).should('not.exist')
    }
  )
})

If you run the test now, in Electron, it will pass (see Figure 8-10).

Figure 8-10. You can view each stage of the online/offline test by clicking on the left panel

Discussion

It should be possible to create similar commands that simulate various network conditions and speeds.

For more information on how the network command works, see this blog post from Cypress.io.

You can download the source for this recipe from the GitHub site.

Nothing beats running your code inside a real browser, and the most common way of writing automated browser-based tests is by using a web driver. You can control most browsers by sending a command to a network port. Different browsers have different commands, and a web driver is a command-line tool that simplifies controlling the browser.

We are going to use the Selenium library. Selenium is a framework that provides a consistent API for a whole set of different web drivers, which means that you can write a test for Firefox and the same code should work in the same way for Chrome, Safari, and Edge.11

We will use the same example application that we are using for all recipes in this chapter. It’s a game that asks the user for the answer to a simple multiplication problem.

The Selenium library is available for a whole set of different languages, such as Python, Java, and C#. We will be using the JavaScript version: Selenium WebDriver.

$ npm install --save-dev selenium-webdriver

$ npm install --save-dev chromedriver

We can now start to create a test. It’s useful to include Selenium tests inside the src folder of the application, because it will make it easier to use an IDE to run the tests manually. So we’ll create a folder called src/selenium and then add a file inside it called 0001-basic-game-functions.spec.js:12

describe('Basic game functions', () => {
  it('should tell me if I won', () => {
    // Given I have started the application
    // When I enter the correct answer
    // Then I will be told that I have won
  })
})

We have outlined the test in the comments.

import { Builder } from 'selenium-webdriver'
let driver

describe('Basic game functions', () => {
  beforeEach(() => {
    driver = new Builder().forBrowser('chrome').build()
  })

  afterEach(() => {
    driver.quit()
  })

  it('should tell me if I won', () => {
    // Given I have started the application
    // When I enter the correct answer
    // Then I will be told that I have won
  })
})

import { Builder } from 'selenium-webdriver'
require('chromedriver')

let driver

describe('Basic game functions', () => {
  beforeEach(() => {
    driver = new Builder().forBrowser('chrome').build()
  })

  afterEach(() => {
    driver.quit()
  })

  it('should tell me if I won', () => {
    // Given I have started the application
    // When I enter the correct answer
    // Then I will be told that I have won
  })
})

import { Builder } from 'selenium-webdriver'
require('chromedriver')

let driver

describe('Basic game functions', async () => {
  beforeEach(() => {
    driver = new Builder().forBrowser('chrome').build()
  })

  afterEach(() => {
    driver.quit()
  })

  it('should tell me if I won', () => {
    // Given I have started the application
    await driver.get('http://localhost:3000')
    // When I enter the correct answer
    // Then I will be told that I have won
  }, 60000)
})

To enter the correct value into the game, we will need to read the two numbers that appear in the question (as shown in Figure 8-11).

Figure 8-11. The game asks the user to calculate a random product

We can find the two numbers on the page and the input and submit buttons using a command called findElement:

const number1 = await driver.findElement(By.css('.number1')).getText()
const number2 = await driver.findElement(By.css('.number2')).getText()
const input = await driver.findElement(By.css('input'))
const submit = await driver.findElement(
  By.xpath("//button[text()='Submit']")
)

If you are ever reading a set of elements from the page and don’t care about resolving them in a strict order, you can use the Promise.all function to combine them into a single promise that you can then await:

const [number1, number2, input, submit] = await Promise.all([
  driver.findElement(By.css('.number1')).getText(),
  driver.findElement(By.css('.number2')).getText(),
  driver.findElement(By.css('input')),
  driver.findElement(By.xpath("//button[text()='Submit']")),
])

In the example application, this optimization will save virtually no time, but if you have a page that renders different components in uncertain orders, combining the promises can improve test performance.

This means we can now complete the next part of our test:

import { Builder, By } from 'selenium-webdriver'
require('chromedriver')

let driver

describe('Basic game functions', async () => {
  beforeEach(() => {
    driver = new Builder().forBrowser('chrome').build()
  })

  afterEach(() => {
    driver.quit()
  })

  it('should tell me if I won', () => {
    // Given I have started the application
    await driver.get('http://localhost:3000')
    // When I enter the correct answer
    const [number1, number2, input, submit] = await Promise.all([
      driver.findElement(By.css('.number1')).getText(),
      driver.findElement(By.css('.number2')).getText(),
      driver.findElement(By.css('input')),
      driver.findElement(By.xpath("//button[text()='Submit']")),
    ])
    await input.sendKeys('' + number1 * number2)
    await submit.click()
    // Then I will be told that I have won
  }, 60000)
})

Notice that we are not combining the promises returned by sendKeys and click because we care that the test enters the answer into the input field before we submit it.

Finally, we want to assert that a You have won! message appears on the screen (see Figure 8-12).

Figure 8-12. The app tells the user they got the correct answer

Now we could write our assertion like this:

const resultText = await driver
  .findElement(By.css('.Result'))
  .getText()
expect(resultText).toMatch(/won/i)

This code will almost certainly work because the result is displayed quickly after the user submits an answer. React applications will often display dynamic results slowly, particularly if they rely upon data from the network. If we modify the application code to simulate a two-second delay before the result appears,¹³ our test will produce the following error:

no such element: Unable to locate element: {"method":"css selector",
    "selector":".Result"}
  (Session info: chrome=88.0.4324.192)
NoSuchElementError: no such element: Unable to locate element: {
    "method":"css selector","selector":".Result"}
  (Session info: chrome=88.0.4324.192)

We can avoid this problem by waiting until the element appears on the screen and then waiting until the text matches the expected result. We can do both of those things with the driver.wait function:

await driver.wait(until.elementLocated(By.css('.Result')))
const resultElement = driver.findElement(By.css('.Result'))
await driver.wait(until.elementTextMatches(resultElement, /won/i))

This gives us the final version of our test:

import { Builder, By } from 'selenium-webdriver'
require('chromedriver')

let driver

describe('Basic game functions', async () => {
  beforeEach(() => {
    driver = new Builder().forBrowser('chrome').build()
  })

  afterEach(() => {
    driver.quit()
  })

  it('should tell me if I won', () => {
    // Given I have started the application
    await driver.get('http://localhost:3000')
    // When I enter the correct answer
    const [number1, number2, input, submit] = await Promise.all([
      driver.findElement(By.css('.number1')).getText(),
      driver.findElement(By.css('.number2')).getText(),
      driver.findElement(By.css('input')),
      driver.findElement(By.xpath("//button[text()='Submit']")),
    ])
    await input.sendKeys('' + number1 * number2)
    await submit.click()
    // Then I will be told that I have won
    await driver.wait(until.elementLocated(By.css('.Result')))
    const resultElement = driver.findElement(By.css('.Result'))
    await driver.wait(until.elementTextMatches(resultElement, /won/i))
  }, 60000)
})

Discussion

In our experience, web driver tests are the most popular form of automated tests for web applications—popular that is, in the sense of frequently used. They are inevitably dependent upon matching versions of browsers and web drivers, and they do have a reputation for failing intermittently. Timing issues usually cause these failures, and those timing issues occur more in Single-Page Applications, which can update their contents asynchronously.

Although it is possible to avoid these problems by carefully adding timing delays and retries into the code, this can make your tests sensitive to environmental changes, such as running your application on a different testing server. Another option, if you experience a lot of intermittent failures, is to move more of your tests to a system like Cypress, which is generally more tolerant of timing failures.

You can download the source for this recipe from the GitHub site.

Applications can look very different when viewed on different browsers. Applications can even look different if viewed on the same browser but on a different operating system. One example of this would be Chrome, which tends to hide scrollbars when viewed on a Mac but display them on Windows. Thankfully, old browsers like Internet Explorer are finally disappearing, but even modern browsers can apply CSS in subtly different ways, radically changing the appearance of a page.

Storybook

This will give us a basic gallery of all of the components, in all relevant configurations, that we need to check.

Selenium

This will allow us to capture the visual appearance of all of the components in Storybook. The Selenium Grid will also allow us to remotely connect to browsers on different operating systems to make comparisons between operating systems.

ImageMagick

Specifically, we’ll use ImageMagick’s compare tool to generate visual differences between two screenshots and provide a numerical measure of how far apart the two images are.

$ npx sb init

You will then need to create stories for each of the components and configurations you are interested in tracking. You can find out how to do this from other recipes in this book or the Storybook tutorials.

Next, we will need Selenium to automate the capture of screenshots. You can install Selenium with this command:

$ npm install --save-dev selenium-webdriver

You will also need to install the relevant web drivers. For example, to automate Firefox and Chrome, you will need the following:

$ npm install --save-dev geckodriver
$ npm install --save-dev chromedriver

Finally, you will need to install ImageMagick, a set of command-line image manipulation tools. For details on how to install ImageMagick, see the ImageMagick download page.

We are going to use the same example game application that we’ve used previously in this chapter. You can see the components from the application displayed in Storybook in Figure 8-13.

Figure 8-13. Components from the application displayed in Storybook

You can run the Storybook server on your application by typing:

$ npm run storybook

Next, we will create a test that will just be a script for capturing screenshots of each of the components inside Storybook. In a folder called src/selenium, create a script called shots.spec.js:14

import { Builder, By, until } from 'selenium-webdriver'

require('chromedriver')
let fs = require('fs')

describe('shots', () => {
  it('should take screenshots of storybook components', async () => {
    const browserEnv = process.env.SELENIUM_BROWSER || 'chrome'
    const url = process.env.START_URL || 'http://localhost:6006'
    const driver = new Builder().forBrowser('chrome').build()
    driver.manage().window().setRect({
      width: 1200,
      height: 900,
      x: 0,
      y: 0,
    })

    const outputDir = './screenshots/' + browserEnv
    fs.mkdirSync(outputDir, { recursive: true })

    await driver.get(url)

    await driver.wait(
      until.elementLocated(By.className('sidebar-item')),
      60000
    )
    let elements = await driver.findElements(
      By.css('button.sidebar-item')
    )
    for (let e of elements) {
      const expanded = await e.getAttribute('aria-expanded')
      if (expanded !== 'true') {
        await e.click()
      }
    }
    let links = await driver.findElements(By.css('a.sidebar-item'))
    for (let link of links) {
      await link.click()
      const s = await link.getAttribute('id')
      let encodedString = await driver
        .findElement(By.css('#storybook-preview-wrapper'))
        .takeScreenshot()
      await fs.writeFileSync(
        `${outputDir}/${s}.png`,
        encodedString,
        'base64'
      )
    }

    driver.quit()
  }, 60000)
})

This script opens a browser to the Storybook server, opens each of the components, and takes a screenshot of each story, which it stores in a subdirectory within screenshots.

We could use a different testing system to take screenshots of each component, such as Cypress. The advantage of using Selenium is that we can remotely open a browser session on a remote machine.

By default, the shots.spec.js test will take screenshots of Storybook at address http://localhost:6006 using the Chrome browser. Let’s say we are running the shots test on a Mac. If we have a Windows machine on the same network, we can install a Selenium Grid server, which is a proxy server that allows remote machines to start a web driver session.

$ export SELENIUM_REMOTE_URL=http://192.168.1.16:4444/wd/hub

$ export START_URL=http://192.168.1.14:6006

We can also choose which browser we want the Windows machine to use:15

$ export SELENIUM_BROWSER=firefox

If we create a script to run shots.spec.js in package.json:

 "scripts": {
  ...
  "testShots": "CI=true react-scripts test --detectOpenHandles \
                  'selenium/shots.spec.js'"
  }

we can run the test and capture the screenshots of each component:

$ npm run testShots

The test will use the environment variables we created to contact the Selenium Grid server on the remote machine. It will ask Selenium Grid to open a Firefox browser to our local Storybook server. It will then send a screenshot of each of the components over the network, where the test will store them in a folder called screenshots/firefox.

Once we’ve run it for Firefox, we can then run it for Chrome:

$ export SELENIUM_BROWSER=chrome
$ npm run testShots

The test will write the Chrome screenshots to screenshots/chrome.

We now need to check for visual differences between the screenshots from Chrome and the screenshots from Firefox, and this is where ImageMagick is useful. The compare command in ImageMagick can generate an image that highlights the visual differences between two other images. For example, consider the two screenshots from Firefox and Chrome in Figure 8-14.

Figure 8-14. The same component in Chrome and Firefox

These two images appear to be identical. If we type in this command from the application directory:

$ compare -fuzz 15% screenshots/firefox/question--basic.png \
  screenshots/chrome/question--basic.png difference.png

we will generate a new image that shows the differences between the two screenshots, which you can see in Figure 8-15.

Figure 8-15. The generated image showing the differences between two screen captures

The generated image shows pixels that are more than 15% visually different between the two images. And you can see that the screenshots are virtually identical.

That’s good, but it still requires a human being to look at the images and assess whether the differences are significant. What else can we do?

The compare command also has the ability to display a numerical measure of the difference between two images:

$ compare -metric AE -fuzz 15% screenshots/firefox/question--basic.png
  screenshots/chrome/question--basic.png difference.png
6774

The value 6774 is a numerical measure (based on the absolute error count, or AE) of the visual difference between the two images. For another example, consider the two screenshots in Figure 8-16, which show the Answer component when given a disabled property.

Figure 8-16. Disabled form rendered by Chrome and Firefox

Comparing these two images returns a much larger number:

$ compare -metric AE -fuzz 15% screenshots/firefox/answer--with-disabled.png
  screenshots/chrome/answer--with-disabled.png difference3.png
28713

Indeed, the generated image (see Figure 8-17) shows precisely where the difference lies: the disabled input field.

Figure 8-17. The visual difference between the Chrome and Firefox forms

Figure 8-18 shows a similarly significant difference (21,131) for a component that displays different font styling between the browsers, resulting from some Mozilla-specific CSS attributes.

Figure 8-18. A component with different text styling in Chrome and Firefox

In fact, it’s possible to write a shell script to run through each of the images and generate a small web report showing the visual differences alongside their metrics:

#!/bin/bash
mkdir -p screenshots/diff
export HTML=screenshots/compare.html
echo '<body><ul>' > $HTML
for file in screenshots/chrome/*.png
do
    FROM=$file
    TO=$(echo $file | sed 's/chrome/firefox/')
    DIFF=$(echo $file | sed 's/chrome/diff/')
    echo "FROM $FROM TO $TO"
    ls -l $FROM
    ls -l $TO
    METRIC=$(compare -metric AE -fuzz 15% $FROM $TO $DIFF 2>&1)
    echo "<li>$FROM $METRIC<br/><img src=../$DIFF/></li>" >> $HTML
done
echo "</li></body>" >> $HTML

This script creates the screenshots/compare.html report you can see in Figure 8-19.

Figure 8-19. An example of the generated comparison report

Discussion

To save space, we have shown only a simplistic implementation of this technique. It would be possible to create a ranked report that showed visual differences from largest to smallest. Such a report would highlight the most significant visual differences between platforms.

You can also use automated visual tests to prevent regressions. You need to avoid false positives caused by minor variations, such as anti-aliasing. A continuous integration job could set some visual threshold between images and fail if any components vary by more than that threshold.

You can download the source for this recipe from the GitHub site.

This recipe is slightly different from the others in this chapter because instead of being about automated testing, it’s about manual testing—specifically manually testing code on mobile devices.

We are going to use a piece of software called Eruda.

Eruda is a lightweight implementation of a development tools panel, which will allow you to view the JavaScript console, the structure of the page, and a whole heap of other plugins and extensions.

To enable Eruda, you will need to install a small amount of reasonably rudimentary JavaScript in the head section of your application. You can download Eruda from a content distribution network. Still, because it can be pretty large, you should enable it only if the person using the browser has indicated that they want to access it.

One way of doing this is by enabling Eruda only if eruda=true appears in the URL. Here’s an example script that you can insert into your page container:¹⁶

<script>
    (function () {
        var src = '//cdn.jsdelivr.net/npm/eruda';
        if (!/eruda=true/.test(window.location)
            && localStorage.getItem('active-eruda') != 'true') return;
        document.write('<scr' + 'ipt src="' + src
            + '"></scr' + 'ipt>');
        document.write('<scr' + 'ipt>');
        document.write('window.addEventListener(' +
            '"load", ' +
            'function () {' +
            '  var container=document.createElement("div"); ' +
            '  document.body.appendChild(container);' +
            '  eruda.init({' +
            '    container: container,' +
            '    tool: ["console", "elements"]' +
            '  });' +
            '})');
        document.write('</scr' + 'ipt>');
    })();
</script>

If you now open your application as http://ipaddress/?eruda=true or http://ipaddress/#eruda=true, you will notice that an additional button has appeared in the interface, as shown in Figure 8-20.

Figure 8-20. If you add ?eruda=true to the URL, a button will appear on the right of the page

If you are using the example application for this chapter, then try entering a few answers into the game.¹⁷ Then, click the Eruda button. The console will appear as shown in Figure 8-21.

Figure 8-21. Clicking the button opens the Eruda tools

Because the endpoint the example application calls is missing, you should find some errors and other logs recorded in the console. The console even supports the much underused console.table function, which is a helpful way of displaying an array of objects in tabular format.

The Elements tab provides a fairly rudimentary view of the DOM (see Figure 8-22).

Figure 8-22. The Eruda elements view

Meanwhile, the Settings tab has an extensive set of JavaScript features that you can enable and disable while interacting with the web page (see Figure 8-23).

Figure 8-23. The Eruda settings view

Discussion

Eruda is a delightful tool that delivers a whole bucket of functionality, with very little work required by the developer. In addition to the basic features, it also has plugins that allow you to track performance, set the screen refresh rate, generate fake geolocations, and even write and run JavaScript from inside the browser. Once you start to use it, you probably find that it quickly becomes a standard part of your manual testing process.

You can download the source for this recipe from the GitHub site.

In a perfect world, tests would always have a completely artificial environment. Tests are examples of how you would like your application to work under explicitly defined conditions. But tests often have to cope with uncertainties. For example, they might run at different times of day. The example application that we have used throughout this chapter has to deal with randomness.

Our example application is a game that presents the user with a randomly generated question that they must answer (see Figure 8-24).

Figure 8-24. The game asks the user to calculate a random multiplication problem

Randomness might also appear in the generation of identifiers within the code or random data sets. If you ask for a new username, your application might suggest a randomly generated string.

But randomness creates a problem for tests. This is an example test that we implemented earlier in this chapter:

describe('Basic game functions', () => {
  it('should notify the server if I lose', () => {
    // Given I started the application
    // When I enter an incorrect answer
    // Then the server will be told that I have lost
  })
})

There was actually a good reason why that test looked at the case where the user entered an incorrect answer. The question asked is always to calculate the product of two numbers between 1 and 10. It’s therefore easy to think of an incorrect answer: 101. It will always be wrong. But if we want to write a test to show what happens when the user enters the correct answer, we have a problem. The correct answer depends upon data that is randomly generated. We could write some code that finds the two numbers that appear on the screen, as in this example from the first Selenium recipe in this chapter:

const [number1, number2, input, submit] = await Promise.all([
  driver.findElement(By.css('.number1')).getText(),
  driver.findElement(By.css('.number2')).getText(),
  driver.findElement(By.css('input')),
  driver.findElement(By.xpath("//button[text()='Submit']")),
])
await input.sendKeys('' + number1 * number2)
await submit.click()

Sometimes this approach is not even possible. For example, Cypress does not allow you to capture data from the page. If we wanted to write a Cypress test to enter the correct answer to the multiplication problem, we would have great difficulty. That’s because Cypress does not allow you to capture values from the page and pass them to other steps in the test.

It would be much better if we could turn off the randomness during a test.

But can we?

Solution

We will look at how we can use the Sinon library to temporarily replace the Math.random function with a faked one of our own making.

const sinon = require('sinon')

function makeRandomAlways(result) {
  if (Math.random.restore) {
    Math.random.restore()
  }
  sinon.stub(Math, 'random').returns(result)
}

Math.random() * 10 + 1

it('should tell you that you entered the right answer', async () => {
  // Given we've rendered the app
  makeRandomAlways(0.5)
  render(<App />)

  // When we enter the correct answer
  const input = screen.getByLabelText(/guess:/i)
  const submitButton = screen.getByText('Submit')
  user.type(input, '36')
  user.click(submitButton)

  // Then we are told that we've won
  await waitFor(() => screen.findByText(/won/i), { timeout: 4000 })
})

The real value of fixing Math.random is when we use a testing framework that explicitly prevents us from capturing a randomly generated value such as Cypress, as we saw earlier.

Cypress.Commands.add('random', (result) => {
  cy.reload().then((win) => {
    if (win.Math.random.restore) {
      win.Math.random.restore()
    }
    sinon.stub(win.Math, 'random').returns(result)
  })
})

you will create a new command called cy.random. We can use this command to create a test for the winning case that we discussed in the introduction:18

describe('Basic game functions', () => {
  it('should notify the server if I win', () => {
    // Given I started the application
    cy.intercept('POST', '/api/result', {
      statusCode: 200,
      body: '',
    }).as('postResult')
    cy.visit('http://localhost:3000')
    cy.random(0.5)
    cy.contains('Refresh').click()

    // When I enter the correct answer
    cy.get('input').type('36')
    cy.contains('Submit').click()

    // Then the server will be told that I have won
    cy.wait('@postResult').then((xhr) => {
      assert.deepEqual(xhr.request.body, {
        guess: 36,
        answer: 36,
        result: 'WIN',
      })
    })
  })
})

Discussion

You can never remove all randomness from a test. For example, the machine’s performance can significantly affect when and how often your components are re-rendered. But removing uncertainty as much as we can is generally a good thing in a test. The more we can do to remove external dependencies from our tests, the better.

We will also look at removing external dependencies in the following recipe.

You can download the source for this recipe from the GitHub site.

Time can be the source of a tremendous number of bugs. If time were simply a scientific measurement, it would be relatively straightforward. But it isn’t. The representation of time is affected by national boundaries and by local laws. Some countries have their own time zone. Others have multiple time zones. One reassuring factor is that all countries have a time zone offset in whole hours, except for places like India, where time is offset by +05:30 from UTC.

We will look at how you can fix the time when testing your React application. There are some issues that you need to consider when testing time-dependent code. First, you should probably avoid changing the time on your server. In most cases, it’s best to set your server to UTC and leave it that way.

That does mean that if you want to fake date and time in your browser, you will have problems as soon as the browser makes contact with the server. That means you will either have to modify the server APIs to accept an effective date or test time-dependent browser code in isolation from the server.19

We will adopt the latter approach for this recipe: using the Cypress testing system to fake any connections with the server.

We will use the same application we use for other recipes in this chapter. It’s a simple game that asks the user to calculate the product of two numbers. We’re going to test a feature of the game that gives the user 30 seconds to provide an answer. After 30 seconds, they will see a message telling them they’ve run out of time (see Figure 8-25).

Figure 8-25. The player will lose if they don’t answer within 30 seconds

We could try writing a test that somehow pauses for 30 seconds, but that has two problems. First, it will slow your test down. You don’t need many 30-second pauses before your tests will become unbearable to run. Second, adding a pause is not a very precise way to test a feature. If you try to pause for 30 seconds, you might pause for 30.5 seconds before looking for the message.

To get precision, we need to take control of time within the browser. As you saw in the previous recipe, Cypress can inject code into the browser, replacing critical pieces of code with stubbed functions, which we can control. Cypress has a built-in command called cy.clock, which allows us to specify the current time.

describe('Basic game functions', () => {
  it('should say if I timed out', () => {
    // Given I have started a new game
    // When 29 seconds have passed
    // Then I will not see the time-out message
    // When another second has passed
    // Then I will see the time-out message
    // And the game will be over
  })
})

describe('Basic game functions', () => {
  it('should say if I timed out', () => {
    // Given I have started a new game
    cy.visit('http://localhost:3000')
    cy.contains('Refresh').click()

    // When 29 seconds have passed
    // Then I will not see the time-out message
    // When another second has passed
    // Then I will see the time-out message
    // And the game will be over
  })
})

Now we need to simulate 29 seconds of time passing. We can do this with the cy.clock and cy.tick commands. The cy.clock command allows you to either specify a new date and time; or, if you call cy.clock without parameters, it will set the time and date back to 1970. The cy.tick() command allows you to add a set number of milliseconds to the current date and time:

describe('Basic game functions', () => {
  it('should say if I timed out', () => {
    // Given I have started a new game
    cy.clock()
    cy.visit('http://localhost:3000')
    cy.contains('Refresh').click()

    // When 29 seconds have passed
    cy.tick(29000)

    // Then I will not see the time-out message
    // When another second has passed
    // Then I will see the time-out message
    // And the game will be over
  })
})

We can now complete the other steps in the test. For details on the other Cypress commands we’re using, see the Cypress documentation:

describe('Basic game functions', () => {
  it('should say if I timed out', () => {
    // Given I have started a new game
    cy.clock()
    cy.visit('http://localhost:3000')
    cy.contains('Refresh').click()

    // When 29 seconds have passed
    cy.tick(29000)

    // Then I will not see the time-out message
    cy.contains(/out of time/i).should('not.exist')

    // When another second has passed
    cy.tick(1000)

    // Then I will see the time-out message
    cy.contains(/out of time/i).should('be.visible')

    // And the game will be over
    cy.get('input').should('be.disabled')
    cy.contains('Submit').should('be.disabled')
  })
})

If we run the test in Cypress, it passes (as you can see in Figure 8-26).

Figure 8-26. By controlling time, we can force a timeout in the test

That’s a relatively simple time-based test. But what if we wanted to test something much more complex, like daylight saving time (DST)?

When DST occurs depends upon your time zone. And that’s a particularly awful thing to deal with in client code because JavaScript dates don’t work with time zones. They can certainly handle offsets; for example, you can create a Date object in a browser like Chrome that is set to five hours before Greenwich Mean Time:20

new Date('2021-03-14 01:59:30 GMT-0500')

But JavaScript dates are all implicitly in the time zone of the browser. When you create a date with a time zone name in it, the JavaScript engine will simply shift it into the browser’s time zone.

The browser’s time zone is fixed at the time that the browser opens. There’s no way to say Let’s pretend we’re in New York from now on.

If developers create tests for DST, the tests might work only in the developer’s time zone. The tests might fail if run on an integration server set to UTC.

There is, however, a way around this problem. On Linux and Mac computers (but not Windows), you can specify the time zone when you launch a browser by setting an environment variable called TZ. If we start the Cypress with the TZ variable set, any browser that Cypress launches will inherit it, which means that while we can’t set the time zone for a single test, we can set it for an entire test run.

First, let’s launch Cypress with the time zone set to New York:

$ TZ='America/New_York' npx cypress open

The example application has a button that allows you to see the current time (see Figure 8-27).

Figure 8-27. The current time is shown on the screen

We can create a test that checks that the time on the page correctly handles the change to DST. This is the test we’ll create:

describe('Timing', () => {
  it('should tell us the current time', () => {
    cy.clock(new Date('2021-03-14 01:59:30').getTime())
    cy.visit('http://localhost:3000')
    cy.contains('Show time').click()
    cy.contains('2021-03-14T01:59:30.000').should('be.visible')
    cy.tick(30000)
    cy.contains('2021-03-14T03:00:00.000').should('be.visible')
  })
})

In this test, we are passing an explicit date to cy.clock. We need to convert this to milliseconds by calling getTime as cy.clock accepts only numeric times. We then check the initial time, and 30 seconds later, we check the time rolls over to 3 a.m., instead of 2 a.m. (as shown in Figure 8-28).

Figure 8-28. After 30 seconds, the time correctly changes from 01:59 to 03:00

Discussion

If you need to create tests that depend on the current time zone, consider placing them into a subfolder so you can run them separately. If you want to format dates into various time zones, you can use the toLocaleString date method:

new Date().toLocaleString('en-US', { timeZone: 'Asia/Tokyo' })

You can download the source for this recipe from the GitHub site.