Organizing Playwright Tests Effectively

When working with end-to-end (E2E) testing in Playwright, maintaining a clean and scalable test suite is crucial. A well-organized structure not only improves maintainability but also makes it easier to onboard new team members. In this post, we'll cover how to best organize your Playwright tests, from folder structures to using hooks, annotations and tags.

Structuring Your Test Folders

Playwright tests are typically organized within a tests folder. You can create multiple levels of sub folders within the tests folder to better organize your tests. When dealing with tests that require user authentication, separating tests into logged-in and logged-out states makes the test suite cleaner. Here's an example folder structure:

/tests
  /helpers
    - list-test.ts        # custom fixture for a page with list of movies
    - list-utilities.ts       # helper functions for creating lists of movies
  /logged-in
    - api.spec.ts    # API tests for logged-in users
    - login.setup.ts      # Tests for logging in
    - manage-lists.spec.ts  # Tests for managing lists of movies
  /logged-out
    - api.spec.ts # Tests API endpoints for logged-out users
    - auth.spec.ts     # Tests login flow
    - movie-search.spec.ts  # Tests for searching movies
    - sort-by.spec.ts       # Tests for sorting movies

Using Test Hooks

Playwright offers test hooks such as beforeEach and afterEach to handle common setup and teardown tasks for each test. These hooks are particularly useful for actions like logging in, initializing test data, or navigating to a specific page before each test.

test.beforeEach(async ({ page }) => {
  await page.goto('');
});

test('should edit an existing list', async ({ page }) => {
  // ...
});

test('should add and delete movies from a list', async ({ page }) => {
  //...
});

When you have common setup and teardown tasks across multiple tests, helper functions can help you avoid code duplication and keep your tests DRY (Don't Repeat Yourself). Below is an example of helper functions used to create a list of movies:

import { test, expect, Page } from '@playwright/test';

export async function createList(
  page: Page,
  listName: string,
  listDescription: string,
) {
  await test.step('create a new list', async () => {
    await page.getByLabel('User Profile').click();
    await page.getByRole('link', { name: 'Create New List' }).click();
    await page.getByLabel('Name').fill(listName);
    await page.getByLabel('Description').fill(listDescription);
    await page.getByRole('button', { name: 'Continue' }).click();
  });
}

export async function openLists(page: Page, name: string = 'My Lists') {
  //...
}

The helper functions can be used in the beforeEach hook in various test files ensuring each test starts with a list of movies and opens on the lists page:

import { test, expect } from '@playwright/test';
import { addMovie, createList, openLists } from '../helpers/list-utilities';

// Before each test, navigate to the base URL, create a list, and open the lists page
test.beforeEach(async ({ page }) => {
  await page.goto('');
  await createList(
    page,
    'my favorite movies',
    'here is a list of my favorite movies',
  );
  await openLists(page);
});

test('should edit an existing list', async ({ page }) => {
  await page.getByRole('link', { name: 'my favorite movies' }).click();
  await page.getByRole('link', { name: 'Edit' }).click();
  // ...
});

test('should add and delete movies from a list', async ({ page }) => {
  await page.getByRole('link', { name: 'my favorite movies' }).click();
  await page.getByRole('button', { name: 'Add/Remove Movies' }).click();
  //...
});

Fixtures

Fixtures can be used instead of a beforeEach hook and are a great way of creating a page context that can be shared across multiple tests. Fixtures can be defined in a separate file and imported into test files where they are needed. Here's an example of a fixture for a movie list page:

export const listTest = baseTest.extend<{ listPage: Page }>({
  listPage: async ({ context }, use) => {
    // fixture setup
    const page = await context.newPage();
    await page.goto('');
    await createList(page, 'my favorite movies', 'list of my favorite movies');

    await listTest.step('add movies to list', async () => {
      await addMovie(page, 'Twisters');
      await addMovie(page, 'The Garfield Movie');
      await addMovie(page, 'Bad Boys: Ride or Die');
    });
  //...
});

The fixture can be used in test files by importing it and passing it as an argument to the test function instead of passing in Playwright's built in page fixture. Each test that uses the custom fixture will start with a page that has a list of movies:

import { expect } from '@playwright/test';
import { listTest as test } from '../helpers/list-test';
import { addMovie } from '../helpers/list-utilities';

test('editing an existing list', async ({ listPage }) => {
  // set the page to the listPage fixture
  const page = listPage;

  await page.getByRole('link', { name: 'Edit' }).click();
  // ...
});

Splitting Tests into Steps with test.step

When you want to add more clarity to your tests, Playwright's test.step function is handy. It breaks down complex tests into more digestible steps, improving readability and reporting.

import { test, expect } from '@playwright/test';

test('should add and delete movies from a list', async ({ page }) => {
  const movieList = page.getByRole('listitem', { name: 'movie' });
  await page.getByRole('link', { name: 'my favorite movies' }).click();
  await page.getByRole('button', { name: 'Add/Remove Movies' }).click();

  await test.step('add and verify movies in the list', async () => {
    await addMovie(page, 'Twisters');
    await addMovie(page, 'Bad Boys: Ride or Die');
    await expect
      .soft(movieList)
      .toHaveText([/Twisters/, /Bad Boys: Ride or Die/]);
  });

  await test.step('remove and verify movies in the list', async () => {
    const movie1 = page.getByRole('listitem').filter({ hasText: 'Twisters' });
    await movie1.getByLabel('Remove').click();
    await expect.soft(movieList).toHaveText([/Bad Boys: Ride or Die/]);
  });
});

Using Built-in Annotations: skip, fail, and fixme

Annotations in Playwright help you mark tests for specific behaviors or conditions, such as skipping them under certain conditions, marking them as expected failures, or flagging them as needing attention. Let's dive into each one with examples:

1. test.skip: Skipping Tests Conditionally

The test.skip annotation is useful when you want to skip a test based on certain conditions or configuration. For example, you might want to skip tests on specific browsers, in particular environments, or when a feature is not available.

import { test, expect } from '@playwright/test';

test('hamburger menu in mobile', async ({ page, isMobile }) => {
  // Skip the test if viewport is mobile
  test.skip(!isMobile, 'Test is only relevant for desktop');

  await page.goto('/');
  //..
});

test('should not run on Safari', async ({ page, browserName }) => {
  // Skip the test on Safari due to known compatibility issues
  test.skip(browserName === 'webkit', 'Safari not supported for this feature');

  await page.goto('');
  //..
});

2. test.fixme: Marking Tests to Be Fixed

Use test.fixme to mark a test that you intend to fix later. This annotation is typically applied to tests that are not yet implemented correctly or tests that are incomplete. Playwright will automatically skip these tests and flag them in reports, serving as a reminder.

import { test, expect } from '@playwright/test';

// Mark this test as a "fixme" since it's not fully implemented yet
test.fixme('log user in and verify profile access', async ({ page }) => {
  await page.goto('');
  await page.getByRole('banner').getByLabel('Log In').click();
  await page
    .getByPlaceholder('you@example.com')
    .fill(process.env.MOVIES_USERNAME!);
  await page.getByPlaceholder('Password').fill(process.env.MOVIES_PASSWORD!);
  await page.getByRole('button', { name: 'login' }).click();
  await page.getByLabel('User Profile').click();
});

How These Annotations Help

• test.skip is excellent for conditionally running tests based on criteria like environment, platform, or feature availability. This helps maintain a green test suite by skipping irrelevant tests instead of letting them fail.
• test.fixme is useful when you have incomplete or not-yet-implemented features. These tests are automatically skipped and reported, keeping them in mind as future to-dos.

Summary of Annotations Usage

Using these built-in annotations can streamline your workflow, reduce false negatives in test results, and communicate known issues and unimplemented features clearly to your team.

Adding custom Annotations

In Playwright, you can add custom annotations to your tests. For example, a common practice is including a link to a related issue. These can be valuable in reports and are also visible in UI mode. Here's an example of how you can add issue links to your tests:

test(
  'should delete a list',
  {
    annotation: {
      type: 'issue',
      description: 'https://github.com/microsoft/demo.playwright.dev/issues/58',
    },
  },
  async ({ page }) => {
    await page.getByRole('link', { name: 'my favorite movies' }).click();
    await page.getByRole('link', { name: 'Edit' }).click();
    await page.getByRole('link', { name: 'Delete List' }).click();
  // ... remaining test steps
});

Using Tags to Filter and Organize Tests

Tags are a powerful way to categorize and filter tests. You can use tags to organize tests based on features, priorities, user roles, or release cycles. This allows you to easily filter and run specific sets of tests or generate targeted reports.

Adding Tags

In Playwright, you can add tags as part of the test definition's metadata. Here's an example of how to define tags for tests:

test('sort by with api mocking', { tag: '@mocking' }, async ({ page }) => {
  // Mock the API call for sorting by popularity
  await page.route('*/**/**sort_by=popularity.desc', async (route) => {
    await route.fulfill({
      path: path.join(__dirname, '../mocks/sort-by-popularity.json'),
    });
  });
});

In this example, we've tagged the test with a "@mocking" tag. This metadata will be used to filter tests in reports or command-line execution.

Running Tests by Tags

To run tests with specific tags, you can use the --grep flag followed by a tag name:

npx playwright test --grep @mocking

To exclude tests with a specific tag, use the --grep-invert flag:

npx playwright test --grep-invert @mocking

Filtering by Tags in HTML Reports

Tags are displayed in the HTML report, making it easy to identify and filter tests based on tags while reviewing results. This can be very helpful for debugging or focusing on a subset of tests related to a specific feature.

Summary

By following these guidelines, you can create a well-structured and maintainable Playwright test suite:

• Organized Folder Structure: Separate tests based on their context (logged-in vs. logged-out) and per feature. For example, testing GitHub we could have tests/repos, tests/prs, tests/issues, etc.
• Using Hooks and Describe blocks: Improve readability and set up common prerequisites.
• Step Definition: Use test.step to break down complex test cases.
• Leveraging Annotations and Tags: Use annotations and tags to mark failing or incomplete tests, link issues, and categorize your tests for better filtering and reporting.

With a thoughtful approach to organizing your tests, you'll be able to create a cleaner and more maintainable test suite that scales well with your application.

Happy testing!