⚙️ Getting Started with AIO Tests Integration

AIO Tests is a Jira-integrated test management tool that helps QA teams manage manual and automated tests with real-time tracking and detailed reporting. It streamlines the testing process while ensuring full traceability from requirements to defects. This article aims to show how to integrate GitHub Actions with AIO Tests to trigger Playwright test workflows directly from the AIO Tests interface. We'll use the native AIO Tests Playwright reporter for automatic test case updates. AIO Tests also supports other reporting options such as manual uploads and API-based integration.

📃 Prerequisites

Before you begin, make sure you have the following:

✅ Set of pre-written Playwright test scripts in your project and create a GitHub repository to manage them

✅ An active Jira account with access to your project

✅ The AIO Tests app installed from the Atlassian Marketplace

🤖 Playwright Framework Overview

This project uses Playwright, a powerful open-source tool by Microsoft for end-to-end testing of web applications across modern browsers like Chrome, Firefox, and Safari. Playwright supports fast, reliable testing with features like auto-waiting, network control, and multilanguage support.

The framework is structured using the Page Object Model (POM) for maintainability and scalability. Key folders include (GitHub Repo):

Folder Structure

Step 1 - Create the test cases in AIO Tests that you intend to automate and copy test case keys we will need them on step 3.

Step 2 - Open your IDE and install this npm package. It adds the AIO Tests Playwright Reporter, which links your Playwright test results to AIO Tests automatically.

npm install aiotests-playwright-reporter

Step 3 - In Step 1, you copied the AIO Tests case key. Now, include it in your test title using the '@' symbol. This acts as a Playwright tag and helps AIO match the test with the correct test case.

test("@PAT-TC-3: Verify that user can add items to the cart", async ({
  page,
  purchasingPage,
  locators,
}) => {
  const PurchasingLocators = locators.Purchasinglocators;
  await purchasingPage.AddItemsToCart();
  await expect(page.locator(PurchasingLocators.Cart)).toContainText("2");
});

Step 4 - Add the following configuration outside the default function in your Playwright config file. Each key helps define how your test results are reported to AIO Tests.

In the next steps, we'll store AIO_API_KEY and JIRA_PAT as GitHub Actions secrets. That's why we reference them as environment variables, so they can be securely injected into the workflow at runtime.

const aioConfigDetails = {
  enableReporting: true, // ✅ Set to true to report test results to AIO Tests (default is false)
  jiraProjectId: "PAT", // ✅ Jira project key where the AIO test cases are managed
  cloud: {
    apiKey: process.env.AIO_API_KEY, // ✅ API key for AIO Cloud, stored securely as an environment variable
  },
  server: {
    jiraServerUrl: "https://yourorgname.atlassian.net/", // ✅ Jira base URL (used only for Jira Server setup)
    pat: process.env.JIRA_PAT, // ✅ Personal Access Token for Jira Server, pulled from environment variable
  },
  cycleDetails: {
    createNewCycle: "false", // ✅ Options: [true, false, "CREATE_IF_ABSENT"]
    // Set to true to always create a new test cycle for this run

    cycleName: "PAT-Test-Cycle-1", // ✅ Used when createNewCycle is true
    // Sets the name for the new test cycle being created

    cycleKey: "PAT-CY-2", // ✅ Used when createNewCycle is false
    // Links the run to an existing cycle by its key
  },

  addAttachmentToFailedCases: true, // ✅ Adds only screenshots to failed test cases if available
};

Configuration Details

Jira Project ID

The first three letters of an AIO test case key usually represent your project ID, but if you're unsure, you can find the project ID on the Project Settings page in Jira.

To get the AIO API Key

Go to AIO Tests in your project → Click the gear icon on the navigation bar → Select My Settings from the dropdown options → Choose the API Token option from the side navigation → Generate a new API token and keep the token safe until you add it as a secret to your repository.

Jira Server URL

You can find your Jira Server URL directly from the browser's address bar when you are logged into your Jira instance.

Jira Personal Access Token

Click the gear icon next to your Jira profile image in Jira → Select "User Management" option at the bottom of the list → Switch to "Settings" tab → Select "API Key" option from side navigation → Create an API token and keep the token safe until you add it as a secret to your repository.

Refer to the documentation to see the full list of supported parameters you can pass.

Step 5 - Add the reporter with the AIO config object as shown below; this configures the AIO Tests reporter with the required aioConfig settings.

reporter: [
  ["aiotests-playwright-reporter", { aioConfig: aioConfigDetails }], // List reporter for console output
],

Step 6 - In your GitHub workflow file, add the following under env:

env:
  AIO_API_KEY: ${{ secrets.AIO_API_KEY }}
  JIRA_PAT: ${{ secrets.JIRA_PAT }}

This makes your AIO API key and Jira PAT token available as environment variables during the workflow run. It allows your Playwright config to securely access them from GitHub Secrets via environment variables (process.env), without exposing the actual values in your code or logs.

Step 7 - Add AIO API key and Jira PAT as repository secrets.

Go to GitHub repo → Switch to "Settings" → Expand "Secrets and Variables" dropdown → Select "Actions" options → Click "New Repository Secrets" button and add AIO API key and Jira PAT.

Note - When adding secrets, make sure the naming conventions are consistent and match exactly where they are referenced in your code or workflows.

GitHub Secrets Configuration

Before setting up AIO Tests to trigger workflows directly from the AIO Tests interface, run the workflow manually to verify whether the test cases are being updated correctly after the workflow execution.

As you can see, we've configured aioConfig to create a new cycle and update the test case statuses, so after the workflow run, a new cycle was created and the test cases were successfully updated.

🪄 Triggering GitHub Actions Workflows via the AIO Tests Interface

Step 8

In this step, the CI/CD system is configured by selecting a hosting platform such as GitHub:

  • Click the "CI" button in the tool ribbon.
  • Click the "Setup CI/CD" system button
  • Select "GitHub" as the system type and choose the version (personal accounts are always under "GitHub Cloud").
  • Enter a name for your CI/CD system.
  • Enter your GitHub username and click the "Save" button.

Step 9

Now that the CI/CD workflow is configured, the next step is to define a job within the workflow to execute the actions:

  • Click the "View Job" icon on the created CI/CD system.
  • Click the "Configure New Job" button.
  • If this is your first time, you'll be prompted to enter a GitHub API token. To do this, generate one from your GitHub account, paste it, and click "Save".
  • Provide a name for the job, add a description, and enter the repository name.
  • Click the "Search" button. If the repository name is correct, the list of branches and workflows will be displayed.
  • Add any job parameters, if required.
  • Under Authentication, choose one of the three available options based on your preference.
  • Once everything is configured, click the "Save" button.
  • To trigger the job, select the job you want to run, click the "Play" icon, and then click the "Trigger" button.

Now that everything is set up, a test cycle was created from the AIO Tests interface, and the aioConfig was updated accordingly:

cycleDetails: {
  createNewCycle: "false", // ✅ Options: [true, false, "CREATE_IF_ABSENT"]
  // Set to true to always create a new test cycle for this run

  cycleName: "PAT-Test-Cycle-1", // ✅ Used when createNewCycle is true
  // Sets the name for the new test cycle being created

  cycleKey: "PAT-CY-2", // ✅ Used when createNewCycle is false
  // Links the run to an existing cycle by its key
},

An assertion was intentionally altered to fail a test case, to verify that failed test cases and their attachments are updated correctly.

🔚 Conclusion

This article explains how to integrate GitHub Actions with AIO Tests to trigger Playwright test workflows directly from the AIO Tests interface. With this setup, you can effortlessly manage and run your Playwright tests through AIO Tests, ensuring efficient execution and accurate reporting.