If you're knee-deep in the world of Playwright, you know it's one of the most powerful tools for end-to-end testing in modern web development. But what happens when a core feature gets a shake-up? Enter the drama of Playwright version 1.34, where the built-in JUnit reporter bid farewell to its Xray enhancements. Fear not; rising from the ashes is the @xray-app/playwright-junit-reporter, a community-driven powerhouse that's not just a replacement but an upgrade. Let's dive into why this is a game-changer for your CI/CD pipelines and test management workflows.
Picture this: You're running automated tests with Playwright, generating clean JUnit XML reports, and seamlessly importing them into Xray for Jira to track defects, coverage, and requirements. It was smooth sailing until Playwright v1.34 streamlined its core reporters, dropping the Xray-specific goodies. But innovation doesn't stop! The developers at Xray-App have forked and enhanced the original, preserving all the beloved features while making it even more flexible. This reporter isn't just a band-aid; it's a full-fledged evolution. It produces JUnit-style XML that's fully compatible with Xray, allowing you to embed rich metadata like test keys, requirements, and even attachments. Whether you're a solo dev or part of a massive QA team, this tool bridges the gap between raw test results and actionable insights in your test management system.
Playwright is a powerful, open-source automation library designed for end-to-end (E2E) testing, web scraping, and browser automation. Launched by Microsoft, it allows you to control Chromium (Chrome, Edge), Firefox, and WebKit (Safari) browsers with a single, unified API. Whether you’re running tests on Windows, macOS, or Linux, Playwright delivers reliability, speed, and flexibility that make it stand out in the crowded world of automation tools.
Think of Playwright as a master puppeteer, orchestrating browsers with precision to simulate real user interactions: clicking buttons, filling forms, or navigating pages, while ensuring everything runs smoothly, even with dynamic, modern web apps. Playwright isn’t just another testing tool; it’s a Swiss Army knife for browser automation. Here’s what makes it sparkle:
Jumping in is a breeze. If you're using npm or yarn, here's how to snag it:
Once installed, configure it in your Playwright setup. Want to output to a file? Use the environment variable for quick runs:
PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml npx playwright test --reporter=@xray-app/playwright-junit-reporter
For more control, tweak your playwright.config.ts (or .js) file:
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [['@xray-app/playwright-junit-reporter', { outputFile: 'results.xml' }]],
});
Pro tip: If you're all-in on Xray, customize with options like embedAnnotationsAsProperties to inject metadata directly into your XML. This turns your reports into treasure troves of info, mapping annotations to JUnit properties for effortless import.
Supercharged Features: Beyond Basic Reporting
What sets this reporter apart? It's the thoughtful enhancements tailored for real-world testing scenarios. Here's the juicy stuff:
1. Annotation Magic for Metadata Mastery
Forget bland reports and embed custom properties to link tests with Xray entities. Use Playwright's testInfo.annotations to add details like test keys, summaries, and requirements:
import { test } from '@playwright/test';
test('using specific annotations for passing test metadata to Xray', async ({}, testInfo) => {
// Xray will process only properties from the Junit XML report that it is aware of; other properties are discarded
testInfo.annotations.push({ type: 'test_id', description: '1234' });
testInfo.annotations.push({ type: 'test_key', description: 'CALC-2' });
testInfo.annotations.push({ type: 'test_summary', description: 'sample summary' });
testInfo.annotations.push({ type: 'requirements', description: 'CALC-5,CALC-6' });
testInfo.annotations.push({ type: 'test_description', description: 'sample description' });
// add some information to custom fields on the Test Run; these custom fields need to be created
// before in Xray settings, eventually on the project settings
// setting some text on a TR custom field of type "text - single line"
testInfo.annotations.push({ type: 'tr:basic_cf', description: 'dummycontent' });
// setting some text on a TR custom field of type "multiselect", with checked options delimited using ;
testInfo.annotations.push({ type: 'tr:multiselect_cf', description: 'a;b;c' });
// setting some text on a TR custom field of type "text"
const multilineString = "Hello world\nThis is a multiline string";
testInfo.annotations.push({ type: 'tr:multiline_cf', description: multilineString });
});
This keeps your reports self-contained, perfect for auditing or debugging without hunting down files.
2. Attachment Embedding: Evidence at Your Fingertips
Screenshots, logs, or custom files? Attach them directly and embed as base64 in the XML.
Set embedAttachmentsAsProperty: 'testrun_evidence' in your config, then:
import { test } from '@playwright/test';
test('embed attachments, including its content, on the JUnit report', async ({}, testInfo) => {
const file = testInfo.outputPath('evidence1.txt');
require('fs').writeFileSync (file, 'hello', 'utf8');
await testInfo.attach('evidence1.txt', { path: file, contentType: 'text/plain' });
await testInfo.attach('evidence2.txt', { body: Buffer.from('world'), contentType: 'text/plain' });
});
3. Flexible Filtering and Customization
These aren't just bells and whistles. They streamline integration with tools beyond Xray, as long as they grok (pun intended) the extended JUnit format.
Imagine slashing hours off your post-test analysis. With this reporter, your Playwright runs feed directly into Xray, auto-linking to Jira issues, tracking coverage against requirements, and even populating custom fields. No more manual uploads or data massaging. Teams using CI tools like GitHub Actions or Jenkins will love the npm downloads (already in the thousands!) and the active Gitter community for support.
Plus, it's open-source under Apache 2.0, based on Playwright's own code. Got ideas? Fork it, contribute, or hit up the maintainers on Twitter.
The to-do list is exciting: Code coverage integration and auto-upload to Xray via their JS automation package are on the horizon. This isn't a static tool. It's evolving with the community.
If you're tired of fragmented test reporting, give @xray-app/playwright-junit-reporter a spin. It's more than an update; it's a lifeline for Xray users in the Playwright ecosystem. Head over to the GitHub repo to star it, try it out, and join the conversation..
Building top-notch software doesn’t have to be a struggle. At NUCIDA, we’ve cracked the code with our B/R/AI/N Testwork testing solution - pairing our QA expertise with your test management tool to deliver streamlined processes, slick automation, and results you can count on. On time. Hassle-free. Ready to ditch future headaches? Let NUCIDA show you how!
Among others, NUCIDA's QA experts are certified consultants for Testiny, SmartBear, TestRail, and Xray software testing tools.
Why Choose NUCIDA?
For us, digitization does not just mean modernizing what already exists but, most importantly, reshaping the future. That is why we have made it our goal to provide our customers with sustainable support in digitizing the entire value chain. Our work has only one goal: your success!
Don’t let testing slow you down. Explore how consulting services can make your software quality soar - headache-free! Got questions? We’ve got answers. Let’s build something amazing together!
With these updates, Playwright is more powerful, flexible, and user-friendly than ever. The improved tool integration with Xray offers you more detailed test result information in Xray directly. Head to our Xray page to get started and see how this Jira-based test management app can elevate your test management and test automation within software testing.
The new Xray Cloud version is packed with features designed to make test management more efficient, insightful, and user-friendly. From project-level status configuration to the powerful Usage Monitor and enhanced visibility of archived test runs, this release empowers teams to optimize their testing processes and drive better outcomes.
Have questions? The NUCIDA QA Team is here to help! Until then, let’s continue to push the boundaries of testing together. Stay tuned, and happy testing!.
Want to know more? Watch our YouTube video, Latest Xray Features Q2 2025, to learn more about the latest developments.
Logo and pictures from pixabay.com, Xray, Playwright, and NUCIDA Group
Article written and published by Torsten Zimmermann