In the fast-paced world of software development, where CI/CD pipelines hum like well-oiled machines and code changes are implemented at a rapid pace, one thing remains a constant thorn in the side: fragmented test results. You've got your Cypress end-to-end tests spitting out JSON reports, your JUnit unit tests generating XML files, and maybe even some Playwright scripts adding their own flavor to the mix, but piecing it all together? That's where the real headache begins concerning scattered data, elusive flaky tests, and hours lost in debugging purgatory.
Enter Testiny: the test management tool that's not just a repository for your manual tests but a powerhouse for centralizing automated results. Imagine having all your test outcomes in one place, tracked over time, with insights that help you spot failing patterns and crush CI failures before they crush your release schedule. Sounds too good to be true? It's not. This memoir will explore how to integrate popular test automation tools with Testiny, utilizing its intuitive CLI and custom reporters. Whether you're a QA engineer, DevOps wizard, or just someone tired of test chaos, by the end, you'll be armed with the knowledge to streamline your workflow like never before.
Before we get our hands dirty with code, let's talk about the why. Testiny isn't just another tool. It's your command center for test intelligence. By feeding your automated test results directly into Testiny, you can:
And the best part? Testiny supports a wide array of frameworks out of the box, from Cypress to Robot Framework. If your tool outputs standardized formats like JUnit XML or Mocha JSON, you're golden. For others, a quick custom setup does the trick. Let's break it down step by step.
At the heart of Testiny's automation magic is its CLI tool. It's lightweight, easy to install via npm (npx @testiny/cli), and handles everything from report imports to creating automated test runs.
First things first:
The --project flag targets your Testiny project by ID or key, --source labels your results (think "e2e" or "backend"), and --format specifies the report type (e.g., --cypress for Cypress reports).
The complete Testiny API documentation is available at https://www.testiny.io/docs/api-quickstart/
Pro Tip: If your framework isn't explicitly listed, fall back to JUnit-style XML—Testiny's CLI loves it. Now, let's explore integrations for popular tools.
Cypress is a crowd favorite for UI testing, and Testiny makes importing its results a breeze. Start by configuring Cypress to output Mochawesome JSON reports:
const { defineConfig } = require("cypress");
module.exports = defineConfig({
reporter: 'mochawesome',
reporterOptions: {
overwrite: false, // Generate per-spec reports
reportDir: 'cypress/reports',
html: false,
json: true
},
});
Run your tests:
npm install
npx cypress run --browser chrome
npx cypress run --browser firefox
Then import:
npx @testiny/cli automation --project YOUR_PROJECT --source "e2e" --cypress cypress/reports/*.json
Testiny automatically merges reports, imports screenshots from cypress/screenshots (or customize with --attachment-path), and handles "pending" tests as "blocked." Add custom fields like browser type with --field-values browser=firefox for multi-config runs. Boom! Your Cypress insights are now live in Testiny!
Playwright's versatility shines in cross-browser testing, and Testiny offers multiple paths: a custom reporter for direct pushes or CLI imports from JSON/JUnit reports.
For the custom reporter:
export default defineConfig({
reporter: [["@testiny/automation/reporters/playwright", {
enable: true,
project: "key",
sourceName: "ui-test",
token: "tny_12..." // Or use env var
}]],
});
Run playwright test, and results stream straight to Testiny.
For JSON reports:
Configure reporter as ["json", { outputFile: "results/results.json" }], then:
npx @testiny/cli automation --project YOUR_PROJECT --source "e2e" --playwright results/*.json
Bonus: Use Playwright annotations for custom fields, imported via --custom-result-fields=type,some_link.
JUnit (and its XML cousins) powers tools like Gradle, Maven, and more. Most generate reports automatically. Just point Testiny's CLI at them:
npx @testiny/cli automation --project YOUR_PROJECT --source "frontend-tests" --junit results/*.xml
Customize titles and folders: --junit-ignore-classname to drop classnames from titles, or --junit-classname-as-folder nested to create hierarchical folders. Handle time units with --junit-duration-unit ms. It's perfect for frameworks like PHPUnit or pytest, too. There is the same CLI flag!
For MochaJS, generate Mochawesome reports: npx mocha --reporter mochawesome. Import:
npx @testiny/cli automation --project YOUR_PROJECT --source "frontend" --mochawesome mochawesome-report/*.json
Generate XML: pytest --junit=results/report.xml. Import:
npx @testiny/cli automation --project YOUR_PROJECT --source "pytest-tests" --junit results/report.xml --custom-result-fields=type,another_property
Log extras with record_property and pull them in as custom fields.
For unlisted tools, standardize to JUnit and use the CLI's quickstart guide..
Overcoming Hurdles: Tips from the Trenches
Encounter issues? Testiny's forum and support team are ready to help. Integrate into CI/CD pipelines (check GitHub examples in the docs) for automated bliss. Remember, attachments like screenshots auto-import where supported, and custom fields add that extra layer of detail.
Integrating your automation tools with Testiny isn't just a setup. It's a transformation. You'll spend less time wrangling reports and more time building features that wow users. Teams report faster debugging, fewer flaky tests, and a clearer path to quality releases. Ready to dive in? Grab your API key, fire up the CLI, and experience the difference. Visit Testiny's documentation for more resources, including full guides and CI/CD examples. Your tests deserve a home. Make it Testiny.
Building top-notch software doesn’t have to be a struggle. At NUCIDA, we’ve cracked the code with our B/R/AI/N Testwork testing solution - pairing our QA expertise with your test management tool to deliver streamlined processes, slick automation, and results you can count on. On time. Hassle-free. Ready to ditch future headaches? Let NUCIDA show you how!
Among others, NUCIDA's QA experts are certified consultants for Testiny, SmartBear, TestRail, and Xray software testing tools.
Why Choose NUCIDA?
For us, digitization does not just mean modernizing what already exists but, most importantly, reshaping the future. That is why we have made it our goal to provide our customers with sustainable support in digitizing the entire value chain. Our work has only one goal: your success!
Don’t let testing slow you down. Explore how consulting services can make your software quality soar - headache-free! Got questions? We’ve got answers. Let’s build something amazing together!
Testiny empowers you to unify your automated testing efforts, turning chaos into clarity. With its robust CLI and flexible integrations, you can effortlessly connect frameworks like Cypress, Playwright, and JUnit to a centralized hub that delivers actionable insights. The GitHub Actions workflows we've explored demonstrate how to automate this process in your CI/CD pipelines, saving time and boosting reliability. Whether you're taming flaky tests or streamlining multi-browser reporting, NUCIDA is your quality partner, helping you to achieve your software quality goals. So, grab your API key, dive into the Cypress and Playwright examples, and start transforming your testing workflow today. Your next great release is waiting. Make it flawless with NUCIDA and Testiny!
Have questions? The NUCIDA QA Team is here to help! Until then, let’s continue to push the boundaries of testing together. Stay tuned, and happy testing!.
Want to know more? Watch our YouTube video, Testiny Test Management, to learn more about the latest developments.
Logo and pictures from pixabay.com, Testiny, and NUCIDA Group
Article written and published by Torsten Zimmermann