Operating automated checks is an efficient factor for software program high quality assurance. Now it’s a must to perceive the outcomes of the present take a look at run. You may be to see how your checks outcomes evolve over time. To realize these targets, you want a software to provide significant checks experiences and there are some open supply instruments that can assist you do that.
This text lists the principle open supply take a look at reporting instruments accessible at present. Use our contact type to make us conscious of some open supply take a look at reporting instruments that we should always add to this listing.
Listed here are some options that you may discover in open supply take a look at reporting instruments:
- Managing information of 1 or many languages and take a look at automation frameworks
- Creation of take a look at experiences in numerous codecs: HTML, PDF, …
- Report distribution through e-mail
- Textual and graphical illustration of outcomes of a single take a look at run
- Timelines graphs for presenting the evolution of outcomes for various take a look at runs
- Integration within the steady integration / steady supply (CI/CD) course of
Generic instruments
This part lists instruments that works with a number of programming languages and take a look at automation frameworks.
* Updates
December 7 2023: added Go-allure, HtmlTestRunner, Subsequent Gen Report (nxgreport),
Attract Framework
The Attract Framework is a versatile and light-weight open supply multi-language take a look at reporting software. It supplies clear graphical experiences and permits everybody concerned within the improvement course of to extract a most of knowledge from the on a regular basis software program testing course of. The Attract framework assist many languages (Java, JavaScript, .NET, Python, Ruby, and many others.) and totally different take a look at automation instruments (Cucumber, Junit, TestNG, SpecFlow, Jasmine, and many others.). The reporting course of might be triggered by a steady integration software like Jenkins or Gradle. One of many particular options of Attract is to have the ability to hyperlink to person tales or epic outlined in Habits-Pushed Growth (BDD) instruments.
Web site: https://github.com/allure-framework
ExtentReports / Klov
ExtentReports is an open supply library that enables creating interactive and detailed experiences in your checks. You possibly can add occasions, screenshots, tags, units, authors or every other related info you determine is vital to create descriptive experiences which you could completely management. This framework works with Java and C# languages and the Cucumber, SpecFlow and TestNG testing instruments. ExtentReports is related to Klov, a software that enables monitoring every of your launches throughout all of your tasks. You possibly can view and analyze how your take a look at carry out extra time, how every execution class has compares intimately, observe exceptions, create subjects for dialogue or reminders and search entities from a number of choices. This open supply framework has additionally a industrial model.
Web site: https://github.com/extent-framework, https://www.extentreports.com/
ReportPortal
ReportPortal is an open supply reporting service, that gives elevated capabilities to hurry up checks outcomes evaluation and reporting via the usage of built-in analytic options. ReportPortal seamlessly integrates with mainstream CI/CD and mission administration platforms comparable to Jenkins, Jira and lots of useful and unit testing frameworks like Cucumber, JBehave, SpecFlow or Jasmine. Take a look at case execution outcomes are saved following the identical construction you might have in your reporting suites and take a look at plan. The take a look at instances are proven along with all associated information in a single place, proper the place you want it: logs, screenshots, binary information. The execution pipeline of sure take a look at instances are additionally accessible for you, so one can see earlier execution ends in one click on. ReportPortal additionally provides you the power to collaboratively analyze the take a look at automation outcomes. Explicit take a look at instances might be related to a product bug, an automation problem, a system problem or might be submitted as a problem ticket straight from the execution outcome. With every execution, ReportPortal mechanically figures out the foundation reason for a fail. On account of this evaluation, ReportPortal is marking a take a look at outcome with a flag. Engineers might be alerted about this problem to offer additional evaluation: if it has been resolved already or which take a look at outcomes require precise human evaluation.
Web site: https://github.com/reportportal/reportportal, https://reportportal.io/
Particular instruments
This part lists instruments that works with a restricted variety of programming languages and take a look at automation frameworks.
CI::Reporter
CI::Reporter is an add-on to Ruby testing frameworks like Take a look at::Unit or RSpec that permits you to generate XML experiences of your take a look at runs. The ensuing information might be learn by a steady integration system that understands Ant’s JUnit report XML format, thus permitting your CI system to trace take a look at/spec successes and failures. CI::Reporter works with tasks that use commonplace Rake duties for operating checks. On this trend, it hooks into testing frameworks utilizing setting variables acknowledged by these customized duties to inject the CI::Reporter code into the take a look at run.
Web site: https://github.com/ci-reporter/ci_reporter
Cluecumber Report Maven Plugin
Cluecumber Report Maven Plugin is an open supply maven plugin for clear and concise Cucumber BDD take a look at reporting. This plugin creates aggregated take a look at experiences from Cucumber appropriate JSON information which might be generated by Cucumber BDD, Karate and different frameworks.
Web site: https://github.com/trivago/cluecumber-report-plugin
Go-allure
Go-allure is an open supply command line utility for changing the output of Go checks into attract experiences.
Web site: https://github.com/robotomize/go-allure
HtmlTestRunner
HtmlTestRunner is an open supply software for ruby that saves ends in a human-readable HTML format. This Package deal was impressed by unittest-xml-reporting and HtmlTestRunner by tungwaiyip and commenced by combining the methodology of the previous with the performance of the latter.
Web site: https://github.com/oldani/HtmlTestRunner
Jasmine Reporters
Jasmine Reporters is a group of JavaScript jasmine reporter lessons that can be utilized with the JasmineBDD testing framework. It contains JUnitXmlReporter for producing junit xml output for operating in CI environments like Jenkins.
Web site: https://github.com/larrymyers/jasmine-reporters
jest-html-reporter
jest-html-reporter is an open supply Jest take a look at outcomes processor for producing a abstract in HTML. Jest is an open supply JavaScript testing framework that goals to work out of the field on most JavaScript tasks.
Web site: https://github.com/Hargne/jest-html-reporter
jest-html-reporters
Jest reporter is an open supply Jest take a look at outcomes processor for producing a abstract in HTML. Jest is an open supply JavaScript testing framework that goals to work out of the field on most JavaScript tasks.
Web site: https://github.com/Hazyzh/jest-html-reporters
karma-htmlfile-reporter
karma-htmlfile-reporter is a karma plugin for exporting unit take a look at outcomes as styled HTML file. This can be a plugin for the Karma Take a look at Runner. By including this reporter to your karma configuration, unit take a look at outcomes might be exported as a styled HTML file. For every take a look at browser, a separate desk is generated.
Web site: https://github.com/matthias-schuetz/karma-htmlfile-reporter
Subsequent Gen Report
Subsequent Gen Report (nxgreport) is an open supply easy light-weighted ruby gem to generate a lovely e-mailable take a look at report. (
Web site: https://github.com/iambalabharathi/nxgreport
ReportBuilder
ReportBuilder is a Ruby gem that merges Cucumber JSON experiences and builds mobile-friendly HTML Take a look at Report, JSON report and retry file.
Web site: https://github.com/rajatthareja/ReportBuilder
ReportNG
ReportNG is an easy HTML open supply reporting plug-in for the TestNG unit-testing framework. It’s meant as a substitute for the default TestNG HTML report. The default report is complete however shouldn’t be really easy to know at-a-glance. ReportNG supplies a easy, color-coded view of the take a look at outcomes. ReportNG generates 100% legitimate XHTML 1.0 information. The output might be custom-made by over-riding the default stylesheet with your personal CSS file.
Web site: https://testng.org/reportng/
Spock Stories
Spock Stories Extension is an open supply international extension for Spock to create take a look at (or, in Spock phrases, Specs) experiences. By default, the report creator generates an HTML report for every Specification, in addition to a abstract of all Specs which have been run (index.html). Should you want to have your personal template to generate experiences from, you should utilize the TemplateReportCreator. This lets you generate experiences in any textual content format.
Web site: https://github.com/renatoathaydes/spock-reports
XCTestHTMLReport
XCTestHTMLReport is an open supply Xcode-like HTML report for Unit and UI Checks. It helps parallel testing and attachments.
Web site: https://github.com/TitouanVanBelle/XCTestHTMLReport
