org.testng:testng

What's Changed

• GITHUB-2765: Propagate timeout stack trace to fix #2765 by @idontusenumbers in #3206
• Streamline working of shared thread pools by @krmahadevan in #3207
• Streamline xml serialisation to string by @krmahadevan in #3208
• test: improve osgi tests by @vlsi in #3213
• chore: setup-java Oracle action supports Java 21+ only by @vlsi in #3215
• Add DynamicImport-Package to load classes by name by @laeubi in #3220
• Use UUID backed instance id instead by @krmahadevan in #3218
• Ensure assertions is contents aware by @krmahadevan in #3228
• Fix: Ensure DataProvider parameters are refreshed on retry when cacheDataForTestRetries=false by @baflQA in #3250
• Improve/test workflow jdk25 by @baflQA in #3253
• Fix: issue 3231 retry infinite loop by @baflQA in #3251
• Fix Release process by @krmahadevan in #3255

New Contributors

• @idontusenumbers made their first contribution in #3206
• @laeubi made their first contribution in #3220

Full Changelog: 7.11.0...7.12.0

Fixed: GITHUB-3231: TestNG retry is going into infinite loop when the data provider returned object is modified before failure (Bartek Florczak) Update: Updated GitHub Actions test matrix to include JDK 25 and JDK 26 EA (Bartek Florczak) Fixed: GITHUB-3236: DataProvider parameters are not refreshed on retry when cacheDataForTestRetries=false (Bartek Florczak) Fixed: GITHUB-3227: assertEqualsNoOrder false positive when collection has same size and actual Collection is subset of expected collection (Krishnan Mahadevan) Fixed: GITHUB-3177: Method org.testng.xml.XmlSuite#toXml do not save new properties like "share-thread-pool-for-data-providers" (Krishnan Mahadevan) Fixed: GITHUB-3179: ClassCastException when use shouldUseGlobalThreadPool(true) property (Krishnan Mahadevan) Fixed: GITHUB-2765: Test timeouts using existing Executor now propagate the stack trace to the ThreadTimeoutException (Charlie Hayes)

What's Changed

• Report deadlocks when running in sharedthreadpool by @krmahadevan in #3119
• Update JCommander to 1.83 by @adessaigne in #3123
• Update Gradle Wrapper from 8.7 to 8.8 by @github-actions in #3132
• Fix array assertEquals failure message when an element is null or non-null by @zAlbee in #3136
• Fix typos on suiteName and testName attributes by @Philzen in #3137
• Fix javadoc for internal/Utils.java by @vlad8x8 in #3148
• Update Gradle Wrapper from 8.8 to 8.9 by @github-actions in #3149
• Fix assertEqualsDeep(Set, Set, String) wrong comparison and add variant without message by @vuriaval in #3150
• Bump github/combine-prs from 5.0.0 to 5.1.0 by @dependabot in #3151
• Bump burrunan/gradle-cache-action from 1 to 2 by @dependabot in #3168
• Specifying dataProvider & successPercentage causes test to always pass by @krmahadevan in #3171
• Update Gradle Wrapper from 8.9 to 8.10.1 by @github-actions in #3174
• Update Gradle Wrapper from 8.10.1 to 8.10.2 by @github-actions in #3178
• Bump oracle-actions/setup-java from 1.3.4 to 1.4.0 by @dependabot in #3176
• Bump gradle-update/update-gradle-wrapper-action from 1 to 2 by @dependabot in #3175
• Streamline invocation numbers in failed xml file by @krmahadevan in #3181
• Bump github/combine-prs from 5.1.0 to 5.2.0 by @dependabot in #3185
• Fix for #3189 - fixed ignored testcases count by @Kanaduchi in #3190
• update slf4j from 1.7.36 to 2.0.16 by @asolntsev in #3191
• Update Gradle Wrapper from 8.10.2 to 8.11.1 by @github-actions in #3188
• Introduce missing assertEquals(long, Long, String) overload by @Stephan202 in #3199
• Support to execlude somes test in option of command line by @YutingZhang-A in #3200
• Use newer build scan plugin by @jaredsburrows in #3203

New Contributors

• @adessaigne made their first contribution in #3123
• @zAlbee made their first contribution in #3136
• @Philzen made their first contribution in #3137
• @vlad8x8 made their first contribution in #3148
• @vuriaval made their first contribution in #3150
• @Stephan202 made their first contribution in #3199
• @YutingZhang-A made their first contribution in #3200
• @jaredsburrows made their first contribution in #3203

Full Changelog: 7.10.2...7.11.0

Fixed: GITHUB-3180: TestNG testng-failed.xml 'invocation-numbers' values are not calculated correctly with retry and dataproviders (Krishnan Mahadevan) Fixed: GITHUB-3170: Specifying dataProvider and successPercentage causes test to always pass (Krishnan Mahadevan) Fixed: GITHUB-3028: Execution stalls when using "use-global-thread-pool" (Krishnan Mahadevan) Fixed: GITHUB-3122: Update JCommander to 1.83 (Antoine Dessaigne) Fixed: GITHUB-3135: assertEquals on arrays - Failure message is missing information about the array index when an array element is unexpectedly null or non-null (Albert Choi) Fixed: GITHUB-3140: assertEqualsDeep on Sets - Deep comparison was using the wrong expected value Fixed: GITHUB-3189: Incorrect number of ignored tests displayed in the XML results Fixed: GITHUB-3196: support to execlude somes tests in option of command line

What's Changed

• Bump gradle/wrapper-validation-action from 2.1.2 to 3.3.0 by @dependabot in #3113
• Streaming working of listener comparator by @krmahadevan in #3118

Full Changelog: 7.10.1...7.10.2

Fixed: GITHUB-3117: ListenerComparator doesn't work (Krishnan Mahadevan)

What's Changed

• Bump org.gradlex.build-parameters from 1.4.3 to 1.4.4 by @dependabot in #3108
• Fix for #3110

Full Changelog: 7.10.0...7.10.1

Fixed: GITHUB-3110: Update from testng 7.9.0 to 7.10.0 break maven build with junit5 (Krishnan Mahadevan)

What's Changed

• Clean-up deprecated code by @krmahadevan in #3024
• Minor discrepancy fixes by @krmahadevan in #3026
• Deleting TestNG eclipse plugin specific classes by @krmahadevan in #3031
• Code clean-up by @krmahadevan in #3027
• Remove deprecated JUnit related support in TestNG by @krmahadevan in #3023
• Bump actions/upload-artifact from 3 to 4 by @dependabot in #3017
• Bump com.github.johnrengelman.shadow:com.github.johnrengelman.shadow.gradle.plugin from 7.0.0 to 8.1.1 by @dependabot in #2952
• Handle exceptions in emailable Reporter by @krmahadevan in #3042
• Add wrapperbot and update workflow order by @juherr in #3051
• Support ITestNGFactory customisation by @krmahadevan in #3060
• Streamline data provider listener invocation by @krmahadevan in #3058
• Streamline Guice Module creation in concurrency. by @krmahadevan in #3062
• Copy test result attributes when unexpected failures by @krmahadevan in #3065
• Upgrade Gradle to 8.6, use Java 17 for building code, target Java 11, polish buildscripts by @vlsi in #3067
• chore: use explicit dependency versions instead of refreshVersions by @vlsi in #3068
• Bump gradle/wrapper-validation-action from 1.1.0 to 2.1.1 by @dependabot in #3063
• Bump actions/checkout from 3 to 4 by @dependabot in #3052
• Remove Ant by @juherr in #3071
• Support ordering of listeners by @krmahadevan in #3072
• Bump oracle-actions/setup-java from 1.3.2 to 1.3.3 by @dependabot in #3073
• Add errorprone by @juherr in #2958
• Allow custom thread pool executors to be wired in. by @krmahadevan in #3075
• Allow data providers to be non cacheable by @krmahadevan in #3076
• Use Locks instead of synchronised keyword by @krmahadevan in #3077
• Document pgp artifact signing keys by @krmahadevan in #3085
• Add Unique Id for all test class instances by @krmahadevan in #3083
• Adding issue management workflows by @krmahadevan in #3090
• Map object to configurations by @krmahadevan in #3089
• Allow listeners to be disabled at runtime by @krmahadevan in #3088
• Streamline Data Provider execution by @krmahadevan in #3092
• Test case failure by @krmahadevan in #3093
• Honour inheritance when parsing listener factories by @krmahadevan in #3096
• Fixing flaky tests by @krmahadevan in #3098
• Tweaks around accessing SuiteResult by @krmahadevan in #3099
• Update Gradle Wrapper from 8.6 to 8.7 by @github-actions in #3101
• Bump oracle-actions/setup-java from 1.3.3 to 1.3.4 by @dependabot in #3103
• Bump gradle/wrapper-validation-action from 2.1.1 to 2.1.2 by @dependabot in #3104
• Streamline random generation by @krmahadevan in #3100
• Minor refactoring around RunOrder enum by @krmahadevan in #3106
• Streamline dependencies for configurations by @krmahadevan in #3107

New Contributors

• @github-actions made their first contribution in #3101

Full Changelog: 7.9.0...7.10.0

Fixed: GITHUB-3000: Method predecessors lookup and/or method sorting is broken in certain inheritance and naming setups (Krishnan Mahadevan) Fixed: GITHUB-3095: Super class annotated with ITestNGListenerFactory makes derived test class throw TestNGException on execution (Krishnan Mahadevan) Fixed: GITHUB-3081: Discrepancy with combination of (Shared Thread pool and Method Interceptor) (Krishnan Mahadevan) Fixed: GITHUB-2381: Controlling the inclusion of the listener at runtime (Krishnan Mahadevan) Fixed: GITHUB-3082: IInvokedMethodListener Iinvoked method does not return correct instance during @BeforeMethod, @AfterMethod and @AfterClass (Krishnan Mahadevan) Fixed: GITHUB-3084: Document project's PGP artifact signing keys (Krishnan Mahadevan) Fixed: GITHUB-3079: Associate a unique id with every test class object instantiated by TestNG (Krishnan Mahadevan) Fixed: GITHUB-3040: replace the usages of synchronized with ReentrantLock (Krishnan Mahadevan) Fixed: GITHUB-3041: TestNG 7.x DataProvider works in opposite to TestNG 6.x when retrying tests. (Krishnan Mahadevan) Fixed: GITHUB-3066: How to dynamically adjust the number of TestNG threads after IExecutorFactory is deprecated? (Krishnan Mahadevan) New: GITHUB-2874: Allow users to define ordering for TestNG listeners (Krishnan Mahadevan) Fixed: GITHUB-3033: Moved ant support under own repository https://github.com/testng-team/testng-ant (Julien Herr) Fixed: GITHUB-3064: TestResult lost if failure creating RetryAnalyzer (Krishnan Mahadevan) Fixed: GITHUB-3048: ConcurrentModificationException when injecting values (Krishnan Mahadevan) Fixed: GITHUB-3050: Race condition when creating Guice Modules (Krishnan Mahadevan) Fixed: GITHUB-3059: Support the ability to inject custom listener factory (Krishnan Mahadevan) Fixed: GITHUB-3045: IDataProviderListener - beforeDataProviderExecution and afterDataProviderExecution are called twice in special setup (Krishnan Mahadevan) Fixed: GITHUB-3038: java.lang.IllegalStateException: Results per method should NOT have been empty (Krishnan Mahadevan) Fixed: GITHUB-3022: Remove deprecated JUnit related support in TestNG (Krishnan Mahadevan)

Fixed: GITHUB-2019: Total thread count in testng parallel tests with dataproviders (Krishnan Mahadevan) Fixed: GITHUB-3006: ITestResult injected at @AfterMethod incorrect when a configuration method failed (Krishnan Mahadevan) Fixed: GITHUB-2980: Data Provider Threads configuration in the suite don't match the documentation (Krishnan Mahadevan) Fixed: GITHUB-3003: BeforeClass|AfterClass with inheritedGroups triggers cyclic dependencies (Krishnan Mahadevan) New: Added @Inherited to the Listeners annotation, allowing it to be used in forming meta-annotations. (Pavlo Glushchenko) Fixed: GITHUB-2991: Suite attributes map should be thread safe (Krishnan Mahadevan) Fixed: GITHUB-2974: Command line arguments -groups and -excludegroups override defined groups in a suite xml file (dr29bart) Fixed: GITHUB-2961: "Unexpected value: 16" error when multiple beforeMethod config methods with firstTimeOnly property run before a test (Krishnan Mahadevan) Fixed: GITHUB-2904: Add location of docs Github to readme and contributions page (Mohsin Sackeer) Fixed: GITHUB-2934: Parallel Dataproviders & retries causes test result count to be skewed (Krishnan Mahadevan) Fixed: GITHUB-2925: Issue in ITestcontext.getAllTestMethods() with annotation @BeforeSuite (Krishnan Mahadevan) Fixed: GITHUB-2928: The constructor of TestRunner encountered NBC changes in 7.8.0 (Krishnan Mahadevan) Fixed: GITHUB-581: Parameters of nested test suites are overridden(Krishnan Mahadevan) Fixed: GITHUB-727 : Fixing data races (Krishnan Mahadevan) Fixed: GITHUB-2913: Maps containing nulls can be incorrectly considered equal (Alex Heneveld)

Fixed: GITHUB-2906: Generate testng-results.xml per test suite (Krishnan Mahadevan) New: GITHUB-2897: Not exception but warning if some (not all) of the given test names are not found in suite files. (Bruce Wen) New: GITHUB-2907: Added assertListContains and assertListContainsObject methods to check if specific object present in List (Dmytro Budym) Fixed: GITHUB-2888: Skipped Tests with DataProvider appear as failed (Joaquin Moreira) Fixed: GITHUB-2884: Discrepancies with DataProvider and Retry of failed tests (Krishnan Mahadevan) Fixed: GITHUB-2879: Test listeners specified in parent testng.xml file are not included in testng-failed.xml file (Krishnan Mahadevan) Fixed: GITHUB-2866: TestNG.xml doesn't honour Parallel value of a clone (Krishnan Mahadevan) Fixed: GITHUB-2875: JUnitReportReporter should capture the test case output at the test case level Fixed: GITHUB-2771: After upgrading to TestNG 7.5.0, setting ITestResult.status to FAILURE doesn't fail the test anymore (Julien Herr & Krishnan Mahadevan) Fixed: GITHUB-2862: Allow test classes to define "configfailurepolicy" at a per class level (Krishnan Mahadevan) Fixed: GITHUB-2796: Option for onAfterClass to run after @AfterClass (Oliver Hughes) Fixed: GITHUB-2857: XmlTest index is not set for test suites invoked with YAML (Sergei Baranov) Fixed: GITHUB-2880: Before configuration and before invocation set 'SKIP' when beforeMethod is 'skip' (Bob Shi) Fixed: GITHUB-2886: testng-results xml reports config skips from base classes as ignored (Krishnan Mahadevan)

Fixed: GITHUB-2854: overloaded assertEquals methods do not work from Groovy (Krishnan Mahadevan)

Fixed: GITHUB-2852: [SECURITY] Fix Zip Slip Vulnerability (Jonathan Leitschuh) Fixed: GITHUB-2792: JUnitTestClass sets XmlTest as null when running JUnit 4 Tests using TestNG (Krishnan Mahadevan) Fixed: GITHUB-2847: Deprecate support for running JUnit tests (Krishnan Mahadevan) Fixed: GITHUB-2844: Deprecate support for running Spock Tests (Krishnan Mahadevan) Fixed: GITHUB-550: Weird @BeforeMethod and @AfterMethod behaviour with dependsOnMethods (Krishnan Mahadevan) Fixed: GITHUB-893: TestNG should provide an Api which allow to find all dependent of a specific test (Krishnan Mahadevan) New: Added .yml file extension for yaml suite files, previously only .yaml was allowed for yaml (Steven Jubb) Fixed: GITHUB-141: regular expression in "dependsOnMethods" does not work (Krishnan Mahadevan) Fixed: GITHUB-2770: FileAlreadyExistsException when report is generated (melloware) Fixed: GITHUB-2825: Programmatically Loading TestNG Suite from JAR File Fails to Delete Temporary Copy of Suite File (Steven Jubb) Fixed: GITHUB-2818: Add configuration key for callback discrepancy behavior (Krishnan Mahadevan) Fixed: GITHUB-2819: Ability to retry a data provider in case of failures (Krishnan Mahadevan) Fixed: GITHUB-2308: StringIndexOutOfBoundsException in findClassesInPackage - Surefire/Maven - JDK 11 fails (Krishnan Mahadevan) Fixed: GITHUB:2788: TestResult.isSuccess() is TRUE when test fails due to expectedExceptions (Krishnan Mahadevan) Fixed: GITHUB-2800: Running Test Classes with Inherited @Factory and @DataProvider Annotated Non-Static Methods Fail (Krishnan Mahadevan) New: Ability to provide custom error message for assertThrows\expectThrows methods (Anatolii Yuzhakov) Fixed: GITHUB-2780: Use SpotBugs instead of abandoned FindBugs Fixed: GITHUB-2801: JUnitReportReporter is too slow Fixed: GITHUB-2807: buildStackTrace should be fail-safe (Sergey Chernov) Fixed: GITHUB-2830: TestHTMLReporter parameter toString should be fail-safe (Sergey Chernov) Fixed: GITHUB-2798: Parallel executions coupled with retry analyzer results in duplicate retry analyzer instances being created (Krishnan Mahadevan)

Fixed: GITHUB-2761: Exception: ERROR java.nio.file.NoSuchFileException: /tmp/testngXmlPathInJar-15086412835569336174 (Krishnan Mahadevan)

Fixed: GITHUB-2741: Show fully qualified name of the test instead of just the function name for better readability of test output.(Krishnan Mahadevan) Fixed: GITHUB-2725: Honour custom attribute values in TestNG default reports (Krishnan Mahadevan) Fixed: GITHUB-2726: @AfterClass config method is executed for EACH @Test method when parallel == methods (Krishnan Mahadevan) Fixed: GITHUB-2752: TestListener is being lost when implenting both IClassListener and ITestListener (Krishnan Mahadevan) New: GITHUB-2724: DataProvider: possibility to unload dataprovider class, when done with it (Dzmitry Sankouski) Fixed: GITHUB-217: Configure TestNG to fail when there's a failure in data provider (Krishnan Mahadevan) Fixed: GITHUB-2743: SuiteRunner could not be initial by default Configuration (Nan Liang) Fixed: GITHUB-2729: beforeConfiguration() listener method should be invoked for skipped configurations as well(Nan Liang) Fixed: assertEqualsNoOrder for Collection and Iterators size check was missing (Adam Kaczmarek) Fixed: GITHUB-2709: Testnames not working together with suites in suite (Martin Aldrin) Fixed: GITHUB-2704: IHookable and IConfigurable callback discrepancy (Krishnan Mahadevan) Fixed: GITHUB-2637: Upgrade to JDK11 as the minimum JDK requirements (Krishnan Mahadevan) Fixed: GITHUB-2734: Keep the initial order of listeners (Andrei Solntsev) Fixed: GITHUB-2359: Testng @BeforeGroups is running in parallel with testcases in the group (Anton Velma) Fixed: Possible StringIndexOutOfBoundsException in XmlReporter (Anton Velma) Fixed: GITHUB-2754: @AfterGroups is executed for each "finished" group when it has multiple groups defined (Anton Velma)

Fixed: GITHUB-2701: Bump gradle version to 7.3.3 to support java17 build (ZhangJian He) Fixed: GITHUB-2646: Streamline Logging Across TestNG (Krishnan Mahadevan) Fixed: GITHUB-2658: Inheritance + dependsOnMethods (Krishnan Mahadevan) Fixed: GITHUB-2664: Order for DependsOnGroups has changed after TestNg 7.4.0 (Krishnan Mahadevan) Fixed: GITHUB-2501: TestNG 7.4.0 throws an exception "sun.net.www.protocol.file.FileURLConnection cannot be cast to java.net.HttpURLConnection" when xml file contain "ENTITY SYSTEM" grammer (Krishnan Mahadevan) Fixed: GITHUB-2693: TestNG ignores 'dataproviderthreadcount' CLA (Krishnan Mahadevan) Fixed: GITHUB-2685: TestInvoker should clear Thread.interrupted flag before calling ITestListeners (Roman Morskyi) Fixed: GITHUB-2684: AfterGroups config annotation does not consider retries for tests (Roman Morskyi) Fixed: GITHUB-2689: Yaml parser: implement loadClasses flag (Dzmitry Sankouski) Fixed: GITHUB-2676: NPE is triggered when working with ITestObjectFactory (Krishnan Mahadevan) Fixed: GITHUB-2674: Run onTestSkipped for each value from data provider (Krishnan Mahadevan) Fixed: GITHUB-2672: Log real stacktrace when test times out. (cdalexndr) Fixed: GITHUB-2669: A failed retry with ITestContext will lose the ITestContext. (Nan Liang) Fixed: GITHUB-2643: assertEquals(Set,Set) now ignores ordering as it did before. (Elis Edlund) Fixed: GITHUB-2653: Assert methods requires casting since TestNg 7.0 for mixed boxed and unboxed primitives in assertEquals. Fixed: GITHUB-2229: Restore @BeforeGroups and @AfterGroups Annotations functionality (Krishnan Mahadevan) Fixed: GITHUB-2563: Skip test if its data provider provides no data (Krishnan Mahadevan) Fixed: GITHUB-2535: TestResult.getEndMillis() returns 0 for skipped configuration - after upgrading testng to 7.0 + (Krishnan Mahadevan) Fixed: GITHUB-2638: "[WARN] Ignoring duplicate listener" appears when running .xml suite with <listeners> and <suite-files> (Krishnan Mahadevan) Fixed: GITHUB-1297: Passed configuration methods appear in testng-failed.xml, when failure was after passed test (Dzmitry Sankouski) New: Decouple configuration unit tests from main suite (Dzmitry Sankouski). Fixed: GITHUB-2536: Problems with Nested Test Classes (Krishnan Mahadevan) Fixed: GITHUB-2558:Make IExecutionListener, ITestListener, IInvokedMethodListener, IConfigurationListener, ISuiteListener finish method with reverse order (dianny) Fixed: GITHUB-2532: Apply commandline switches for suites in jar files (Dzmitry Sankouski). Fixed: GITHUB-2558: Make IExecutionListener, ITestListener, IInvokedMethodListener, IConfigurationListener, ISuiteListener execute in the order of insertion (Krishnan Mahadevan) Fixed: GITHUB-2611: Config Failures not included in testng-failed.xml when its part of a different test class (Krishnan Mahadevan) Fixed: GITHUB-2613: Ignored Tests are not retrieved for a mixed test class (test with enabled, disabled and ignored test method) (Krishnan Mahadevan) Fixed: GITHUB-849: Performance improvement by fixing hashCode (testn & Vladimir Sitnikov) Fixed: GITHUB-2570: Use Guice injector for instantiate IRetryAnalyzer (Krishnan Mahadevan) Fix: use proper instances for beforeClass callback when different instances collide on hash codes Fixed: Fix parallel and configfailurepolicy parsing in tr_TR locale Fixed: Wrong results from GuiceBasedObjectDispenser when there's Object#hashCode collision Changed: GITHUB-2564: Source code is split into several modules for better modularity in the future (for now only a combined jar is released as it was before) Added: GITHUB-2564: Added license files as META-INF/LICENSE.txt within the released jar Test: GITHUB-2564: Added pax-exam-based OSGi test to verify the manifest Changed: GITHUB-2564: Migrated the build to Gradle 7.0.2 Fixed: GITHUB-2576: Guice 5.0 drops no-aop variant, so TestNG should probably upgrade and avoid no-aop dependency (Nan Liang) Fixed: GITHUB-2566: Reporter#getOutput(ITestResult tr) uses Map.get(tr.hashCode()) which might result in surprising results (Krishnan Mahadevan) Fixed: GITHUB-2565: Dataprovider only supporting a raw type for Iterator return type (Krishnan Mahadevan) Fixed: GITHUB-2557: Flaky test: ThreadAffinityTest#testThreadAffinity (Krishnan Mahadevan) Fixed: GITHUB-2567: MethodHelper#CANONICAL_NAME_CACHE is never reset, so it could result in a memory leak (Krishnan Mahadevan) Fixed: GITHUB-2540: assertEquals(Collection) need check order (Stuart Marks & Julien Herr) Fixed: GITHUB-2360: Groovy 3 internal generated methods are detected as test methods (Ian Springer) Fixed: GITHUB-2522: TestNG 7.4.0 Can not skip test through listener (Nan Liang) Fixed: GITHUB-2529: Link to testng.xml in CONTRIBUTING.md file was dead Fixed: GITHUB-2521: The method has a separate string with 'invocation-number' parameters for each failure (does not group) in 'testng-failed.xml' file for DataProvider + Factory (Pavel Sakharchuk) Fixed: GITHUB-2426: New feature TestNG - getFactoryMethodParamsInfo on ConfigurationMethod (Krishnan Mahadevan) Fixed: GITHUB-2517: Factory data-provider parameters not displayed in 'testng-failed.xml' file (Pavel Sakharchuk) Fixed: GITHUB-279: Guice dependency injection into listeners and reporters (Krishnan Mahadevan) Fixed: GITHUB-2504: Data provider data cannot be populated to onTestStart hook when BeforeMethod is failed (Krishnan Mahadevan) Fixed: GITHUB-2489: Hierarchical base- and test-class @AfterClass methods out of order using groups (Krishnan Mahadevan) Fixed: GITHUB-2493: Avoid NPE from TextReporter execution when a dataprovider method provides null (baflQA) Fixed: GITHUB-2483: Asymmetric not equals (cdalexndr) Fixed: GITHUB-2486: assertSame/assertNotSame broken after GITHUB-2296 (Vitalii Diravka) Fixed: GITHUB-2490: assertNotEquals returns fast when argument is null, not calling equals(Object) (Anindya Roy) Fixed: GITHUB-2500: Mention in assertEqualsNoOrder doc that arrays are not compared deeply (Marcono1234) Fixed: GITHUB-2544: TestNG Retry Fails on complex data-provider arguments (Anindya Roy)

New : GITHUB-2459: Support configurable start time - emailable report (Barry Evans) Fixed: GITHUB-2467: XmlTest does not copy the xmlClasses during clone (C.V.Aditya) Fixed: GITHUB-2469: Parameters added in XmlTest during AlterSuiteListener not available in SuiteListener (C.V.Aditya) Fixed: GITHUB-2296: Fix for assertEquals not working for sets as order is not guaranteed. (Prashant Maroti) Fixed: GITHUB-2465: Fix bux where Strings.join returns empty String Fixed: GITHUB-1632: throwing SkipException sets iTestResult status to Failure instead of Skip (Julien Herr & Krishnan Mahadevan) New : GITHUB-2456: Add onDataProviderFailure listener (Krishnan Mahadevan) Fixed: GITHUB-2445: NPE in FailedReporter.java With Tests Created in Factory (Arham Jain) Fixed: GITHUB-2428: Configuration methods have the same test class instance when @Factory is being used (Nan Liang) Fixed: GITHUB-2440: Fixed an issue when case timeout returned an incorrect exception and effect the next other test case (Yao Ma) New: GITHUB-2407: Adds "overrideIncludedMethods" to the global config as a command-line argument, which excludes explicitly included test methods if they belong to any excluded groups (Nikhil Suri) Fixed: GITHUB-2432: Rework MethodInheritance.fixMethodInheritance to "soft" dependencies (Krishnan Mahadevan) Fixed: GITHUB-2429: Seggregate Dependency Injection out as a clear implementation (Krishnan Mahadevan) Fixed: GITHUB-2435: getParameterIndex() always return 0 in test listener Fixed: GITHUB-2406: TestNG 7.3.0 transitive vulnerability CVE-2020-11022 and CVE-2020-11023 due to JQuery 3.4.1 (Krishnan Mahadevan) Fixed: GITHUB-2405: Regression: Using TestNG via Maven breaks when optional Guice dependency is unavailable (Krishnan Mahadevan) Fixed: GITHUB-2427: Guice module (suite parent-module and test module) configure() method is called multiple times (Jacek Centkowski) Fixed: GITHUB-2419: TestNG JUnit reports are not valid if system output contains XML tags (Lorenzo Orsatti) Fixed: GITHUB-188: suite parallel="methods" does not work when there are multiple <test> tags in the testng.xml (Krishnan Mahadevan) Fixed: GITHUB-346: When a method is annotated with both BeforeGroups and AfterGroups only AfterGroup is executed (Krishnan Mahadevan) Fixed: GITHUB-2403: Suite.xml files attempt to make web request when suite references standard TestNG DTD using HTTP (Krishnan Mahadevan) New: GITHUB-2385: Make @Listeners can work for implemented interfaces and Inherited class (Nan Liang) Fixed: GITHUB-2053: MethodHelper.collectAndOrderMethods() Hangs when Parallel Instance and dependsOnGroups (Krishnan Mahadevan) Fixed: GITHUB-2400: BeforeClass/Method (and AfterClass/Method) configuration methods that override default methods are invoked multiple times (Krishnan Mahadevan) Fixed: GITHUB-2396: @Ignore on method level doesn't work as expected (Krishnan Mahadevan) Fixed: GITHUB-2382: TestNG version should be specified in MANIFEST.MF (Krishnan Mahadevan) Fixed: GITHUB-2096: 7.0.0-beta6 memory issues (regression) (Krishnan Mahadevan) Fixed: GITHUB-2355: TestNG creates multiple Guice Module Instances (Krishnan Mahadevan) Fixed: GITHUB-2374: Add file name to the warning message (Krishnan Mahadevan) Fixed: GITHUB-2321: -Dtestng.thread.affinity=true do not work when running multiple instance of test in parallel (Nan Liang) Fixed: GITHUB-2363: JS error when switching theme (Krishnan Mahadevan) Fixed: GITHUB-2361: No way to enforce @Test(singleThreaded = true) when test defined in base class (Krishnan Mahadevan) Fixed: GITHUB-2343: Injectors are not reused when they share the same set of modules (Krishnan Mahadevan) Fixed: GITHUB-2346: ITestResult attributes are null when retrieved by Listener onTestStart if test fails at BeforeMethod (Krishnan Mahadevan) Fixed: GITHUB-2357: TestNG 7.3.0 transitive dependencies

Fixed: GITHUB-2328: Add ability to get test method for which configuration method was called (Krishnan Mahadevan) Fixed: GITHUB-2327: Parameters not present on skipped Test (Eric Kubenka) Fixed: GITHUB-2232: Null Pointer Exception in ConfigInvoker.setMethodInvocationFailure (Krishnan Mahadevan) Fixed: GITHUB-2312: IAnnotationTransformer called multiple time Discrepancy between 6.x and 7.x (Krishnan Mahadevan) New: GITHUB-2315: TextReporter console output does not nicely print native array data parameters (James Sassano) Fixed: GITHUB-2301: Add support for object-based reporter configurations (Scott Babcock) New: Deprecate org.testng.ReporterConfig (Julien Herr) Fixed: GITHUB-2300: Vulnerable Dependency: Please upgrade JCommander to 1.75 or above (Krishnan Mahadevan) Fixed: GITHUB-2182: Removed exception catching as valid behaviour Fixed: GITHUB-2273: Use SPI to load Guice modules (Bartosz Popiela) Fixed: GITHUB-2280: Prevent Retry from happening endlessly (Paweł Nadolski) Fixed: GITHUB-553: Better error message when dealing with classes having both Constructor and Factory methods (Krishnan Mahadevan) Fixed: GITHUB-2267: RetryAnalyzer is not set properly and consistent when using dataprovider (Eric Kubenka) Fixed: GITHUB-2266: Support Test retries via Callbacks (Krishnan Mahadevan) Fixed: GITHUB-2209: @Before and @After are not executed as expected when a combination of class and method level grouping is applied (Krishnan Mahadevan) Fixed: GITHUB-2259: Missing configuration for ServiceLoader Listeners (Krishnan Mahadevan) Fixed: GITHUB-2257: Facilitate retry of configuration methods via call backs (Krishnan Mahadevan) Fixed: GITHUB-2223: testng 7.1.0 java.lang.ClassNotFoundException: com.google.inject.Stage (Krishnan Mahadevan) Fixed: GITHUB-217: Configure TestNG to fail when all tests are skipped (Krishnan Mahadevan) Fixed: GITHUB-2255: Ensure test method parameters are visible in BeforeMethod config method (Krishnan Mahadevan) Fixed: GITHUB-2251: NullPointerException at test with timeOut (Krishnan Mahadevan) Fixed: GITHUB-2249: Not abstract super-classes mess up test run order (Sergii Kim) Fixed: GITHUB-2195: NPE Using groups and @Before/@AfterMethod with alwaysRun and dependsOnMethods (Tomas & Julien Herr) Fixed: GITHUB-2238: Parameter values should be overridable from JVM arguments (Krishnan Mahadevan) Fixed: GITHUB-2231: Incorrect hierarchy in testng.xml file created programmatically and failed to run. (Krishnan Mahadevan) Fixed: GITHUB-2235: expectedExceptions mismatch because of wrapping (Krishnan Mahadevan) Fixed: GITHUB-2220: ITestListener's methods get called multiple times for one test, when @Listeners annotation is used in multiple test classes (Krishnan Mahadevan) Fixed: GITHUB-2211: assertEquals for Map sometimes does not use provided message (Krishnan Mahadevan) Fixed: GITHUB-1632: throwing SkipException sets iTestResult status to Failure instead of Skip (Krishnan Mahadevan) Fixed: GITHUB-2207: Allow dependency injector factory to be configured via args (dmikhievich) Fixed: GITHUB-2193: Ignore local url for DTD security check (Li.Zhao & Julien Herr) Fixed: GITHUB-1968: Upgrade to Gradle 6.0

New : GITHUB-2199: Allow users to provide their own Injector for Dependency Injection (Krishnan Mahadevan) Fixed: GITHUB-2180: Write scenario details for Retried tests (Devendra Raju K) Fixed: GITHUB-2124: JUnit Report should contain the output of org.testng.Reporter (Krishnan Mahadevan) Fixed: GITHUB-2171: Ability to embed attachments and make them available on TestNG XML report (Krishnan Mahadevan) Fixed: GITHUB-2152: Multiple Test Groups Causing @BeforeMethod and @AfterMethod to be called multiple times for a single test (Krishnan Mahadevan) Fixed: GITHUB-2172: Suite summary report table issue with EmailableReporter2.java (Devendra Raju K) Fixed: GITHUB-2148: TestNG - configfailurepolicy=“continue” is not working for retried test (Krishnan Mahadevan) Fixed: GITHUB-2163: Test is executed infinite number of times when the data provider returns a new object (Krishnan Mahadevan) Fixed: GITHUB-2157: NullPointerException occurs when a Retried test has an exception in DataProvider (Krishnan Mahadevan) Fixed: GITHUB-2150: Upgraded jQuery from 1.7.1 to 3.4.1 to resolve reported prototype pollution vulnerability Fixed: GITHUB-2149: Handle NoClassDefFoundError when classloader fails to load a class New: GITHUB-2111: Provide an interceptor for Data Provider (Krishnan Mahadevan) Fixed: GITHUB-1709: @Ignore doesn't work when used on child class, and parent has multiple @Test methods (Krishnan Mahadevan) New: GITHUB-2118: Default assertion message (Jiong Fu) Fixed: GITHUB-2080: Wrong text for assertTrue (Jiong Fu) Fixed: GITHUB-2078: Xml to Yaml does not capture the dependency definition in generated Yaml File (Jiong Fu)

Fixed: GITHUB-2137: Issue with Priority with too low values (Krishnan Mahadevan) Fixed: GITHUB-2138: Factory calls are not intercepted by IAnnotationTransformer (Krishnan Mahadevan) Fixed: GITHUB-2121: SoftAssert: allow custom message (dr29bart) Fixed: GITHUB-2110: NPE is thrown when running with Thread affinity (Krishnan Mahadevan) Fixed: GITHUB-2069: JUnit TestSuite not handled correctly in Reports (Krishnan Mahadevan) Fixed: GITHUB-2075: Thread interrupt flag persists between test methods (Krishnan Mahadevan) Fixed: GITHUB-2074: Thread Interrupted when using expectedException (Krishnan Mahadevan) Fixed: GITHUB-2802: some methods' javadoc in ISuiteListener and ITestListener is misleading (Yehui Wang) Fixed: GITHUB-2061: java.util.ConcurrentModificationException after registration of SuiteListener at the runtime (Krishnan Mahadevan) Fixed: GITHUB-2055: It's not possible to register a new ITestListener at the runtime (Krishnan Mahadevan) Fixed: GITHUB-1035: @BeforeClass not executed in parallel when parallel="instances" (Krishnan Mahadevan) Fixed: GITHUB-2043: IConfigurationListener is not executed and IDataProviderListener is not added to the list of the listeners if a new listener is added at the runtime (Krishnan Mahadevan) Fixed: GITHUB-1835: Configurable ThreadPoolExecutor (Krishnan Mahadevan) Fixed: GITHUB-1691: Support reading data provider information from Class level @Test annotation (Krishnan Mahadevan) Fixed: GITHUB-326: When group-by-instances is set to true the instances created by @Factory does not run in parallel (Krishnan Mahadevan) Fixed: GITHUB-1930: Running testng-failed.xml correctly runs failed test in child class but incorrectly runs all tests in base class (Krishnan Mahadevan) Fixed: GITHUB-1987: Retrieve the data provider method's reference from the test method (Krishnan Mahadevan) Fixed: GITHUB-1976: Add a proper message when types are missing during annotation parsing (Krishnan Mahadevan) Fixed: GITHUB-2000: ThreadPoolExecutor ConcurrentModificationException (Krishnan Mahadevan) Fixed: GITHUB-2022: Suite.xml files make web request when suite uses DTD over HTTPS (Krishnan Mahadevan) Fixed: GITHUB-2017: Don't use child injectors for Guice tests (Joe Barnett) New: GITHUB-2003: Add to @Test interface fields IDs and issues (Krishnan Mahadevan) Fixed: GITHUB-2009: Test Timeout not respected in parallel="methods" mode (Krishnan Mahadevan) Fixed: GITHUB-2008: Preserve parameters in each class and not all of the test parameters Fixed: GITHUB-1981: Fixes NPE in Assert.assertEquals when an array contains null (Maneesh MS) New: Upgrade to gradle5 New: Added a method in Assertion class to allow downstream TestNG consumers to override the error message (Ryan Laseter) Fixed: GITHUB-165: @AfterGroups is not executed when group member fails or is skipped (Krishnan Mahadevan) Fixed: GITHUB-118: @BeforeGroups only called if group is specified explicitly (Krishnan Mahadevan) Fixed: GITHUB-182: Inherited test methods do not get expected group behavior (Krishnan Mahadevan) Fixed: GITHUB-1988: Add Automatic-Module-Name to MANIFEST.MF (Krishnan Mahadevan) Fixed: GITHUB-1985: Custom "IMethodSelector" implementation doesn't filter methods properly (Krishnan Mahadevan) Fixed: GITHUB-993: Handle null test names from ITest implementation (Krishnan Mahadevan) Fixed: GITHUB-1942: assertDeepEquals takes long time to evaluate size mismatch (Kumaran Bharathan) Fixed: GITHUB-1967: IInvokedMethod ITestResult status set as -1 for test methods skipped due to config failure (Krishnan Mahadevan) Fixed: GITHUB-1952: Provide a TestNGListener that can be invoked when a test fails due to a timeout (Krishnan Mahadevan) Fixed: GITHUB-1953: TestNG throws a misleading error when @Factory method returns empty array. (Krishnan Mahadevan) Fixed: GITHUB-1946: Retry analyzer does not work properly when coupled with a data provider (Krishnan Mahadevan) Fixed: GITHUB-1924: Testclass instantiation fails when both no-arg constructor and factory method present (Krishnan Mahadevan) Fixed: GITHUB-1935: Wrong text for assertEquals (Krishnan Mahadevan) Fixed: GITHUB-1931: [NPE] Reporter org.testng.reporters.jq.Main failed (Krishnan Mahadevan, Oleg Shaburov) New: Remove raw type warnings in ConversionUtils Fixed: GITHUB-1480: Parallel=methods not working when tests have different priorities set (Micah Lapping-Carr) Fixed: GITHUB-1041: Factory data-provider parameters not displayed in test-result (Krishnan Mahadevan) Fixed: GITHUB-1901: The overall reported Test time for suite containing parallel tests should be max(tests_times) (Krishnan Mahadevan) Fixed: GITHUB-1893: Streamline invocation of "init" method within TestResult to be private (Krishnan Mahadevan) Fixed: GITHUB-1892: Configurable InvokedMethodListener (Krishnan Mahadevan) Fixed: GITHUB-435: Apply <packages> at suite level to all tests (Siegmar Alber) Fixed: GITHUB-1870: Fix Ambiguous behavior of IInvokedMethodListener by clarifying javadocs(Krishnan Mahadevan) Fixed: GITHUB-1878: Provide visibility into the actual method (config/test) that caused a downstream test to be skipped (Krishnan Mahadevan) Fixed: GITHUB-1883: Establish Listener invocation order as tests for documentation purposes (Krishnan Mahadevan) Fixed: TestNG doesn't use the searchable loaders for JUnit tests (Igor Ignatev) Fixed: GITHUB-1880: @AfterGroups(alwaysRun = true) is not called if there is an exception in @BeforeGroups (Krishnan Mahadevan) Fixed: GITHUB-1874: Prevent circular dependency error when suite includes different methods from same class (Krishnan Mahadevan) Fixed: GITHUB-1863: IMethodInterceptor will be invoked twice when listener implements both ITestListener and IMethodInterceptor via eclipse execution way.(Bin Wu) Fixed: GITHUB-1865: testngXmlPathInJar- cannot be cleared when vm terminate(Yehui Wang) Fixed: GITHUB-1850: Parser returns a wrong structure when parent suite has duplicate child suites (Chao Qin) Fixed: GITHUB-1803: Added new methods for comparing float and double arrays with delta (Atul Agrawal) Fixed: GITHUB-1661: Fixed Assert logic for two dimensional arrays (Atul Agrawal) Fixed: GITHUB-1734: Added unit tests for NaN, Max, Min, Positive and Negative infinity (Atul Agrawal) Fixed: GITHUB-1740: Bumped up artifact version of dependencies for java 8 support (Atul Agrawal) Fixed: GITHUB-1834: Ensure group dependency defined via suite xml is considered (Krishnan Mahadevan) New : Expose Graph Visualisation representation for users to build real-time debugging tools (Krishnan Mahadevan) Fixed: GITHUB-574: String cannot be cast to Integer if log property is set in maven pom.xml (Krishnan Mahadevan) Fixed: GITHUB-1402: TestNG reporting - Quickly identify Retry-d Tests (Krishnan Mahadevan) Fixed: GITHUB-1697: Need ability to mark "failed but up for retry" tests differently from "skipped" tests (Krishnan Mahadevan) Fixed: GITHUB-1810: Nullpointer exception when running TestNG tests from CMD (Krishnan Mahadevan) Fixed: GITHUB-1590: Started configuration method has wrong status and end time during execution (Krishnan Mahadevan) Fixed: GITHUB-298: Avoid Javascript errors in function name when suite name has special characters (Krishnan Mahadevan) Fixed: GITHUB-1048: Fix validation errors in TestNG CSS to adhere to CSS level 3 + SVG (Krishnan Mahadevan) Fixed: GITHUB-341: TestNG does not respect "-parallel classes" when running with a jar file (Krishnan Mahadevan) Fixed: GITHUB-1790: IAnnotationTransformer transform method is not called for all test classes annotated with @Test (Krishnan Mahadevan) Fixed: GITHUB-1787: Method level parameters are missing when using Yaml generation utilities (Krishnan Mahadevan) New : TestNG now guarantees thread-affinity for methods that use either preserve-order (or) dependsOnMethods (Krishnan Mahadevan) Fixed: GITHUB-1773: Parallel="classes" executes methods of test class in different threads (Krishnan Mahadevan) Fixed: GITHUB-1185: DependsOnMethods made parallel class use several threads (Krishnan Mahadevan) Fixed: GITHUB-1173: Parallel="classes" executes methods of test class in different threads (Krishnan Mahadevan) Fixed: GITHUB-1066: Regression is in priority. It broke parallel mode (Krishnan Mahadevan) Fixed: GITHUB-1050: Parallel classes runs methods from one class in different threads, interleaves two classes in one thread (Krishnan Mahadevan) Fixed: GITHUB-89: parallel="classes" is not forcing test methods from the same testClass to be run in the same thread as it is suposed to (Krishnan Mahadevan) Fixed: GITHUB-1719: successPercentage does not work correctly for tests with dataProvider (Krishnan Mahadevan) Fixed: GITHUB-1241: Streamline Retry Analyzer usage when same test is run multiple times (Krishnan Mahadevan) Fixed: GITHUB-1777: ITestListener.onTestStart() not called after fail or skip from @BeforeMethod (Krishnan Mahadevan) Fixed: GITHUB-1778: SoftAssert#fail swallows actual root cause (Krishnan Mahadevan) Fixed: GITHUB-1665: Failed test after rerun would impact next test case result (Yehui Wang) Fixed: GITHUB-1770: Factory annotation does not work with Parameters annotation on method constructor (Krishnan Mahadevan) Fixed: GITHUB-1767: Prevent NPE when XmlTest accessed without a proper XmlSuite (Krishnan Mahadevan) Fixed: GITHUB-1766: Testng generates a lot of temp folders like testngXmlPathInJar- (Andrey) Fixed: GITHUB-1759: core interfaces refactoring to support default methods (Sergey Korol) Fixed: GITHUB-1753: TestResult for an SKIP test lose attributes contributed by @BeforeMethod's or @AfterMethod's (Krishnan Mahadevan) Fixed: GITHUB-1756: ITest .getTestName() doesn't return the actual test name for skipped methods (Krishnan Mahadevan) Fixed: GITHUB-1602: beforeInvocation method don't get called for skipped tests (Krishnan Mahadevan) Fixed: GITHUB-549 and GITHUB-780: Introduce onlyForGroups attribute for @BeforeMethod and @AfterMethod (Sergei Tachenov) Fixed: GITHUB-1745: Support native injection for @Factory methods (Krishnan Mahadevan) Fixed: GITHUB-1746: Make InvokedMethodNameListener thread-safe, fixing occasional build failures (Sergei Tachenov) Fixed: GITHUB-1538: Dependent methods don't get invoked when a failed test method is retried via a RetryAnalyser (Krishnan Mahadevan) Fixed: GITHUB-426: firstTimeOnly ignored for threadPoolSize > 1 (Krishnan Mahadevan) Fixed: GITHUB-466: JUnitReportReporter always converts the system time into GMT (Krishnan Mahadevan) Fixed: GITHUB-1723: Standardize timezone reference in XML reports (Krishnan Mahadevan) Fixed: GITHUB-489: Incorrect timezone info in <test-method> element of testng-results.xml (Krishnan Mahadevan) Fixed: GITHUB-1735: IExecutionListener.onStart() running twice when used as annotation (Krishnan Mahadevan) New : Removed deprecated methods across TestNG (Krishnan Mahadevan) Fixed: GITHUB-1726: Allow user-defined method interceptors to have the last say in method re-ordering (Krishnan Mahadevan) Fixed: GITHUB-564: Support using @Optional on @Test method parameters to use null values (Krishnan Mahadevan) New : Upgrade TestNG to start needing at-least JDK8 (Krishnan Mahadevan) Fixed: GITHUB-1637: Remove dependency on a pre-created jar for running unit tests for JarFileUtils (Krishnan Mahadevan) Fixed: GITHUB-1710: Inefficient DynamicGraph causes hang with only 100 tests. (Steve Prentice) Fixed: GITHUB-1716: Potential NPE in jq.Main reporter (Krishnan Mahadevan) Fixed: GITHUB-1709: @Ignore doesn't work when used on child class, and parent has @Test methods (Krishnan Mahadevan) Fixed: GITHUB-1706: Retrying of methods fail when test method involves native injection (Krishnan Mahadevan) New : Removed deprecated attributes from annotations (Julien Herr) New : Support all JSR-223 compatible script engine Fixed: GITHUB-1827: TestNG does not throw an error when a test class does not have a proper constructor (Julien Herr & Krishnan Mahadevan) Fixed: Annotated default methods of indirectly implemented interfaces should still be called (Ilya Korobitsyn) New : GITHUB-2105: Include assertEquals(Map, Map) into Assert class

Fixed: GITHUB-1077: TestNG cannot handle load (Aheiss) Fixed: GITHUB-1081: group-by-instances with test dependencies causes instantiation of tests to exponentially slow (Aheiss) Fixed: GITHUB-1700: Test ignored if @BeforeMethod in base class fails for another test class (Krishnan Mahadevan) Fixed: GITHUB-1694: @BeforeGroups executed multiple times when tests run in parallel, once if not parallel (Krishnan Mahadevan) Fixed: GITHUB-1688: @Ignore annotation on base class doesn't ignore tests in child classes (Krishnan Mahadevan) Fixed: GITHUB-1687: NullPointerException is thrown when beanshell evaluates to null (Krishnan Mahadevan)

Fixed: GITHUB-1674: beanshell methodselector applied at suite level is ignored (Krishnan Mahadevan) Fixed: GITHUB-1668: "Invalid Method Selector" exception triggered when using suite level beanshell methodselectors (Krishnan Mahadevan) Fixed: GITHUB-1659: New line characters are removed from stack traces in testng-results.xml (Krishnan Mahadevan) Fixed: GITHUB-1503: Consider adding a -n (no execute) option (Krishnan Mahadevan) Fixed: GITHUB-1648: Depends on method is not respected on the sequential run on second test that extends same base testClass (Krishnan Mahadevan) Fixed: GITHUB-1636: Parallel test run is not working in 6.13.1 (Krishnan Mahadevan) New : GITHUB-1634: Make "-xmlpathinjar" support <suite-files> (Yehui Wang) New : GITHUB-1634: Make "-testnames" find tests from Multi-level parent-child suites (Yehui Wang) Fixed: GITHUB-1641: The time for test-method is 8 hours ahead of the the time for suite/class in testng-result xml file (Krishnan Mahadevan) Fixed: GITHUB-1649: @Test annotated methods cannot inject java.lang.reflect.Method (Krishnan Mahadevan) Fixed: GITHUB-1625: Null fields in parallel method tests (Krishnan Mahadevan) Fixed: GITHUB-1605: Research the usefulness of the JVM argument "experimental" (Krishnan Mahadevan) New : GITHUB-1631: data provider class name injection into Factory meta-data (Sergey Korol)

No functional changes. Released with newer version JCommander (1.72.0)

Fixed: GITHUB-1619: ConcurrentHashMap doesn't secure insertion order.(Yehui Wang) Fixed: GITHUB-1616: Test cases with priority and dependsOnGroups dependencies, execution order is chaos. (Yehui Wang) Fixed: GITHUB-1613: The constructor removed from TestRunner would stop Eclipse working.(Yehui Wang) Fixed: GITHUB-1600: Updated in afterInvocation() testResult.status is not used in willRetry condition (Krishnan Mahadevan) Fixed: GITHUB-1598: Injected types parameter and optional parameters cannot be used together. (Yehui Wang) Fixed: GITHUB-1594: Cannot filter by "testnames" when suite xml is a suite of suites (Krishnan Mahadevan) Fixed: GITHUB-1584: Can't run tests from IDEA (Krishnan Mahadevan) Fixed: GITHUB-1589: TestNGAntTask should be consistently using the Ant Log API for writing log messages (Krishnan Mahadevan) Fixed: GITHUB-1587: TestNG can not guarantee the ExecutionListener Instance as singleton(Yehui Wang) Fixed: GITHUB-217: exception in DataProvider doesn't fail test run (Krishnan Mahadevan) Fixed: GITHUB-987: Parameters threadCount and parallel doesn't work with maven (Krishnan Mahadevan) Fixed: GITHUB-1472: Optimize DynamicGraph.getUnfinishedNodes (Krishnan Mahadevan & Nathan Reynolds) Fixed: GITHUB-1566: Invalid XML characters in Params in testng-results.xml (Krishnan Mahadevan) Fixed: GITHUB-1554: @Parameters and parameter injection not wroking when used on the same method (Krishnan Mahadevan) Fixed: GITHUB-990: NullPointerExceptions after a superclass configuration method fails with configfailurepolicy="continue" (Krishnan Mahadevan) Fixed: GITHUB-461 : Annotate annotations with @Documented (Krishnan Mahadevan) Fixed: GITHUB-778 : XmlSuite toXml() does NOT add the suite time-out property (Krishnan Mahadevan) Fixed: GITHUB-1029: Issue with getting XmlTest from test method (Krishnan Mahadevan) Fixed: GITHUB-212: Enable support for providing a URI as suite file location (Krishnan Mahadevan) Fixed: GITHUB-161 : Provide a way to customize SAXParserFactory implementation (Krishnan Mahadevan) Fixed: GITHUB-1455: Configure XML output of XmlSuite (Krishnan Mahadevan) Fixed: GITHUB-1465: Failure policy CONTINUE handling is broken for tests that are skipped in @BeforeMethod method (Krishnan Mahadevan) Fixed: GITHUB-1533: Duplicate child suites get added when working with parent/child suite scenario (Krishnan Mahadevan) Fixed: GITHUB-949: dependsOnMethods with alwaysRun = true and inheritance fails to find method (Krishnan Mahadevan) Fixed: GITHUB-1519: Possibility to retry a test until it FAILED (Krishnan Mahadevan) Fixed: GITHUB-980: TestNG run inherited method twice (Krishnan Mahadevan) Fixed: GITHUB-1409: Regression: on expectedExceptionsMessageRegExp expected and actual messages are not printed (Krishnan Mahadevan) Fixed: GITHUB-1517: TestNG exits with a zero when there are configuration failures (Krishnan Mahadevan) New : GITHUB-1490: Add a listener for data provider interception (Krishnan Mahadevan) Fixed: GITHUB-1456: Remove/Warn support of constructor with String param (Krishnan Mahadevan) Fixed: GITHUB-1509: Improve error message when data provider returns a null value (Krishnan Mahadevan) Fixed: GITHUB-1507: TestNG runs all methods when filtering via <include> fails (Krishnan Mahadevan) Fixed: GITHUB-1493: Wrong exception msg when timeout on test (Krishnan Mahadevan) Fixed: GITHUB-328: Attempt to fix unnecessary execution of @Factory-ctors (@beverage & Julien Herr) Fixed: GITHUB-1384: Huge performance issue between 6.5.2 and 6.11 (Denis Bazhenov) New: Remove Serializable (Julien Herr) Fixed: GITHUB-1496: If method contains "$", run only one method, all methods will be run (@JF-Rabbit & Julien Herr) Fixed: GITHUB-1220: Recognize annotations on default methods in implemented interface New: GITHUB-861: Add @Ignore annotation which disables all tests in a class or a package (Julien Herr)

2017/02/27 Fixed: GITHUB-1351: FailurePolicy failing with YAML (Steven Zaluk & Julien Herr) New: The name of all TestNG threads follow "TestNG-<thread type>-<number>" pattern (Julien Herr) Fixed: GITHUB-1339: Alter ClassHelper to use Maps instead of Lists for extracting methods (Krishnan Mahadevan) Fixed: GITHUB-1338: @BeforeGroups method is run on a wrong instance (Pavel Vetokhin & Julien Herr) Fixed: GITHUB-1332: Make EmailableReport name configurable (Krishnan Mahadevan) Fixed: GITHUB-1297: testng-failed.xml includes setup and tearDown (before and after annotations) of passed tests (Krishnan Mahadevan) Fixed: GITHUB-1319: ITestResult#getInstance() returns null in IConfigurationListener implementation (Krishnan Mahadevan) New: Skipped methods are not supposed to be executed (Julien Herr) Fixed: GITHUB-1197: Ability to dynamically set the status and exception of a test via ITestResult (Anthony Nguyen) Fixed: GITHUB-1302: When 'parallel' is set to 'classes', ConcurrentModificationException can be thrown(Jianhua Li) Fixed: GITHUB-772: Severe thread contention while running large test with parallel methods (Shaburov Oleg) Fixed: GITHUB-1307: TestNGException when using an anonymous class in Factory (Mike Cowan) Fixed: GITHUB-1298: ITestResult injection is failing in BeforeMethod method (Krishnan Mahadevan) Fixed: GITHUB-1293: Beanshell based execution does not work any more (Krishnan Mahadevan) Fixed: GITHUB-1262: Testcases out of order in XML file in junitreport folder when using testng (Krishnan Mahadevan) Fixed: GITHUB-116 : BaseTestMethod does respect general contract of Comparable (Testo Nakada) Fixed: GITHUB-1265: JUnit Reporter includes redundant ignored methods (Krishnan Mahadevan) Fixed: GITHUB-1266: JUnit Reporter produces a wrong number of total test methods (Krishnan Mahadevan) Fixed: GITHUB-1257: Group parameter not applying on included <suite-files> Fixed: GITHUB-1284: Listeners on the child suites are not applied (Vimalraj Selvam) New: GITHUB-1313: Add Java9 as test environment on Travis (Julien Herr) Fixed: GITHUB-1296: Configuration listeners run multiple times (@mikimrozowski & Julien Herr) Fixed: GITHUB-1300: Add deep assertions to assert on array values (array reference by default) (Jordan Zimmerman & Julien Herr)

2016/11/28 Fixed: GITHUB-551: Failed configuration method always has 0 execution time (dr29bart) Fixed: GITHUB-1250: Testng-failed.xml is getting test level parameters into suite level parameters (Krishnan Mahadevan) Fixed: GITHUB-1046: Provide a mechanism to customize a test method name for reporting (Krishnan Mahadevan) Fixed: GITHUB-1211: Include disabled/ ignored test methods in JUnit reports (Krishnan Mahadevan) Fixed: GITHUB-1213: Include "ignored" test count in testng-results.xml (Krishnan Mahadevan) Fixed: GITHUB-674: Enrich Test method skips due to configuration failures with throwable data (Krishnan Mahadevan) Fixed: GITHUB-1240: Enrich the test results showing mechanism in Travis CI (Krishnan Mahadevan) Fixed: GITHUB-1232: Prevent TestNG from adding duplicate instances of the same listener (Krishnan Mahadevan) Fixed: GITHUB-1170: Fixing the test DataProviderTest.shouldNotThrowConcurrentModification (Krishnan Mahadevan) Fixed: GITHUB-1231: Make IExecutionListener implementation be the last reporter call before JVM exit(Krishnan Mahadevan) Fixed: GITHUB-1227: Prevent multiple instances of same Reporter from being injected into TestNG (Krishnan Mahadevan) Fixed: GITHUB-1165: Better message to user when param injection is not good (Krishnan Mahadevan) Fixed: GITHUB-1228: Control stacktrace levels in XmlReports via a JVM configuration (Krishnan Mahadevan) Fixed: GITHUB-1203: Add flush to BufferedWriter; fixes incomplete XML reports (Nathan Bruning) Fixed: GITHUB-1181: Fix MethodMatcherException: Data provider mismatch (Krishnan Mahadevan) Fixed: GITHUB-1107: TestNG does not report/print/log throwables in data providers (Krishnan Mahadevan) Fixed: GITHUB-1186: NullPointerException in JUnit reporter when used with Spock (Ian Robertson & Julien Herr) Fixed: GITHUB-1180: NullPointerException on getting excluded/included groups (Krishnan Mahadevan) Fixed: GITHUB-1064: Incorrect logging of parallel mode of a test Fixed: GITHUB-1178: Halt execution when invalid testname is provided. (Krishnan Mahadevan) Fixed: GITHUB-1139: DataProvider could support Object[] as a valid return type (Julien Herr) Fixed: GITHUB-1182: Cannot run multiple @Factory-annotated methods in the same class (Ian Donovan & Julien Herr) New: Hierarchy on order features (from less important to more important): groupByInstance, preserveOrder, priority, dependsOnGroups, dependsOnMethods Fixed: GITHUB-1156: test execution dependant upon class name order and fails with TestNGException: No free nodes found (@t-weil & Julien Herr) Fixed: GITHUB-1221: ConcurrentModificationException in TextReporter (Nick Tan) Fixed: testng-eclipse/issues/298: IConfigurationListener was not loaded when running Test in Eclipse Plugin (@jmcgrail & Nick Tan) New: Resources for test reports (header, images) now live in a sub directory org/testng to remove conflicts with other similar files in the class loader.

2016/09/23

2016/09/23

2016/09/22

2016/09/22

2013/03/30

Added: Descriptions in the HTML reports Added: Various improvements to EmailableReporter (Abraham Lin) Added: Allow injection of java.lang.reflect.Constructor and org.testng.ITestNGMethod into DataProvide (Vladislav Rassokhin) Fixed: Assertions in the Assertions class were not failing properly. Fixed: GITHUB-337: ConfigurationMethod#m_instance set to Boolean.FALSE due to incorrect constructor call in clone() + auto-boxing (davidely) Fixed: Fix NPE for dependency methods/groups (Krishnan Mahadevan) Fixed: preserve-order bug (found by VladSarrokhin). Fixed: GITHUB-300: OutOfMemoryException from reporters when there are a lot of tests Fixed: GITHUB-137: Main parameters with a default value should be overridden if a main parameter is specified Fixed: GITHUB-107: Allow enum values without converting them to uppercase. Fixed: @Guice with no modules specified is now supported Fixed: Reporter.log() invoked from listeners were being discarded

Eclipse: Added: Predefined listeners (Tim Wu) Fixed: Compare dialog

===========================================================================

2012/07/15

Added: Big performance improvement when generating the reports (Frank Pavageau) Added: <dependencies> allows you to specify group dependencies in testng.xml Added: Blow up early if trying to include/exclude an unknown method Added: <parameters> can now be specified under <include> (Storm Qi) Added: GITHUB-243: Add Reporter Output per Test in XMLReporter (dunse) Fixed: Better HTML escaping of the stack traces Fixed: The failed assertions now use [] as delimiters instead of <> (better for the HTML reports) Fixed: GITHUB-237: Wrong time format in XML reporter Fixed: Threads were started sequentially instead of being interleaved Fixed: dataProvider(parallel = true) was not killing its threads properly Fixed: XmlSuite#toXml wasn't outputting the <groups> tag correctly Fixed: testng-failed.xml was not carrying over the parameters from the original testng.xml Fixed: BeforeClass failing in parent failed to skip methods in sub classes Fixed: Better error message if <suite name=""> is missing Fixed: GITHUB-221: Honor excludeGroups on testng tests when run in mixed mode (criccio) Fixed: dependsOnGroups = {regexp} wasn't working properly (Alistair Ward) Fixed: GITHUB-205: white-space was spelled whitespace in testng.css (carlin-scott)

Eclipse: Fixed: Environment is not transferred when rerunning failed tests. Fixed: Rerunning failed tests will preserve the environment of the original launch

===========================================================================

04/10/2012

Added: <suite allow-return-values="true"> (and in <test> as well) Added: data-provider attribute to testng-results.xml Added: Reporter display the results in the same order as test methods (Libor Zoubek) Added: Support for running JUnit 4 tests (Lukas Jungmann) Added: Ability to auto-detect JUnit tests ('-mixed' mode) (Lukas Jungmann) Added: Support for ResourceCollections in an Ant tasks (requires Ant >= 1.7.0) (Lukas Jungmann) Fixed: GITHUB-198: JUnitReportsReporter use commas in certain locales, which JUnitReports doesn't like Fixed: GITHUB-173: Dependent methods executed out-of-order if method names match across classes (jjedMoriAnktah) Fixed: ThreadLocal<ITestResult> leak (aslakknutsen) Fixed: In the HTML reports, only show the first 100 characters of the parameters Fixed: SkippedException are considered as real exception with @Test(expectedExceptions)

Eclipse: Fixed: Java constants are properly resolved if they are used as group names (susanin) Fixed: @Test(groups = Foo.CONSTANT) (susanin) Fixed: Failed tests with allow-return-values="true" were not rerun Added: <suite allow-return-values="true"> (and in <test> as well)

===========================================================================

6.4: 02/15/2012

Added: @DataProvider(indices) to return specific indices of a data provider Added: New HTML reports Added: configfailurepolicy=continue with DataProviders (toddq) Added: ITestResult#getTestContext (bpedman) Fixed: invocationCount > 1 + timeOut wasn't timing out properly Fixed: When running TestNG programmatically, child xml suites are not run (when added using setSuiteFIles()) (Gaurav Gupta) Fixed: GITHUB-145: Excessive test method execution (githubCast) Fixed: GITHUB-149: reversed arguments in failAssertEqualsNoOrder(). Fixed: EmailableReporter: methods are now really sorted chronologically.

Eclipse:

Added: You can now add the testng.jar sources as a library (Nick Tan) Added: Upgraded the plug-in to 3.4+ (Nick Tan) Added: dependsOnGroups now fully supported Fixed: @Parameters now works with both ("foo") and ({"foo"}) (davekerber)

===========================================================================

10/22/2011

Added: New system property: dataproviderthreadcount (Bill Ross) Fixed: Configuration methods were reported incorrectly in listeners. Fixed: Was creating too many listeners (Jacek Pulut) Fixed: IAnnotationTransformer2 beforeTest/afterTest booleans were not being set Fixed: GITHUB-92: @BeforeTest method in a super class will be called multiple time when alwayRun = true (Bubuntux) Fixed: GITHUB-111: @AfterClass on base classes run once too many (lrivera) Fixed: GITHUB-107: Displaying 0 tests run if a listener modifies the parameters of the suite

===========================================================================

10/17/2011

Added: "description" attribute on <include>, made available on ITestNGMethod#getDescription Added: RemoteTestNG waits infinitely for a connection (Aleksey Kabanov) Fixed: A method that's both a test and a factory would not invoke its data provider Fixed: @AfterClass was not called if one of the methods was not enabled (Aleksey Kabanov) Fixed: Groovy access bug Fixed: The XML parser doesn't recognize parallel="instances" Fixed: NPE when using inner classes Fixed: GITHUB-90: @AfterClass not being run when the class contains included and not included methods Fixed: @AfterClass not being run in some subclassing situations

Eclipse:

Fixed: Verbose levels specified in suites not respected Fixed: Variable substitution in VM arguments is not working properly (svenhoff)

===========================================================================

08/18/2011

Added: xmlpathinjar to the TestNG ant task Added: TestNG can now invoke package protected constructors Added: Injectors created by the @Guice annotation are now shared at the <test> level Added: IConfigurationListener is now a public listener, along with a new one: IConfigurationListener2 Added: When a method fails, only dependents of the same instance will be skipped Added: parallel=instances for factory instance parallel runs Added: @Factory(enabled) Fixed: JUnitReports reports now report the cumulated time @{Before,After}Method+@Test for each test method Fixed: JUnitReports reports have the name of the <test> instead of that of the first class Fixed: Using preserve-order with a factory that creates instances of a different class causes NPE Fixed: GITHUB-74: Bad ordering of test methods when using a @Factory constructor with dataProvider Fixed: Changing the test result from success to failure in a listener would still count the test as a success Fixed: ServiceLoader wasn't resolving correctly if no service loader classloader was specified Fixed: Better ordering with mixed priorities and dependencies Fixed: Improved detection of graph cycles in parallel runs Fixed: @BeforeTest was invoked multiple times if a factory is used Fixed: GITHUB-57: Allow usage of package protected constructor of test classes Fixed: Injecting both Object[] and Method in @BeforeMethod didn't always work Fixed: testng-results.xml now lists the results chronologically Fixed: @Listeners specified on a base class will only be run once per listener class (dbriones) Fixed: -groups and -excludegroups were no longer overriding testng.xml

Eclipse:

Added: Each data provider method now has a separate node entry in the TestNG view Fixed: Nodes in error would sometimes remain green Fixed: The TestNG context menu no longer appears where it shouldn't

===========================================================================

7/5/2011

Fixed: https://github.com/cbeust/testng/issues/56 testng-results.xml was reporting the instance name instead of the method name Fixed: NPE when using preserve-order and factories. Fixed: Depending on a skipped method would not cause a method to be skipped

===========================================================================