RandomizedTesting, Change Log ============================= For an up-to-date CHANGES file see https://github.com/randomizedtesting/randomizedtesting/blob/master/CHANGES ======================= randomizedtesting 2.8.1 ==================== Bug Fixes: * GH-307: Use monotonic clock instead of wall clock for detecting deadlines as VMs and operating systems may shift wall clock resulting in unpredictable behavior. ======================= randomizedtesting 2.8.0 ==================== Other: * GH-303: Messages from ThreadLeakControl.checkThreadLeaks can be inconsistent due to mismatched use of thread data. * GH-304: Replace master/slave with main/forkedjvm terminology. This change may result in backward incompatibilities if you used low-level event bus subscription methods or JSON reports. Grep and replace 'slave' with 'forkedJvm' and 'master' with 'main' seed. ======================= randomizedtesting 2.7.9 ==================== Other: * Update ASM to 9.2 to support JDK 17 * Update JUnit dependency to 4.13.1. * Update Ant dependency to 1.9.15. ======================= randomizedtesting 2.7.8 ==================== Improvements * GH-289: Bump asm to 8.0.1. * PR-287: Bump dom4j from 2.1.1 to 2.1.3. ======================= randomizedtesting 2.7.7 ==================== Bug Fixes * GH-283: RandomizedRunner throws `NoTestsRemainException` from the constructor when method filter is supplied and no methods match (Mark Vieira). ======================= randomizedtesting 2.7.6 ==================== Improvements * Use thread join internally to wait for forked timeouting test thread instead of a hardcoded sleep (dumb!). ======================= randomizedtesting 2.7.5 ==================== Improvements * Switch to simple-xml-safe. * throw NoTestsRemainException if Filterable.filter removes all tests. ======================= randomizedtesting 2.7.4 ==================== Bug Fixes * GH-280: Potential race condition in global temp folder generation. ======================= randomizedtesting 2.7.3 ==================== Bug Fixes * GH-276: Annotations on overriden methods are not picked up (annotations on overrides of @Test methods are not picked up). ======================= randomizedtesting 2.7.2 ==================== Bug Fixes * GH-275: Timeouts can deadlock on trying to interrupt leaked threads. Improvements * GH-274: PrintEventListener should emit display name if method name is not available in testIgnored callback. ======================= randomizedtesting 2.7.1 ==================== Bug Fixes * GH-271: Methods with ( or ) in them can't be selected directly using glob filters. ======================= randomizedtesting 2.7.0 ==================== Bug Fixes * GH-269: 2.6.x runners and junit 4.12 show much noise for a single test method in IntellJ Idea (thanks Vladimir Dolzhenko). Changes in Backwards Compatibility * GH-270: Upgrade to support JUnit 4.12. This also means that Hamcrest is required (as JUnit 4.12 uses it). ======================= randomizedtesting 2.6.4 ==================== Bug Fixes * GH-247: Allow java.io.tmpdir to be placed under cwd with isolated directories and nonEmptyWorkDirAction=wipe. * GH-255: Invalid event nesting on security logging crashes the runner. * GH-266: QUIT event may not be read from the event log on slow machines. * GH-267: Project does not build/ test on Java 11. Improvements * An additional property 'junit4.tempDir' is passed to forked JVMs to allow easier configuration of security policy files. This folder must have read/ write file policy. * Added a test case that runs with a security manager enabled. * GH-253: IntelliJ only recognizes proper test methods for re-runs when descriptions have " [...]" instead of " {...}. This fix uses square brackets on IntelliJ runner. Changes in Backwards Compatibility * GH-265: Deprecate RandomizedTest.randomInt(v) and RandomizedTest.randomLong(v). ======================= randomizedtesting 2.6.3 ==================== Bug Fixes * GH-263: event bus dispatch error (covariant incompatibility due to jdk 9 compilation to java 8 bytecode). ======================= randomizedtesting 2.6.2 ==================== New Features * GH-261: Ant and Maven runner can specify external JUnit 4 RunListeners. (Steven Zaluk) ======================= randomizedtesting 2.6.1 ==================== Bug Fixes * GH-260: maven plugin should use dom4j 2.0.2 (avoids illegal access warnings under Java 9 and later). ======================= randomizedtesting 2.6.0 ==================== Changes in Backwards Compatibility * GH-259: Java 1.8 is now the minimum supported JRE. * GH-258: add support for parsing class files compiled into Java 9 or Java 10 bytecode. ======================= randomizedtesting 2.5.3 ==================== Bug Fixes * GH-251: Fixes test case filtering in IDEs (IntelliJ, Eclipse). The tests are now filtered out eagerly from the returned suite's Description and don't show in the user interface at all. This also applies to -Dtests.method=... patterns. (Mike Sokolov via Dawid Weiss) ======================= randomizedtesting 2.5.2 ==================== Bug Fixes * GH-250: Non-bootstrapped ForkedJvmInfo causes NullPointerExceptions (can happen on slow machines with low heartbeat setting). (Doug Turnbull via Dawid Weiss). ======================= randomizedtesting 2.5.1 ==================== Changes in Backwards Compatibility * GH-248: Cleaning up some confusion on RandomizedTest.randomAscii* and RandomStrings.randomAscii* methods. These methods generated random strings based on letters from the Unicode basic latin block (C0). All these methods are now deprecated and will be removed in favor of more explicit methods that tell exactly (?) what kind of letters are involved (Ryan Ernst, Dawid Weiss). For details, see: RandomStrings.randomAsciiLettersOfLength(..) RandomStrings.randomAsciiLettersOfLengthBetween(..) RandomStrings.randomAsciiAlphanumOfLength(..) RandomStrings.randomAsciiAlphanumOfLengthBetween(..) ======================= randomizedtesting 2.5.0 ==================== Changes in Backwards Compatibility * GH-243: Switches from File to NIO (Path) APIs in a few places may result in slightly different runner's behavior (on exceptional conditions). * GH-244: the new onNonEmptyWorkDirectory attribute's default value is 'fail' which may break existing builds that have garbage in work directories of forked JVMs. Change explicitly to any of the alternatives ('ignore' or 'wipe') or fix your build to wipe the temporary folders yourself. * GH-246: RequireAssertionsRule enforcement optional (consistent with "-Dtests.asserts"); explicit target class is also required. (Dawid Weiss) New Features * GH-244: A new attribute "onNonEmptyWorkDirectory" controls the action to take on non-empty initial work directory (for isolated JVMs). (Dawid Weiss) Bug Fixes Other Changes * GH-246: RequireAssertionsRule enforcement optional (consistent with "-Dtests.asserts"); explicit target class is also required. (Dawid Weiss) * GH-243: Switches from File to NIO (Path) APIs in a few places in the ANT runner. Improved messages concerning non-empty temporary JVM folder (both initially and at the end of the tests' run). (Dawid Weiss) * GH-245: Integrate Forbidden-APIs check and fix follow-up issues. (Dawid Weiss) ======================= randomizedtesting 2.4.1 ==================== New Features * GH-240: Add RandomPicks.randomFrom on primitive arrays. (Dawid Weiss) Bug Fixes * GH-242: RandomizedTest.nextTempName uses locale-sensitive String.format. (Dawid Weiss) ======================= randomizedtesting 2.4.0 ==================== Changes in Backwards Compatibility * GH-237: Methods previously found in RandomInts have been moved to RandomNumbers. (Dawid Weiss) * GH-234: Run strict validation (thread-ownership, lifecycle) in AssertingRandom only with assertions enabled. This change means that running without -ea will from now on ignore the fact that random context's Random instance is shared where it shouldn't be. On the other hand, any additional memory barriers can be now avoided, in particular when used in conjunction with @TestContextRandomSupplier (see GH-235 below). To enable consistent checking, enable assertions selectively (for com.carrotsearch.randomizedtesting package only). (Dawid Weiss) * GH-229: The default implementation of the Random instance returned from the randomization context has been changed to Xoroshiro128P, details here: http://xoroshiro.di.unimi.it/ The returned Random isn't synchronized and should yield much better pseudo-random sequences than the default Java's Random. (Dawid Weiss) New Features * GH-238: Add BiasedNumbers (random selection with bias towards "evil" numbers, in particular floats) as an alternative to RandomNumbers. (Dawid Weiss) * GH-237: Rename RandomInts to RandomNumbers and add methods operating on longs. (Dawid Weiss). * GH-235: A new class annotation was added: @TestContextRandomSupplier allows declaring a custom class (RandomSupplier) that supplies a Random for the given seed. This supplier can return any Random instance for the given initial seed (secure random, faster-than-jdk random, non-random random...). (Dawid Weiss) Bug Fixes * GH-233: some adjustments and refactorings in an attempt to avoid deadlocks on suite timeouts. Very hard to reproduce (i.e., impossible) and some very odd stack traces too. (Dawid Weiss) Other Changes * GH-232: Rework reporting of ignored, assumption-ignored, filtering-expression ignored tests. (Dawid Weiss) * GH-218: There are some internal changes in handling @Nightly annotation. The support for @Nightly is now provided via GroupEvaluator (and this class is now public, with methods allowing inspection of whether a test would be filtered out or not). (Dawid Weiss) ======================= randomizedtesting 2.3.4 ==================== Bug Fixes * GH-227: Static leak detector should print the "original" field in the test class that lead to ram usage tester failure (PR #227, Uwe Schindler) ======================= randomizedtesting 2.3.3 ==================== Bug Fixes * GH-225: Maven mojo's additionalClasspathElements option only adds empty strings to classpath. (@Anaros via Dawid Weiss). ======================= randomizedtesting 2.3.2 ==================== Changes in Backwards Compatibility * GH-222: Make unexpected JVM output available to listeners. This issue makes JVM output available to ANT JUnit4's listeners so that they can log it properly. Previously the JVM output was copied to System.out which could be hard to work with. See TextReport.onJvmOutput for an example. The default value of 'jvmOutputAction' has been changed from 'pipe, warn' to 'listeners, warn' so that listeners get notified about JVM output instead of it being copied directly to sysout. (Dawid Weiss, Ryan Ernst). Other Changes * GH-220: Add an indication of the number of failed suites so far to text report. (Dawid Weiss, Chris Hostetter). * GH-224: Remove excess use of Guava's collections. (Dawid Weiss) * GH-221: Upgrade to Guava 19.0-rc3. (Dawid Weiss, Ryan Ernst) Bug Fixes * PR #223: Access private fields within doPrivileged (Robert Muir) ======================= randomizedtesting 2.3.1 ==================== New Features * GH-219: Add randomBytes generator utility. (Dawid Weiss) * GH-190: RandomizedContext should provide the current test case method (via RandomizedContext.current().getTargetMethod()). (Dawid Weiss) * GH-217: Permit running tests on the same instance of the suite class via TestCaseInstanceProvider annotation. See Test018TestCaseInstanceProviders example. (Dawid Weiss). ======================= randomizedtesting 2.3.0 ==================== New Features * GH-216: Custom test case argument formatters in parameter factories. See ParametersFactory.argumentFormatting and Test007ParameterizedTests example. (Dawid Weiss). * GH-194: Flexible test case ordering via the new annotation (TestCaseOrdering). (Dawid Weiss) Bug Fixes * GH-215: In certain cases test case names could be duplicated (be identical), which could confuse IDEs. (Dawid Weiss) ======================= randomizedtesting 2.2.0 ==================== Changes in Backwards Compatibility * PR #178: Move to Java 1.7 and use the java NIO 2 API for temp files. (Simon Willnauer) * GH-184: RandomizedTest should not extend Assert. This permits easier use of other frameworks (like assertj) via static imports without naming conflicts. (Olivier Bourgain, Dawid Weiss) New Features * GH-191: "Tests with failures" should repeat the main seed at the end of the report. (Dawid Weiss) Bug Fixes * GH-192: Reworked json streaming to get rid of gson and permit dumping JVM-related diagnostics (security manager policy dumps, etc.). (Dawid Weiss) * GH-206: Potential NPE in checking getDefaultUncaughtExceptionHandler. (Dawid Weiss) Other Changes * GH-210: Make RR compilable/ testable under Java 1.9. (Dawid Weiss) * GH-209: Remove dependency on commons-io. (Dawid Weiss) * GH-204: modernize and clean up project layout. (Dawid Weiss) ======================= randomizedtesting 2.1.17 ============================ Bug Fixes * GH-201: Tests still running (and zombified) after suite timeout occurred. (Dawid Weiss) Other Changes * GH-205: Report test seed in halt on failure message. (Dawid Weiss) * GH-203: report the number of completely ignored suites in the summary. (Dawid Weiss) * PR #202: support running with security manager and limit the required permissions. (Robert Muir) ======================= randomizedtesting (pre 2.2.0) ======================= No separate changelog was maintained. See git commit history or issues at GitHub.