Flush out some bugs with the 'parallel methods' running in the legacy runner.
Review Request #3922 — Created May 23, 2016 and submitted
|benjyw, gmalmquist, jsirois, stuhood|
A few bugs were brought to the light after adding -default-concurrency PARALLEL_METHODS around running all tests.
- Some tests were not properly being ignored and caused extra XML output.
- The list of sharded tests was not deterministic. This is because the method list returned by Class.getMethods() does not return methods in a deterministic order.
These problems would have only been seen by someone using the experimental PARALLEL_METHODS mode of running the test runner so shouldn't have much impact for existing users.
- Use the parameterized test runner to run with -default-concurrency PARALLEL_METHODS.
- Extract TestMethod to a standalone class an34d add tests.
- Make sure list of TestMethods created in a deterministic order.
- Make sure that we only consider testable, runnable classes before making requests for each method.
- Consolidate functions that analyze test classes into the Util class.
- Add more tests for the Util class.
CI is green at https://travis-ci.org/pantsbuild/pants/builds/132169647