Flush out some bugs with the 'parallel methods' running in the legacy runner.

Review Request #3922 — Created May 23, 2016 and submitted

Eric Ayers
benjyw, gmalmquist, jsirois, stuhood

A few bugs were brought to the light after adding -default-concurrency PARALLEL_METHODS around running all tests.
- Some tests were not properly being ignored and caused extra XML output.
- The list of sharded tests was not deterministic. This is because the method list returned by Class.getMethods() does not return methods in a deterministic order.

These problems would have only been seen by someone using the experimental PARALLEL_METHODS mode of running the test runner so shouldn't have much impact for existing users.

- Use the parameterized test runner to run with -default-concurrency PARALLEL_METHODS.
- Extract TestMethod to a standalone class an34d add tests.
- Make sure list of TestMethods created in a deterministic order.
- Make sure that we only consider testable, runnable classes before making requests for each method.
- Consolidate functions that analyze test classes into the Util class.
- Add more tests for the Util class.

CI is green at https://travis-ci.org/pantsbuild/pants/builds/132169647

Benjy Weinberger
  1. Ship It!
Garrett Malmquist
  1. Ship It!
Eric Ayers
Review request changed

Status: Closed (submitted)

Change Summary:

Thanks Benjy & Garrett. Commit defdfaadce7aa4a9e0e75dc62284467a522bd2b7