Prepare for a global --shard flag.
Review Request #3560 - Created March 10, 2016 and submitted
|patricklaw, stuhood, zundel|
- Refactor shard spec parsing and application to a common utility.
- Rename test.pytest's --shard to --test-shard (and deprecate the
old name) so it won't collide with the global name, and for
uniformity with test.junit's similar flag.
Note that the name --test-shard is slightly awkward, because
'test' is the name of a scope, so using the short form in the
'test.pytest' context won't work. But I stuck with that name
for uniformity with test.junit, and because this is a wider
problem we need to think about anyway.
When global sharding is implemented, the test tasks' sharding will
subshard on top of that. This is useful because sharding the input
targets may not be fine-grained enough (some repos have large numbers
of tests covered by a single target). However global sharding
may be good enough in many cases, and will prevent us from having
to implement sharding over and over again in every task that might
Note: Replaces https://rbcommons.com/s/twitter/r/3363/, which is now
CI passes: https://travis-ci.org/pantsbuild/pants/builds/115140711
I think this is a great alternative to the previous patch.
I was surprised there was no code to refactor from junit_run but it looks like all that shard parsing logic is in java (ConsolerRunnerImpl.java)
specusually refers to the string representation of a target and I was a bit confused by the name. Here it means a string that matches [\d]+/[\d]+.
Could you document the type and exepcted format of 'spec' in pydoc here? Maybe its obvious to other folks, but I was looking around for a type that defined
partition()but then it dawned on me that this was a method on string.
Adress code review comments.
Revision 2 (+178 -66)
Status: Closed (submitted)