I am wary of adding new syntax, but I have no particular objection. I wouldn't mind seeing this as a global option before it becomes a piece of syntax, but I'm sufficiently ambivalent that I won't block shipping on that.
Shadowing a builtin
first arg is superfluous (default is 0)
Can use a generator comp instead of list (not that it matters much, but it looks nicer)
Support sharding of descendant (::) specs.
Review Request #3363 — Created Jan. 21, 2016 and discarded
|patricklaw, stuhood, zundel|
NOTE: Discarded in favor of https://rbcommons.com/s/twitter/r/3560/.
This new syntax looks like path/to/root::shard/nshards.
For example, foo/bar::4/7 will take only those targets in foo/bar::
whose hash modulo 7 is equal to 4.
This is useful primarily for splitting up tests and running them in
multiple concurrent disjoint batches. E.g., N CI workers could each run:
./pants test tests/python/pants_test::i/N --tag=integration
with a different value of 0<=i<N for each worker.
This could potentially supersede the existing support for sharding tests.
The advantages of this approach:
- It works not just for test running but for all tasks, where that might be useful.
- The implementation is simple, and in one place in pants code, rather than being
re-implemented inside each test runner task for each language, possibly involving
complicated interactions with the underlying test running framework.
- It provides more random distribution, as the existing test sharding simply mods
the test's index in an enumeration influenced by the ordering of, and within,
source files, which is hardly random.
Potential disadvantages of this approach:
- Sharding is at the target level, not the individual test level.
- This requires use of the descendant spec (
Overall this should really simplify the test runners, and pants's own ci.sh
script, so I think it would be a win to use it instead.
CI passes: https://travis-ci.org/pantsbuild/pants/builds/103991853
IMO, this doesn't really make sense as syntax.