Support for scanning addresses, implement `list`.

Review Request #2952 - Created Oct. 9, 2015 and submitted

Information
John Sirois
pants
jsirois/engine/graph/compatible_list_command
2308, 2312, 2349
2945, 2947, 2944
3ed0b70...
Reviewers
pants-reviews
benjyw, ity, nhoward_tw, patricklaw, stuhood, zundel
This also introduces a `legacy_python_callbacks_parser` that can work on
most legacy BuildFileAliases with proper configuration.  This is used to
create a self-contained list command that can be used in legacy BUILD
trees (possibly with some `--spec-excludes` to look past trick macros)
to start perf testing aspects of this experimental parsing system
against the 'legacy' (current) system.

 src/python/pants/engine/exp/legacy/BUILD                                   | 26 +++++++++++++++
 src/python/pants/engine/exp/legacy/__init__.py                             |  0
 src/python/pants/engine/exp/legacy/commands.py                             | 58 +++++++++++++++++++++++++++++++++
 src/python/pants/engine/exp/legacy/parsers.py                              | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++
 src/python/pants/engine/exp/mapper.py                                      | 40 ++++++++++++++++++++---
 src/python/pants/engine/exp/objects.py                                     | 12 +++++++
 tests/python/pants_test/engine/exp/examples/mapper_test/a/d/d.BUILD.json   |  4 +++
 tests/python/pants_test/engine/exp/examples/mapper_test/a/d/e/e.BUILD.json |  9 ++++++
 tests/python/pants_test/engine/exp/examples/mapper_test/root.BUILD.json    |  4 +++
 tests/python/pants_test/engine/exp/test_mapper.py                          | 54 +++++++++++++++++++------------
 10 files changed, 268 insertions(+), 26 deletions(-)

Manually compared the new list command timings with ./pants list, although
for a more fair comparison, ran both from unzipped pexes, ie:

$ ./pants binary \
  src/python/pants/engine/exp/legacy:list \
  src/python/pants/bin:pants_local_binary
$ mkdir dist/pants dist/list && \
  unzip -qd dist/pants dist/pants_local_binary.pex && \
  unzip -qd dist/list dist/list.pex

pants list:

$ time python2 dist/pants list --spec-excludes=contrib/go/examples list
...
real    0m1.807s
user    0m1.393s
sys 0m0.137s

new list:

$ time python2 dist/list --pythonpath=$PWD/src/python --spec-excludes=contrib/go/examples
...
real    0m1.103s
user    0m0.983s
sys 0m0.083s

These times are representative over many runs with low variance.

It would be great if reviewers could run a similar test on their large trees and report back anecdotes good or bad.

CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/84431862

Eric Ayers
John Sirois
John Sirois
Review request changed

Status: Closed (submitted)

Loading...