Add fail-slow handling to `Engine`.

Review Request #3040 — Created Oct. 26, 2015 and submitted

jsirois
pants
jsirois/issues/2452
915, 2452, 2458
e0d103c...
pants-reviews
benjyw, ity, nhoward_tw, patricklaw, stuhood
This allows for an engine execution to be in `fail_slow` mode, in which
case as much of the execution graph as can be executed is and failure
paths are recorded using a `FailedToProduce` product type.

A test is added with a failing path and a good path to exercise this
handling in both the serial and multiprocess engines.

 src/python/pants/engine/exp/BUILD                                                               |   1 +
 src/python/pants/engine/exp/engine.py                                                           | 339 +++++++++++++++++++++++++++++++++++++++++++-------
 src/python/pants/engine/exp/examples/BUILD                                                      |   1 +
 src/python/pants/engine/exp/examples/planners.py                                                |  12 +-
 src/python/pants/engine/exp/examples/visualizer.py                                              |  18 +--
 src/python/pants/engine/exp/scheduler.py                                                        |  45 ++++---
 src/python/pants/engine/exp/targets.py                                                          |  34 +++--
 tests/python/pants_test/engine/exp/examples/scheduler_inputs/src/java/codegen/selector/BLD.json |  10 ++
 tests/python/pants_test/engine/exp/examples/scheduler_inputs/src/thrift/BLD.json                |   7 ++
 tests/python/pants_test/engine/exp/test_engine.py                                               |  43 ++++++-
 tests/python/pants_test/engine/exp/test_scheduler.py                                            |  23 ++--
 11 files changed, 439 insertions(+), 94 deletions(-)
CI went green here:
  https://travis-ci.org/pantsbuild/pants/builds/87523475
  • 0
  • 0
  • 1
  • 0
  • 1
Description From Last Updated
JS
BE
  1. Psyched to see this all come together.

  2. src/python/pants/engine/exp/engine.py (Diff revision 1)
     
     

    Docstring?

  3. 
      
JS
JS
JS
JS
  1. Thanks Benjy - submitted @ https://github.com/pantsbuild/pants/commit/06e62bd1f00e130d76ada31b932062c5531cd717
  2. 
      
JS
Review request changed

Status: Closed (submitted)

ST
  1. Rather than being a global scheduler property, is this something that Planners should be driving on a promise-by-promise basis?

    1. Not sure.  Examples of when we'd want to cede control to a planner to potentially spin alot of cores would be useful to think about this.  In particular cases where it wouldn't make sense for one planner but it would for others.
  2. 
      
Loading...