Add fail-slow handling to `Engine`.

Review Request #3040 - Created Oct. 26, 2015 and submitted

Information
John Sirois
pants
jsirois/issues/2452
915, 2452, 2458
e0d103c...
Reviewers
pants-reviews
benjyw, ity, nhoward_tw, patricklaw, stuhood
This allows for an engine execution to be in `fail_slow` mode, in which
case as much of the execution graph as can be executed is and failure
paths are recorded using a `FailedToProduce` product type.

A test is added with a failing path and a good path to exercise this
handling in both the serial and multiprocess engines.

 src/python/pants/engine/exp/BUILD                                                               |   1 +
 src/python/pants/engine/exp/engine.py                                                           | 339 +++++++++++++++++++++++++++++++++++++++++++-------
 src/python/pants/engine/exp/examples/BUILD                                                      |   1 +
 src/python/pants/engine/exp/examples/planners.py                                                |  12 +-
 src/python/pants/engine/exp/examples/visualizer.py                                              |  18 +--
 src/python/pants/engine/exp/scheduler.py                                                        |  45 ++++---
 src/python/pants/engine/exp/targets.py                                                          |  34 +++--
 tests/python/pants_test/engine/exp/examples/scheduler_inputs/src/java/codegen/selector/BLD.json |  10 ++
 tests/python/pants_test/engine/exp/examples/scheduler_inputs/src/thrift/BLD.json                |   7 ++
 tests/python/pants_test/engine/exp/test_engine.py                                               |  43 ++++++-
 tests/python/pants_test/engine/exp/test_scheduler.py                                            |  23 ++--
 11 files changed, 439 insertions(+), 94 deletions(-)
CI went green here:
  https://travis-ci.org/pantsbuild/pants/builds/87523475

Issues

  • 0
  • 1
  • 0
  • 1
Description From Last Updated
John Sirois
Benjy Weinberger
John Sirois
John Sirois
John Sirois
John Sirois
John Sirois
Review request changed

Status: Closed (submitted)

Stu Hood

Rather than being a global scheduler property, is this something that Planners should be driving on a promise-by-promise basis?

  1. Not sure.  Examples of when we'd want to cede control to a planner to potentially spin alot of cores would be useful to think about this.  In particular cases where it wouldn't make sense for one planner but it would for others.
Loading...