[engine] Introduce static analysis model and replace validator with it

Review Request #4251 - Created Sept. 28, 2016 and submitted

Nick Howard (Twitter)
kwlzn, stuhood, yujiec

Introduces a Rule Graph that only contains the rules that cannot result in a Noop due to missing selector values. Replaces the existing validator with one backed by this new model. This means that for instance, intrinsics, or product graphs that depend on intrinsics, but are ultimately Nooped can be eliminated early.

There's still a number of TODOs left in the patch, but it's to a point where I feel like I need feedback before making further changes.

* better error reporting over the previous implementation: The validator will now report transitive failures. Subject type match failures are also reported


Found 1 rules with errors:
  (A, (Select(B),), noop)
    There is no producer of Select(B)


Rules with errors: 1
  (A, (Select(B),), noop):
    no matches for Select(B) with subject types: SubA

Doesn't cover
* cycles: I've got some ideas, but this has gotten pretty big already. Cycles won't cause non-termination because the graph generation algorithm only visits each rule/subject-type entry once.
* noops caused by tasks that can return None
* actually generating nodes base on the graphs
* variants
* HasProducts

Follow on work:
* performance. There are improvements to the model generation that could be done. But, I want to split that work out from this initial cut.
* integration into the scheduler. After this lands, I want to adapt the scheduler to use it for pre-filling dependencies of nodes to execute, and ultimately to simplify or eliminate SelectNodes.

Introducing and running tests for the graph and validator locally. CI passed at https://travis-ci.org/pantsbuild/pants/builds/165056642


  • 0
  • 1
  • 0
  • 1
Description From Last Updated
Nick Howard (Twitter)
Stu Hood
Yujie Chen
Nick Howard (Twitter)
Nick Howard (Twitter)
Yujie Chen
Nick Howard (Twitter)
Nick Howard (Twitter)
Review request changed

Status: Closed (submitted)

Change Summary:

Submitted as https://github.com/pantsbuild/pants/commit/9bb59594708105402856879e9a6970fbfadb0b41