[engine] Introduce static analysis model and replace validator with it

Review Request #4251 - Created Sept. 28, 2016 and submitted

Information
Nick Howard (Twitter)
pants
3890
Reviewers
pants-reviews
kwlzn, stuhood, yujiec

Introduces a Rule Graph that only contains the rules that cannot result in a Noop due to missing selector values. Replaces the existing validator with one backed by this new model. This means that for instance, intrinsics, or product graphs that depend on intrinsics, but are ultimately Nooped can be eliminated early.

There's still a number of TODOs left in the patch, but it's to a point where I feel like I need feedback before making further changes.

Improvements
* better error reporting over the previous implementation: The validator will now report transitive failures. Subject type match failures are also reported

Before

Found 1 rules with errors:
  (A, (Select(B),), noop)
    There is no producer of Select(B)

After

Rules with errors: 1
  (A, (Select(B),), noop):
    no matches for Select(B) with subject types: SubA

Doesn't cover
* cycles: I've got some ideas, but this has gotten pretty big already. Cycles won't cause non-termination because the graph generation algorithm only visits each rule/subject-type entry once.
* noops caused by tasks that can return None
* actually generating nodes base on the graphs
* variants
* HasProducts

Follow on work:
* performance. There are improvements to the model generation that could be done. But, I want to split that work out from this initial cut.
* integration into the scheduler. After this lands, I want to adapt the scheduler to use it for pre-filling dependencies of nodes to execute, and ultimately to simplify or eliminate SelectNodes.

Introducing and running tests for the graph and validator locally. CI passed at https://travis-ci.org/pantsbuild/pants/builds/165056642

Issues

  • 0
  • 1
  • 0
  • 1
Description From Last Updated
Nick Howard (Twitter)
Stu Hood
Yujie Chen
Nick Howard (Twitter)
Nick Howard (Twitter)
Yujie Chen
Nick Howard (Twitter)
Nick Howard (Twitter)
Review request changed

Status: Closed (submitted)

Change Summary:

Submitted as https://github.com/pantsbuild/pants/commit/9bb59594708105402856879e9a6970fbfadb0b41
Loading...