[engine] Rule Graph construction perf improvements

Review Request #4281 — Created Oct. 5, 2016 and submitted

nhoward_tw
pants
3928
pants-reviews
kwlzn, stuhood, yujiec

This follows up https://rbcommons.com/s/twitter/r/4251/, cleaning up the perf todos.

It passes through the full graph construction's intermediate states so that they can short circuit subsequent subgraph constructions.

It also moves transitive unfulfillable rule elimination so that it happens once for both full graph and subgraph instead of once per construction.

There's also a bug fix in RuleEdges matching, which was caused by matching being overly broad.

Ran engine tests locally. CI passed at https://travis-ci.org/pantsbuild/pants/builds/165317483

  1. Ship It!
    1. I have a vague idea of what is going on here, but expect it to become more clear when this is actually used to instantiate Nodes.

    2. Mostly it's just eliminating a whole bunch of redundant work.

  2. 
      
  1. Ship It!
  2. 
      
  1. lgtm!

  2. src/python/pants/engine/rules.py (Diff revision 1)
     
     
     

    nit: indent

  3. 
      
Review request changed

Status: Closed (submitted)

Change Summary:

Submitted as https://github.com/pantsbuild/pants/commit/7a79f52f014d838d2178c4e2c08b1bb2b8ce4c15
Loading...