Create a complete product graph, and use it to validate inputs

Review Request #3245 - Created Dec. 14, 2015 and submitted

Stu Hood
2525, 2756
jsirois, patricklaw

This change adds input product requirements to Planners, which allows us to inspect the product graph to determine (recursively) whether an output product can be produced for a subject.

This allows us to kill the required parameter on def promise, because the method will only ever be called when a product can definitely be produced. It also means that the def plan method of a Planner will only ever be called when all of its declared inputs are present... meaning that no input validation is necessary in Planners.

  • Declare input product types on Planners as a series of input product requirements in DNF form.
    • (We can likely add a small DSL to make this less confusing to declare)
  • Use the declared Product requirements to determine which Planners are capable of producing an output product for a subject (recursively).
    • Treat Configuration and Sources as products for the purposes of the validation, so that a planner can declare a product requirement on Configuration
    • Add NoPlanner to produce Plans that lift Sources and Configuration defined directly on a subject into the product namespace
  • Make thrift configuration types unique per language in order to avoid needing to implement field matching for product requirements
  • Remove the required flag, and use the product graph to determine when to request products. Planners must now always produce some Plan when requested

Finally, this also allows us to explode for the absence of config (see #2525). A new example target and test demonstrate this:

$ ./pants run src/python/pants/engine/exp/examples:viz -- \
  tests/python/pants_test/engine/exp/examples/scheduler_inputs \
  compile \
pants.engine.exp.scheduler.PartiallyConsumedInputsError: While attempting to produce Classpath for Target(address=src/thrift/codegen/unconfigured), some products could not be consumed:
  To consume Sources(u'.thrift'):
    ScroogePlanner needed (ScroogeScalaConfiguration OR ScroogeJavaConfiguration)
    ApacheThriftPlanner needed (ApacheThriftJavaConfiguration)

The error message indicates that the Sources.of('.thrift') product was not successfully used by any Planner, and lists what additional inputs would be necessary in order to use it.

Stu Hood
Stu Hood
Stu Hood
Stu Hood
Review request changed

Status: Closed (submitted)

Change Summary:

Merged tbr as 7f6b9e3791be3b69c69c3a3bd0b30c21b38a5898