[engine] Tighten input validation

Review Request #3448 - Created Feb. 11, 2016 and submitted

Information
Stu Hood
pants
2869, 2912
Reviewers
pants-reviews
benjyw, ity, jsirois, nhoward_tw, patricklaw, peiyu

This review deals with the "partially specified configuration" issue described on #2869 and #2525. It addresses (what I think is) the last issue blocking integration of the new engine. I will begin working on integration while this is under review.

  • Reintroduced usage of PartiallyConsumedInputsError, which allows for useful errors in cases where only some of a task's required (literal*) inputs were specified. See r3245 for the original implementation of this check.
    • Example error received when a target has ThriftSources, but none of the configuration types that might be necessary to consume it:
pants.engine.exp.scheduler.PartiallyConsumedInputsError:
Some products were partially specified for `src/thrift/codegen/selector:selector`:
  To consume `thrift` and produce `java`:
    gen_apache_thrift also needed (`apache_thrift_java_configuration`)
    gen_scrooge_thrift also needed (`scrooge_java_configuration`)
  To consume `thrift` and produce `scala`:
    gen_scrooge_thrift also needed (`scrooge_scala_configuration`)
  • Add the Noop state to differentiate missing inputs from actual user-relevant failures. This allows all Throw nodes to be considered actual errors, so they are now rendered in the visualizer. Many errors that used to be synchronous now fail via Throw instead. See the attached "multiple sources of product" graph, which is much more usable than it would have been otherwise.
  • Allow inputs to be marked 'optional' for tasks via SelectOptional. Optional inputs will be None if they cannot be produced.
  • Associate all "goals" with a single product type, which makes a "goal" just an alias for a particular product. In cases where a goal represents multiple products, a synthetic product type extending Goal is used, with a task to validate that at least one of the products was produced. See the example of GenGoal which, depends on all of the Sources classes.
  • Reintroduce the failure when multiple tasks are able to produce a product for a subject (at least until #2526 is implemented and we can merge them instead).

* A 'literal' input here is considered to be anything recorded in the SymbolTable that is used to parse BUILD files. Since users can only actually specify literal values, we only raise errors for task failures in cases where a user:
1. might have put a literal value in a BUILD file, and
2. might be able to add additional literal values to allow a task to produce a requested product

Fixed some tests that were marked xfail, and added a few more.

https://travis-ci.org/pantsbuild/pants/builds/108637875

Files


Stu Hood
Timur Abishev
Timur Abishev
Peiyu Wang
Stu Hood
Stu Hood
Stu Hood
Stu Hood
Stu Hood
Stu Hood
Review request changed

Status: Closed (submitted)

Change Summary:

Merged --tbr as 6e2d807a8f676a3a0eac27343dd31c4e7d9bed8d

Loading...