Restore Sources custom types per extension.

Review Request #3011 - Created Oct. 21, 2015 and submitted

Information
John Sirois
pants
jsirois/engine/exp/sources_custom_pickle
2423
3010
98b23e5...
Reviewers
pants-reviews
benjyw, ity, kwlzn, nhoward_tw, patricklaw, stuhood
This was removed to support the multiprocess engine which relies on
pickling.  The dynamic class objects generated by `Sources.of` were not
pickle compatible.  Restore `Sources.of` memoized Sources subtypes
per-extension by implementing `__reduce__` and exposing the `Sources`
subtypes at the module level.

 src/python/pants/engine/exp/examples/BUILD           |  1 +
 src/python/pants/engine/exp/examples/planners.py     | 35 +++++++++++++++++++++++++----------
 src/python/pants/engine/exp/examples/visualizer.py   |  9 ++-------
 tests/python/pants_test/engine/exp/test_scheduler.py |  6 +++---
 4 files changed, 31 insertions(+), 20 deletions(-)

REPL testing to both check quickly pickle/unpicle now worked and type
identity was maintained. This was backed up by the engine test which
pickles and unpickles Sources as part of running the multiprocess
engine.

Also re-rendered the simple codegen plan, make more sense now that the
Sources product type carries the extension - see attached and compare
to the image in https://rbcommons.com/s/twitter/r/3010/:

$ ./pants run src/python/pants/engine/exp/examples:viz -- \
  tests/python/pants_test/engine/exp/examples/scheduler_inputs \
  compile src/java/codegen/simple

CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/86753888

Files


John Sirois
John Sirois
John Sirois
John Sirois
Review request changed

Status: Closed (submitted)

Loading...