Add native support for incremental caching, and use it in jvm_compile

Review Request #2991 - Created Oct. 18, 2015 and submitted

Information
Stu Hood
pants
2394
8a54cd7...
Reviewers
pants-reviews
benjyw, ity, jsirois, nhoward_tw, patricklaw

This moves jvm_compile to the cache_target_dirs setting in order to help kill the manual caching API before the engine refactor. In order to do that, it was necessary to add native support for incremental compilation to cache_manager.py and task.py.

Optional incremental builds are implemented by copying from the previous results_dir for a VersionedTarget to the current one. This has the advantage that results_dirs are immutable (concurrent ./pants run.jvm invocations should now be safe, regardless of changing ivy resolves).

  • Implement optional incrementalism for tasks, with copy-on-write results_dirs
  • Switch jvm_compile to cache_target_dirs, and drop all usage of self.workdir
  • Replace the cache_hit_callback with a rmtree of a targets' results_dir before extracting a cache hit
  • Privatize-stale and delete-unused methods for task.py
  • Move the no_cache tag from jvm_target to target in order to implement it in task.py
  • Add vt.id to the path for vt.results_dir, which fixed an issue where two targets with identical fingerprints would share a results_dir (even thought that will "usually" work, it's confusing)
  • Add mark_dirty methods to Payload and PayloadField, for testing purposes.

https://travis-ci.org/pantsbuild/pants/builds/86336104

Issues

  • 0
  • 7
  • 0
  • 7
Description From Last Updated
John Sirois
Benjy Weinberger
Stu Hood
Benjy Weinberger
Stu Hood
Stu Hood
Benjy Weinberger
Benjy Weinberger
Benjy Weinberger
Nick Howard (Twitter)
Nick Howard (Twitter)
Stu Hood
Review request changed

Status: Closed (submitted)

Change Summary:

Merged as ab571f16a675703375a363f47201f498ff41118f

Loading...