Clean up after failed artifact extractions

Review Request #4255 - Created Sept. 22, 2016 and submitted

Information
Stu Hood
pants
3894, 3895
Reviewers
pants-reviews
patricklaw, peiyu, yujiec

In the case of a corrupted remote artifact, we rely on tarball checksums to determine whether an extraction attempt has failed. But because we don't see tarball extraction errors until after some of the contents of the tarball might have been extracted, we might have already polluted the workspace.

Luckily, for Tasks that set cache_target_dirs, the contents of the cache artifact should all be located under the results_dir, meaning that clearing the results_dir in case of failure to extract should purge partially extracted artifacts.

  • Clean up after failed artifact extractions which specify a results_dir.
  • Move initialization of results_dirs before cache lookups and validate that they are are set by accessing them via their property whenever cache_target_dirs is set.
  • Deprecate compression level ==0, which disables gzip compression entirely, and doesn't seem to do any checksumming (without compression, none of truncating/mutating/etc of the test file were detected)
  • Extract the pants_test.cache.cache_server helper to provide a "remote" cache server for integration testing
  • Add an end-to-end corruption test for java compilation

green ci: https://travis-ci.org/pantsbuild/pants/builds/165385610

Stu Hood
Peiyu Wang
Stu Hood
Stu Hood
Peiyu Wang
Peiyu Wang
Stu Hood
Peiyu Wang
Peiyu Wang
Stu Hood
Stu Hood
Yujie Chen
Peiyu Wang
Stu Hood
Review request changed

Status: Closed (submitted)

Change Summary:

Merged as ce5f34706e4791bfe32c94de82f1233360e3b110

Loading...