Refactor detect_duplicates with some user friendly features

Review Request #3178 - Created Nov. 25, 2015 and submitted

Eric Ayers
gmalmquist, ity, jsirois, nhoward_tw

I was having a hard time making the duplicate output make sense in the Square repo. First of all, developers don't want to see this output, so I wanted a way for them to just turn it off until I can get the dups out of properly excluded. Also, we have known conflicts in our resources that I wanted to exclude which the exising logic was unable to suppress. Many of the files that were coming up were nusciance, I wanted more flexibility on how to exclude them.

- Deprecate the 'excludes' argument.  The only thing that won't be accepted by the new arguments is that `--excludes` accepts a list of comma separated values.
- Added --exclude-files to replace --excludes
- Added --exclude-dirs to exclude directories
- Added --exclude-patterns to use a regular expression
- Added --skip to easily turn off the duplicate checking feature
- Moved logic out of __init__()
- Moved the logic to check for excludes to a central place where it can
    be used to exclude classes on the classpath, not just artifacts in jars
- Default to excluding the security files META-INF/*.RSA META-INF/*.DSA META-INF/*.SF.  The security file prefixes are automatically excluded by jvm_binary as well. Added a regular expression feature to cull them out.
- Default to excluding META-INF/services files.  Any file that begins with META-INF/services is expected to have conflicts - the logic in jvm_binary merges them together.

migrations/options/src/python/ | 1 +
src/python/pants/backend/jvm/tasks/BUILD | 2 ++
src/python/pants/backend/jvm/tasks/ | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------
tests/python/pants_test/tasks/ | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
4 files changed, 137 insertions(+), 22 deletions(-)

CI is green at
Updated the unit tests
Tested this in our repo and confirmed the internal resources can now be excluded.


  • 0
  • 3
  • 0
  • 3
Description From Last Updated
Eric Ayers
Eric Ayers
Benjy Weinberger
Eric Ayers
Benjy Weinberger
Eric Ayers
Garrett Malmquist
Eric Ayers
Eric Ayers
Review request changed

Status: Closed (submitted)

Change Summary:

Thanks Benjy & Garrett. Commit deee390