Introduce ignore_patterns option

Review Request #3414 - Created Feb. 3, 2016 and submitted

Information
Timur Abishev
pants
tabishev/pants_build_ignore_option
2880
Reviewers
pants-reviews
dturner-tw, patricklaw, stuhood, zundel

Original motivation:
npm install command creates a lot of BUILD.md files which pants fail to interpret as pants BUILD files.

Other problems:
Currently there is no good way to prevent some directories / BUILD files from parsing. spec_path stands only for spec prefixes, exclude_target_regexp for targets.
Ability to exclude some directories not only based on prefixes can be useful in more cases than original motivation: with it we can exclude node_modules and bower_components directories from scanning which can be good for performance. Also there are /target/ exclusion in Square's code which is also target for this change.

Solution:
Introduce ignore_patterns option which behave like .gitignore but for pants BUILD files.
Dependency on pathspec library (sources: https://github.com/cpburnz/python-path-specification) was added to make .gitignore-like filtration. It doesn't contain any C/C++ extensions.

Performance impact:
I've checked ignore_patterns on big twitter repository trying to execute "scanning heavy" commands such as filter and dependees. There is no noticeable impact on scan_build_files performance (about 15%, but in general it's only about 2%) and some noticeable impact on get_build_files_family performance (about x2.5, but in general it's only about 7%).

But executions of get_build_files_family can be much faster as their result most likely was calculated already in scan_build_files. I will publish follow-up PR with this change.
Upd: I've implemented the fix and it doesn't seems to make situation better. Also it doesn't work well with spec_excludes code. So I've decided to return to this question after all deprecated code will be removed and with profiler.

5-6 diff contains spec_excludes deprecation and deprecation of ability to filter broken BUILD files using exclude_target_regexp.

https://travis-ci.org/ttim/pants/builds/107142688

Also I've tested --spec-excludes after deprecation and checked exclude_target_regexp for ignoring BUILD files.

Issues

  • 0
  • 12
  • 2
  • 14
Description From Last Updated
Timur Abishev
Timur Abishev
Patrick Lawson
Stu Hood
Timur Abishev
Timur Abishev
Patrick Lawson
Benjy Weinberger
Timur Abishev
Timur Abishev
Benjy Weinberger
Timur Abishev
Stu Hood
Benjy Weinberger
Timur Abishev
Timur Abishev
Timur Abishev
Timur Abishev
Review request changed

Status: Closed (submitted)

Change Summary:

6f4b8511bb181e7dbd15726084f5f0897490d96c

Benjy Weinberger
Ship It!
Benjy Weinberger

This is marked as submitted, but I'm not seeing it in master?

Benjy Weinberger

This is marked as submitted, but I'm not seeing it in master?

  1. Looks like it's in https://github.com/pantsbuild/pants/commit/6f4b8511bb181e7dbd15726084f5f0897490d96c.

  2. My bad, my IDE wasn't refreshing.

Loading...