Add support for pants-ignore to ProjectTree

Review Request #3698 — Created April 15, 2016 and submitted — Latest diff uploaded

2956, 3232
benjyw, gmalmquist, stuhood

This review adds built-in support to src/python/pants/base/ for an option --ignore-patterns. This will be a new option, independent from the existing --ignore-patterns option.

The idea behind the split is that there are patterns that should never ever be matched by pants (similar to the things that are utterly ignored by git: those specified by .gitignore files) and should thus be pushed down into ProjectTree, which we hope will eventually be responsible for all filesystem access in pants. Additionally, there is a separate, higher-level set of ignore patterns that only applies to BUILD file parsing. The current --ignore-patterns option represents the latter high level option, as it is only (and should only) be applied to BUILD file parsing.

Changes made:
1. A new option has been added in
2. ProjectTree takes an additional init argument which is a list of ignore patterns (default to be [])
3. All public methods of FileSystemProjectTree and ScmProjectTree have been made private and renemed to ..._raw since they don't have ignore logic.
4. ProjectTree base class now has the implementation of common APIs for FileSystem and SCM project tree, with pants ignore logic. When querying ignored path, it will be shown as non-existent. When accessing ignored path, an exception will be thrown.
5. ProjectTree created in legacy package is defaulted to ignore dotfiles.
6. add 2 new test bases - ProjectTreeTestBase and PantsIgnoreTestBase (derived from ProjectTreeTestBase)
7. 2 new test files to test pants ignore, 1 for file system project tree, 1 for scm project tree.
8. remove "API public" doc string that was accodentally added in

Something to notice:
1. For pants ignore to work, there should be no "." in relative path to build_root.

Latest CI: