Support plugins resolution in pants.

Review Request #2622 - Created Aug. 14, 2015 and submitted

Information
John Sirois
pants
jsirois/plugins/bootstrap
1976, 1977
7647867...
Reviewers
pants-reviews
benjyw, davidt, stuhood, wickman
Previously, pants could only load plugins already in the working set
(PYTHONPATH).  With this change, pants both resolves (and caches)
plugins as well as loading backends through their entry points.

In order to support this, two small changes were made to the options
system:

 1. OptionsBootstrapper was altered to cache full options per-known
    scope info set.  This allows the work of bootstrapping to be shared
    by the PluginResolver and the main options loading sequence.
 2. Options was refactored to support creating a copy that behaves the
    same as the original except that option values from command line
    flags are ignored.

These changes combine to give the early stage PluginResolver a clean,
performant means of using the configured PythonSetup and PythonRepos
subsystems to resolve plugins from the repos and indices the user has
set up for their pants installation.

 src/python/pants/base/extension_loader.py                   |  18 +++----
 src/python/pants/bin/BUILD                                  |  22 +++++----
 src/python/pants/bin/goal_runner.py                         |   6 +--
 src/python/pants/bin/pants_exe.py                           |  11 ++++-
 src/python/pants/bin/plugin_resolver.py                     | 125 ++++++++++++++++++++++++++++++++++++++++++++++++
 src/python/pants/option/options.py                          |  61 ++++++++++++++++++++----
 src/python/pants/option/options_bootstrapper.py             |  28 ++++++-----
 src/python/pants/option/parser.py                           |   6 ++-
 tests/python/pants_test/base/test_extension_loader.py       |   5 +-
 tests/python/pants_test/bin/BUILD                           |  22 +++++++++
 tests/python/pants_test/bin/__init__.py                     |   0
 tests/python/pants_test/bin/test_plugin_resolver.py         | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/python/pants_test/option/test_options.py              |  67 ++++++++++++++++++++++----
 tests/python/pants_test/option/test_options_bootstrapper.py |  24 +++++++++-
 14 files changed, 462 insertions(+), 61 deletions(-)

Manual testing as follows:

  1. Bump the pants version in another clone and do a local dry run
    release to build the new entry_points carrying plugins (see:
    https://rbcommons.com/s/twitter/r/2615/)

  2. Disable the local android backend:

$ git diff -U1
diff --git a/pants.ini b/pants.ini
index 4756c9b..d7a67a9 100644
--- a/pants.ini
+++ b/pants.ini
@@ -27,3 +27,3 @@ backend_packages: [
     "internal_backend.utilities",
-    "pants.backend.android",
+#    "pants.backend.android",
     "pants.contrib.cpp",

  1. Run pants using the new plugin resolver/installed mechanism:
$ rm -rf ~/.cache/pants/plugins/
$ PANTS_PLUGINS="['pantsbuild.pants.backend.android==0.0.42']" PANTS_PYTHON_REPOS_REPOS="['/home/jsirois/dev/3rdparty/jsirois-pants3/dist']" ./pants goals --all
INFO:pants.bin.plugin_resolver:Resolving new plugins...:
  pantsbuild.pants.backend.android==0.0.42
INFO] Detected git repository at /home/jsirois/dev/3rdparty/jsirois-pants4 on branch jsirois/plugins/bootstrap
Installed goals:
                    apk: Build an android bundle with compiled code and assets.
...

CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/75655671

John Sirois
Stu Hood
John Sirois
Benjy Weinberger
John Sirois
John Sirois
John Sirois
Review request changed

Status: Closed (submitted)

Loading...