Initial support for Go BUILD gen.

Review Request #2676 - Created Aug. 21, 2015 and submitted

Information
John Sirois
pants
jsirois/issues/2016
2016, 2039
2689
8190b71...
Reviewers
pants-reviews
kwlzn, mateor, patricklaw, zundel

This adds support for both dynamically discovering and injecting
Go targets into the in-memory BuildGraph as well as adding support
for materializing BUILD files for those targets to disk.

The target creation scope can optionally include direct remote
dependencies, but otherwise defaults to only handling local source
targets.

Additionally, a simple templating system is provided to allow for
BUILD materialization that suits the local BUILD style.

To support some of the BUILD gen calculations and keep path calculation
code DRY, utilities are moved into the go target hierarchy.

Auto re-gen Go BUILD files in the repo with:
./pants buildgen.go --remote --materialize ::

Also used this task to generate Go BUILD graphs in the Medium mono repo
by just seeding go_binary targets for their main packages' dirs with:
echo "go_binary(name='[parent dir]')" > BUILD
and then running buildgen.go, finally pinning revs in the BUILD files
generated to their 3rdparty/golang go_remote_library source root.

contrib/go/examples/3rdparty/go/github.com/AdRoll/goamz/BUILD | 10 +-
contrib/go/examples/3rdparty/go/github.com/bitly/go-simplejson/BUILD | 3 +
contrib/go/examples/3rdparty/go/github.com/cbroglie/mapstructure/BUILD | 3 +
contrib/go/examples/3rdparty/go/github.com/gorilla/context/BUILD | 3 +
contrib/go/examples/3rdparty/go/github.com/gorilla/mux/BUILD | 3 +
contrib/go/examples/src/go/hello/BUILD | 7 +-
contrib/go/examples/src/go/libA/BUILD | 5 +-
contrib/go/examples/src/go/libB/BUILD | 5 +-
contrib/go/examples/src/go/libC/BUILD | 5 +-
contrib/go/examples/src/go/libD/BUILD | 7 +-
contrib/go/examples/src/go/libE/BUILD | 7 +-
contrib/go/examples/src/go/server/BUILD | 5 +-
contrib/go/src/python/pants/contrib/go/register.py | 3 +
contrib/go/src/python/pants/contrib/go/subsystems/fetchers.py | 25 +++-
contrib/go/src/python/pants/contrib/go/targets/BUILD | 14 +-
contrib/go/src/python/pants/contrib/go/targets/go_local_source.py | 35 ++++-
contrib/go/src/python/pants/contrib/go/targets/go_remote_library.py | 31 +++--
contrib/go/src/python/pants/contrib/go/targets/go_target.py | 67 +++++++++
contrib/go/src/python/pants/contrib/go/tasks/BUILD | 25 ++++
contrib/go/src/python/pants/contrib/go/tasks/go_buildgen.py | 366 ++++++++++++++++++++++++++++++++++++++++++++++++++
contrib/go/src/python/pants/contrib/go/tasks/go_fetch.py | 1 +
contrib/go/src/python/pants/contrib/go/tasks/go_workspace_task.py | 3 +-
contrib/go/tests/python/pants_test/contrib/go/tasks/BUILD | 18 +++
contrib/go/tests/python/pants_test/contrib/go/tasks/test_go_buildgen.py | 277 ++++++++++++++++++++++++++++++++++++++
24 files changed, 886 insertions(+), 42 deletions(-)

New tests for GoBuildgen:

$ ./pants test.pytest --coverage=modules:pants.contrib.go.tasks.go_buildgen contrib/go/tests/python/pants_test/contrib/go/tasks/:go_buildgen
...
contrib/go/tests/python/pants_test/contrib/go/tasks/test_go_buildgen.py ................

=========== 16 passed in 1.97 seconds ============
Name                                                       Stmts   Miss Branch BrMiss  Cover
--------------------------------------------------------------------------------------------
contrib/go/src/python/pants/contrib/go/tasks/go_buildgen     192      4     72      8    95%

An example of the buildgen.go output with emphasis on contrib/go/examples/3rdparty
and a note that the FLOATING lines are warn/yellow:

$ ./pants buildgen.go --remote --materialize ::
...
9:32:29 00:00 [main]
               (To run a reporting server: ./pants server)
19:32:29 00:00   [bootstrap]
19:32:29 00:00   [setup]
19:32:29 00:00     [parse]
               Executing tasks in goals: buildgen
19:32:30 00:01   [buildgen]
19:32:30 00:01     [go]
                   Auto generated the following Go BUILD files: BUILD file (import paths)
                    contrib/go/examples/src/go/libA/BUILD (libA)
                    contrib/go/examples/src/go/libC/BUILD (libC)
                    contrib/go/examples/src/go/libB/BUILD (libB)
                    contrib/go/examples/src/go/libE/BUILD (libE)
                    contrib/go/examples/src/go/libD/BUILD (libD)
                    contrib/go/examples/3rdparty/go/github.com/gorilla/context/BUILD (github.com/gorilla/context) FLOATING
                    contrib/go/examples/3rdparty/go/github.com/AdRoll/goamz/BUILD (github.com/AdRoll/goamz/dynamodb) 51b70b0035cf28c4ca6d719156347ac56a41d922
                    contrib/go/examples/src/go/hello/BUILD (hello)
                    contrib/go/examples/src/go/server/BUILD (server)
                    contrib/go/examples/3rdparty/go/github.com/bitly/go-simplejson/BUILD (github.com/bitly/go-simplejson) v0.4.3
                    contrib/go/examples/3rdparty/go/github.com/gorilla/mux/BUILD (github.com/gorilla/mux) FLOATING
                    contrib/go/examples/3rdparty/go/github.com/cbroglie/mapstructure/BUILD (github.com/cbroglie/mapstructure) 25325b46b67d1c3eb7d58bad37d34d89a31cf9ec
19:32:30 00:01   [complete]
               SUCCESS

CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/76587511

John Sirois
Cody Gibb
John Sirois
Mateo Rodriguez
John Sirois
Review request changed

Status: Closed (submitted)

Loading...