Remove use of twitter.common.util.topological_sort in SortTargets

Review Request #3121 - Created Nov. 14, 2015 and submitted

Information
Tansy Arron-Walker
pants
2543, 2561
Reviewers
pants-reviews
jsirois, patricklaw, stuhood, zundel

Use our own topological sort which already operates over targets. Allows us to remove a twitter commons dependency.

This is what our cycle detection on sort looks like now:

]$ ./pants sort testprojects/src/java/org/pantsbuild/testproject/cycle1::
WARN] Injecting dependency from BuildFileAddress(FilesystemBuildFile(/home/tansy/code/pants/testprojects/src/java/org/pantsbuild/testproject/cycle2/BUILD), cycle2) on testprojects/src/java/org/pantsbuild/testproject/cycle1:cycle1, but the dependency is not in the BuildGraph.  This probably indicates a dependency cycle, but it is not an error until sort_targets is called on a subgraph containing the cycle.
INFO] Detected git repository at /home/tansy/code/pants on branch master

FAILURE

Exception caught: (<class 'twitter.common.util.DependencyCycle'>)
  File "/home/tansy/code/pants/src/python/pants/bin/pants_exe.py", line 32, in <module>
    main()
  File "/home/tansy/code/pants/src/python/pants/bin/pants_exe.py", line 26, in main
    LocalPantsRunner(exiter).run()
  File "/home/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 60, in run
    self._maybe_profiled(self._run)
  File "/home/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 57, in _maybe_profiled
    runner()
  File "/home/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 90, in _run
    result = goal_runner.run()
  File "/home/tansy/code/pants/src/python/pants/bin/goal_runner.py", line 353, in run
    result = self._execute_engine()
  File "/home/tansy/code/pants/src/python/pants/bin/goal_runner.py", line 342, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/home/tansy/code/pants/src/python/pants/engine/engine.py", line 26, in execute
    self.attempt(context, goals)
  File "/home/tansy/code/pants/src/python/pants/engine/round_engine.py", line 218, in attempt
    goal_executor.attempt(explain)
  File "/home/tansy/code/pants/src/python/pants/engine/round_engine.py", line 47, in attempt
    task.execute()
  File "/home/tansy/code/pants/src/python/pants/backend/core/tasks/console_task.py", line 57, in execute
    for value in self.console_output(targets):
  File "/home/tansy/code/pants/src/python/pants/backend/core/tasks/sorttargets.py", line 43, in console_output
    for group in topological_sort(depmap):
  File "/home/tansy/code/pants/build-support/pants_dev_deps.venv/local/lib/python2.7/site-packages/twitter/common/util/__init__.py", line 101, in topological_sort
    raise DependencyCycle('Data contained a cycle! Unsatisfied deps: %s' % remaining_deps)

Exception message: Data contained a cycle! Unsatisfied deps: set([u'testprojects/src/java/org/pantsbuild/testproject/cycle1:cycle1', u'testprojects/src/java/org/pantsbuild/testproject/cycle2:cycle2'])

This is what a cycle detected exception looks like with this change:

$ ./pants sort testprojects/src/java/org/pantsbuild/testproject/cycle1::
WARN] Injecting dependency from BuildFileAddress(FilesystemBuildFile(/home/tansy/code/pants/testprojects/src/java/org/pantsbuild/testproject/cycle2/BUILD), cycle2) on testprojects/src/java/org/pantsbuild/testproject/cycle1:cycle1, but the dependency is not in the BuildGraph.  This probably indicates a dependency cycle, but it is not an error until sort_targets is called on a subgraph containing the cycle.
INFO] Detected git repository at /home/tansy/code/pants on branch tansy.issue2543

FAILURE

Exception caught: (<class 'pants.build_graph.build_graph.CycleException'>)
  File "/home/tansy/code/pants/src/python/pants/bin/pants_exe.py", line 32, in <module>
    main()
  File "/home/tansy/code/pants/src/python/pants/bin/pants_exe.py", line 26, in main
    LocalPantsRunner(exiter).run()
  File "/home/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 60, in run
    self._maybe_profiled(self._run)
  File "/home/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 57, in _maybe_profiled
    runner()
  File "/home/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 90, in _run
    result = goal_runner.run()
  File "/home/tansy/code/pants/src/python/pants/bin/goal_runner.py", line 353, in run
    result = self._execute_engine()
  File "/home/tansy/code/pants/src/python/pants/bin/goal_runner.py", line 342, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/home/tansy/code/pants/src/python/pants/engine/engine.py", line 26, in execute
    self.attempt(context, goals)
  File "/home/tansy/code/pants/src/python/pants/engine/round_engine.py", line 218, in attempt
    goal_executor.attempt(explain)
  File "/home/tansy/code/pants/src/python/pants/engine/round_engine.py", line 47, in attempt
    task.execute()
  File "/home/tansy/code/pants/src/python/pants/backend/core/tasks/console_task.py", line 57, in execute
    for value in self.console_output(targets):
  File "/home/tansy/code/pants/src/python/pants/backend/core/tasks/sorttargets.py", line 22, in console_output
    sorted_targets = sort_targets(targets)
  File "/home/tansy/code/pants/src/python/pants/build_graph/build_graph.py", line 459, in sort_targets
    roots, inverted_deps = invert_dependencies(targets)
  File "/home/tansy/code/pants/src/python/pants/build_graph/build_graph.py", line 451, in invert_dependencies
    invert(target)
  File "/home/tansy/code/pants/src/python/pants/build_graph/build_graph.py", line 444, in invert
    invert(dependency)
  File "/home/tansy/code/pants/src/python/pants/build_graph/build_graph.py", line 444, in invert
    invert(dependency)
  File "/home/tansy/code/pants/src/python/pants/build_graph/build_graph.py", line 437, in invert
    raise CycleException(cycle)

Exception message: Cycle detected:
    testprojects/src/java/org/pantsbuild/testproject/cycle1:cycle1 ->
    testprojects/src/java/org/pantsbuild/testproject/cycle2:cycle2 ->
    testprojects/src/java/org/pantsbuild/testproject/cycle1:cycle1

CI is running: https://travis-ci.org/pantsbuild/pants/builds/91062204

Tansy Arron-Walker
Tansy Arron-Walker
John Sirois
John Sirois
Tansy Arron-Walker
Review request changed

Status: Closed (submitted)

Eric Ayers
Ship It!
Loading...