[pantsd] Implement PantsRunner->[LocalPantsRunner,RemotePantsRunner] et al.

Review Request #3286 - Created Dec. 30, 2015 and submitted

Kris Wilson
benjyw, jsirois, nhoward_tw, patricklaw, stuhood, zundel

This is the initial v1/python closed loop pantsd runner that completes the plumbing needed to fully execute pants runs in-daemon via pants itself. This bootstraps us for further hardening and iteration in and up to v1. Then the plan is to circle back in the v2 timeframe to implement an embedded C-based nailgun client for optimal runtime performance.

  • Implement RemotePantsRunner - a Nailgun client runner for the pailgun.
  • Supplant LocalPantsRunner with a higher level PantsRunner that uses either RemotePantsRunner or LocalPantsRunner depending on the configuration and environmental conditions.
  • Implement tty capability detection and env passing per the nailgun spec for the client.
  • Implement remote control-c handling support for parity in control-c behavior for daemon runs.
  • Consolidate, simplify and improve ProcessManager metadata API (and docs) and tease out a ProcessMetadataManager superclass for general @classmethod/key-oriented usage for metadata consumers/clients.
  • Improve cleanup speed of unreaped throwaway parent processes created via the ProcessManager double fork.
  • Implement conditional startup of the NailgunStreamReader thread in NailgunClient via honoring of the START_READING_INPUT chunk.
  • Improve unhandled exception output parity with non-daemon runs.
  • Improve process title entries for pantsd-spawned runs.
  • Eliminate all hardcoded buffer sizes in the Nailgun stack in favor of io.DEFAULT_BUFFER_SIZE.
  • Misc cleanups.
  • Tests - including a new closed-loop integration test for pantsd.

extensive manual testing using the cli w/ --enable-pantsd for various scenarios +

CI is green @ https://travis-ci.org/pantsbuild/pants/builds/99477465


  • 0
  • 5
  • 0
  • 5
Description From Last Updated
Stu Hood
Patrick Lawson
Benjy Weinberger
Nick Howard (Twitter)
Kris Wilson
Nick Howard (Twitter)
Kris Wilson
Review request changed

Status: Closed (submitted)

Change Summary:

thanks Stu, Patrick, Benjy & Nick! this is in @ https://github.com/pantsbuild/pants/commit/84b8f96b05155e134b3bc979c53a5c3845dab689