Incremental project import for controlled indexing

Review Request #4032 — Created June 28, 2016 and submitted — Latest diff uploaded

4181, 4186, 4030
benjyw, nhoward_tw, peiyu, stuhood, zundel

This change allows user to incrementally import a large project to minimize the time blocked by indexing before iterating on code or running tests.

1. At import stage, "Enable Incremental Project Import" option is added.
2. If selected, GUI will ask user how many levels of build graph to import given the max level. level 0 means target roots, level 1 means up to direct deps, so on and so forth.
3. If user wants to incrementally import more of the project, simply click 'Pants -> refresh project', then the dialog in step 2 will reappear once pants export is done.

level 0: shows Distances is not resolved.
level 1: shows Distances is resolved.

Sample stats for two internal projects regarding time saved before user can iterate:
Project 1: The indexing time is down from 590s to 346s (41% saving) with level 2 (total 8 levels)
Project 2: The indexing time is down from 350s to 160s (52% saving) with level 1 (total 6 levels)

Next step:
Currently IntelliJ does not recognize some of the test classes because their junit dependency is too far, so user has to trial and error to figure out the ideal depth to import, but we can optimize that down the road given

  • Junit will be injected into junit_test() target
  • Include all transitive junit dependees during import

Other minor changes:
* Pants cache invalidate action and Refresh project are now DumbAware, meaning they can be clicked while project is still indexing.

Loading file attachments...