Fix ivy_resolve message: Missing expected ivy output file .../.ivy2/pants/internal-...-default.xml
Review Request #2015 - Created March 31, 2015 and submitted
|jsirois, patricklaw, stuhood|
This change forces the exec_ivy() to run if the report is not present. Sometimes the
targets can all be up to date, but that combination of targets
has not been run through ivy.
This is important because ivy_resolve now depends on
having a valid report from the ivy run to compute the classpath correctly.
The change to use UnionProducts and a more limited compile classpath in https://rbcommons.com/s/twitter/r/1761/
relies on parsing the ivy report. I added some more error checking in https://rbcommons.com/s/twitter/r/1856/
which started tripping in our build.
Passed CI @ https://travis-ci.org/pantsbuild/pants/builds/56629304
We got this error message 7 times re-compiling all java_binary() targets in our repo with about 200 targets.
We always run through the targets in alphabetical order: the error starts about 30 minutes into the build after the 80th target.
I created a number of test targets that referenced the same jars in differenc combinations, but was unable to reproduce this outside of our repo.
However, I did put in some debugging before and after exec_ivy() in ivy_resolve
to show what was going on:INFO] Pre exec_ivy() xml report path=/Users/square/.ivy2/pants/internal-61e19aa02fd5ea667f0e39802a333fa1cd4a74a7-default.xml exists=False INFO] All invalidation targets up to date. Skipped execing ivy. INFO] Post exec_ivy() xml report path=/Users/square/.ivy2/pants/internal-61e19aa02fd5ea667f0e39802a333fa1cd4a74a7-default.xml exists=False
Does the 200 target build you reference delete parts of /Users/square/.ivy2/pants ? If it doesn't it seems like the only explanation can be pants deleting them somehow - which would be clearly un-intended. Either way - if you agree the only way this can happen is if the ivy cache is independently mutated, that gives the crease for the unit test. Just have the test use a custom PANTS_IVY_CACHE_DIR that it can do selected out-of-band nukes in.
Update comment to capture info on the bug
Revision 2 (+12 -1)
A test would still be nice to have since this does now guard against external deletion - root cause or not - and that's a nice feature to preserve since users do in fact muck in their cache often enough.