Fix cache_setup.py so build doesn't fail if configured cache is empty.

Review Request #3142 - Created Nov. 17, 2015 and submitted

Information
Tansy Arron-Walker
pants
2320, 2585
Reviewers
pants-reviews
jsirois, patricklaw, stuhood, zundel

Add test for read_cache_available

Add test for write_cache_available()

Move delay_handler into helper modulefix task.py do_check_artifact_cache. No longer tries to access read_cache when read_cache is None

mark test_scalastyle_integration and test_checkstyle_integration tests to fail.

CI is running: https://travis-ci.org/pantsbuild/pants/builds/92122014

ran ./pants clean-all && ./pants compile src/scala/org/pantsbuild/zinc:: --cache-bootstrap-read-from='["http://localhost:9999"]'

confirmed that the build failed:

INFO] Detected git repository at /Users/tansy/code/pants on branch master

17:03:51 00:00 [main]
               (To run a reporting server: ./pants server)
17:03:51 00:00   [bootstrap]
17:03:51 00:00   [setup]
17:03:51 00:00     [parse]
               Executing tasks in goals: clean-all
17:03:51 00:00   [clean-all]
17:03:51 00:00     [ng-killall]INFO] killing nailgun server pid=7455
INFO] killing nailgun server pid=7462
INFO] killing nailgun server pid=7470
INFO] killing nailgun server pid=7485

17:03:52 00:01     [invalidate]
17:03:52 00:01     [clean-all]
17:03:53 00:02   [complete]
               SUCCESS
INFO] Detected git repository at /Users/tansy/code/pants on branch master

17:03:54 00:00 [main]
               (To run a reporting server: ./pants server)
17:03:54 00:00   [bootstrap]
17:03:54 00:00   [setup]
17:03:54 00:00     [parse]
               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> jvm-platform-validate -> deferred-sources -> gen -> resolve -> resources -> compile
17:03:54 00:00   [bootstrap]
17:03:54 00:00     [bootstrap-jvm-tools]
17:03:54 00:00   [imports]
17:03:54 00:00     [ivy-imports]
17:03:54 00:00   [unpack-jars]
17:03:54 00:00     [unpack-jars]
17:03:54 00:00     [unpack-libs]
17:03:54 00:00   [jvm-platform-validate]
17:03:54 00:00     [jvm-platform-validate]
                   Invalidated 5 targets.
17:03:54 00:00   [deferred-sources]
17:03:54 00:00     [deferred-sources]
17:03:54 00:00   [gen]
17:03:54 00:00     [thrift]
17:03:54 00:00     [protoc]
17:03:54 00:00     [antlr]
17:03:54 00:00     [ragel]
17:03:54 00:00     [jaxb]
17:03:54 00:00     [wire]
17:03:54 00:00     [aapt]
17:03:54 00:00     [scrooge]
17:03:54 00:00     [spindle]
17:03:54 00:00   [resolve]
17:03:54 00:00     [ivy]
17:03:54 00:00       [cache]
                     No reachable artifact caches.
17:03:54 00:00   [complete]
               FAILURE

Exception caught: (<type 'exceptions.AttributeError'>)
  File "/Users/tansy/code/pants/src/python/pants/bin/pants_exe.py", line 32, in <module>
    main()
  File "/Users/tansy/code/pants/src/python/pants/bin/pants_exe.py", line 26, in main
    LocalPantsRunner(exiter).run()
  File "/Users/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 60, in run
    self._maybe_profiled(self._run)
  File "/Users/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 57, in _maybe_profiled
    runner()
  File "/Users/tansy/code/pants/src/python/pants/bin/pants_runner.py", line 90, in _run
    result = goal_runner.run()
  File "/Users/tansy/code/pants/src/python/pants/bin/goal_runner.py", line 353, in run
    result = self._execute_engine()
  File "/Users/tansy/code/pants/src/python/pants/bin/goal_runner.py", line 342, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/Users/tansy/code/pants/src/python/pants/engine/engine.py", line 26, in execute
    self.attempt(context, goals)
  File "/Users/tansy/code/pants/src/python/pants/engine/round_engine.py", line 218, in attempt
    goal_executor.attempt(explain)
  File "/Users/tansy/code/pants/src/python/pants/engine/round_engine.py", line 47, in attempt
    task.execute()
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/ivy_resolve.py", line 84, in execute
    executor = self.create_java_executor()
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/nailgun_task.py", line 68, in create_java_executor
    classpath = os.pathsep.join(self.tool_classpath('nailgun-server'))
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/jvm_tool_task_mixin.py", line 42, in tool_classpath
    return self.tool_classpath_from_products(self.context.products, key, scope=self._scope(scope))
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/subsystems/jvm_tool_mixin.py", line 169, in tool_classpath_from_products
    return callback()
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/bootstrap_jvm_tools.py", line 302, in bootstrap_classpath
    cache['classpath'] = self._bootstrap_jvm_tool(dep_spec, jvm_tool)
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/bootstrap_jvm_tools.py", line 291, in _bootstrap_jvm_tool
    return self._bootstrap_classpath(jvm_tool, targets)
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/bootstrap_jvm_tools.py", line 207, in _bootstrap_classpath
    classpath, _, _ = self.ivy_resolve(targets, silent=True, workunit_name=workunit_name)
  File "/Users/tansy/code/pants/src/python/pants/backend/jvm/tasks/ivy_task_mixin.py", line 216, in ivy_resolve
    fingerprint_strategy=fingerprint_strategy) as invalidation_check:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/tansy/code/pants/src/python/pants/backend/core/tasks/task.py", line 352, in invalidated
    self.check_artifact_cache(self.check_artifact_cache_for(invalidation_check))
  File "/Users/tansy/code/pants/src/python/pants/backend/core/tasks/task.py", line 440, in check_artifact_cache
    return self.do_check_artifact_cache(vts)
  File "/Users/tansy/code/pants/src/python/pants/backend/core/tasks/task.py", line 458, in do_check_artifact_cache
    res = self.context.subproc_map(call_use_cached_files, items)
  File "/Users/tansy/code/pants/src/python/pants/goal/context.py", line 179, in subproc_map
    return res.get()
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 558, in get
    raise self._value

Exception message: 'NoneType' object has no attribute 'use_cached_files'

Wrote a test and confirmed it failed. Fixed the code, confirmend the test passed, ran the original command again and got a successful build with "No reachable artifact caches" warnings throughout:

$ ./pants clean-all && ./pants compile src/scala/org/pantsbuild/zinc:: --cache-bootstrap-read-from='["http://localhost:9999"]'
INFO] Detected git repository at /Users/tansy/code/pants on branch tansy.issue2320

17:06:09 00:00 [main]
               (To run a reporting server: ./pants server)
17:06:09 00:00   [bootstrap]
17:06:09 00:00   [setup]
17:06:09 00:00     [parse]
               Executing tasks in goals: clean-all
17:06:09 00:00   [clean-all]
17:06:09 00:00     [ng-killall]
17:06:09 00:00     [invalidate]
17:06:09 00:00     [clean-all]
17:06:09 00:00   [complete]
               SUCCESS
INFO] Detected git repository at /Users/tansy/code/pants on branch tansy.issue2320

17:06:10 00:00 [main]
               (To run a reporting server: ./pants server)
17:06:10 00:00   [bootstrap]
17:06:10 00:00   [setup]
17:06:10 00:00     [parse]
               Executing tasks in goals: bootstrap -> imports -> unpack-jars -> jvm-platform-validate -> deferred-sources -> gen -> resolve -> resources -> compile
17:06:10 00:00   [bootstrap]
17:06:10 00:00     [bootstrap-jvm-tools]
17:06:10 00:00   [imports]
17:06:10 00:00     [ivy-imports]
17:06:10 00:00   [unpack-jars]
17:06:10 00:00     [unpack-jars]
17:06:10 00:00     [unpack-libs]
17:06:10 00:00   [jvm-platform-validate]
17:06:10 00:00     [jvm-platform-validate]
                   Invalidated 5 targets.
17:06:10 00:00   [deferred-sources]
17:06:10 00:00     [deferred-sources]
17:06:10 00:00   [gen]
17:06:10 00:00     [thrift]
17:06:10 00:00     [protoc]
17:06:10 00:00     [antlr]
17:06:10 00:00     [ragel]
17:06:10 00:00     [jaxb]
17:06:10 00:00     [wire]
17:06:10 00:00     [aapt]
17:06:10 00:00     [scrooge]
17:06:10 00:00     [spindle]
17:06:10 00:00   [resolve]
17:06:10 00:00     [ivy]
                   No reachable artifact caches.
17:06:10 00:00       [bootstrap-nailgun-server]
                   Caching artifacts for 1 target.
                   Invalidated 5 targets.
17:06:11 00:01       [ivy-resolve]
17:06:12 00:02     [go]
17:06:12 00:02     [npm]
17:06:12 00:02   [resources]
17:06:12 00:02     [prepare]
17:06:12 00:02     [services]
17:06:12 00:02   [compile]
17:06:12 00:02     [compile]
17:06:12 00:02     [jvm]
17:06:12 00:02       [jvm-compilers]
17:06:12 00:02         [zinc-pre]
17:06:12 00:02         [zinc-prepare]
17:06:12 00:02           [isolation-zinc-pool-bootstrap]
17:06:12 00:02         [zinc-execute]
                       Invalidated 5 targets in 5 target partitions.
                       Compiling 2 zinc sources in 1 target (src/scala/sbt/compiler/javac:javac).
                       Compiling 1 zinc source in 1 target (src/scala/sbt/inc:inc).
17:06:12 00:02           [compile]

                       Compiling 2 zinc sources in 1 target (src/scala/org/pantsbuild/zinc/cache:cache).
                       Compiling 2 zinc sources in 1 target (src/scala/org/pantsbuild/zinc/logging:logging).
17:06:12 00:02           [compile]

17:06:12 00:02           [compile]

17:06:12 00:02           [compile]

                         No reachable artifact caches.
17:06:12 00:02             [bootstrap-scalac]
                         Caching artifacts for 1 target.
                         No reachable artifact caches.
17:06:13 00:03             [bootstrap-compiler-interface]
                         Caching artifacts for 1 target.
                         No reachable artifact caches.
17:06:13 00:03             [bootstrap-sbt-interface]
                         Caching artifacts for 1 target.
                         No reachable artifact caches.
                         Invalidated 1 target.
                         No reachable artifact caches.
17:06:14 00:04             [bootstrap-zinc]
                         Caching artifacts for 1 target.
                         No reachable artifact caches.
17:06:15 00:05             [bootstrap-jar-tool]
                         Caching artifacts for 1 target.
17:06:15 00:05             [jar-tool]
                         No reachable artifact caches.
17:06:17 00:07             [bootstrap-jarjar]
                         Caching artifacts for 1 target.
17:06:18 00:08             [shade-zinc]
                         Caching artifacts for 1 target.
17:07:05 00:55             [zinc]

17:07:05 00:55             [zinc]

17:07:05 00:55             [zinc]

17:07:05 00:55             [zinc]
                           [info] Compiling 2 Scala sources to /Users/tansy/code/pants/.pants.d/compile/jvm/zinc/src.scala.org.pantsbuild.zinc.cache.cache/eeb1e2549a66/classes...
                           [info] Compiling 2 Scala sources to /Users/tansy/code/pants/.pants.d/compile/jvm/zinc/src.scala.org.pantsbuild.zinc.logging.logging/d72b2003574d/classes...
                           [info] Compiling 2 Scala sources to /Users/tansy/code/pants/.pants.d/compile/jvm/zinc/src.scala.sbt.compiler.javac.javac/8f67c3241a01/classes...
                           [info] Compiling 1 Scala source to /Users/tansy/code/pants/.pants.d/compile/jvm/zinc/src.scala.sbt.inc.inc/b413330da1b6/classes...
                           [info] Compile success at Nov 17, 2015 5:07:10 PM [4.536s]
                           [info] Compile success at Nov 17, 2015 5:07:11 PM [5.268s]
                           [info] Compile success at Nov 17, 2015 5:07:11 PM [5.514s]
                           [info] Compile success at Nov 17, 2015 5:07:11 PM [5.852s]

                       Compiling 9 zinc sources in 1 target (src/scala/org/pantsbuild/zinc:zinc).
17:07:12 01:02           [compile]

17:07:12 01:02             [zinc]
                           [info] Compiling 9 Scala sources to /Users/tansy/code/pants/.pants.d/compile/jvm/zinc/src.scala.org.pantsbuild.zinc.zinc/d039ed176d7f/classes...
                           [warn] Class javax.annotation.Nullable not found - continuing with a stub.
                           [warn] /Users/tansy/code/pants/src/scala/org/pantsbuild/zinc/Compiler.scala:86: class ZincLocalJavaCompiler in package javac is deprecated: Backport of changes that should be available in 0.13.10
                           [warn]           new javac.ZincLocalJavaCompiler(jc)
                           [warn]                     ^
                           [warn] /Users/tansy/code/pants/src/scala/org/pantsbuild/zinc/Compiler.scala:170: object ZincPrivateAnalysis in package inc is deprecated: Temporary class used to work around an accidentally package-protected method.
                           [warn]         (ZincPrivateAnalysis.empty(incOptions.nameHashing), None)
                           [warn]          ^
                           [warn] /Users/tansy/code/pants/src/scala/org/pantsbuild/zinc/Settings.scala:164: method transactional in object ClassfileManager is deprecated: Use overloaded variant that takes additional logger argument, instead.
                           [warn]       ClassfileManager.transactional(backup.get)
                           [warn]                        ^
                           [warn] four warnings found
                           [info] Compile success at Nov 17, 2015 5:07:19 PM [6.999s]

17:07:19 01:09         [zinc-finalize]
17:07:19 01:09         [zinc-post]
17:07:19 01:09     [jvm-dep-check]
17:07:19 01:09     [checkstyle]
17:07:19 01:09     [scalastyle]
                   Invalidated 5 targets.
                   No reachable artifact caches.
17:07:19 01:09       [bootstrap-scalastyle]
                   Caching artifacts for 1 target.
17:07:19 01:09       [org.scalastyle.Main]
17:07:20 01:10     [cpp]
17:07:20 01:10     [go]
17:07:20 01:10     [python-eval]
17:07:20 01:10     [pythonstyle]
               Waiting for background workers to finish.
17:07:20 01:10   [complete]
               SUCCESS

Repeated testing with --write-to with the same results.

Eric Ayers
Patrick Lawson
Tansy Arron-Walker
Eric Ayers
Tansy Arron-Walker
Patrick Lawson
Tansy Arron-Walker
Eric Ayers
Eric Ayers
Tansy Arron-Walker
Review request changed

Status: Closed (submitted)

Loading...