Enabling publishing of java and scala sources in one jar.

Review Request #225 — Created April 15, 2014 and submitted

tejal
pants
53
pants-reviews
benjyw, jsirois, travis

tests.python.example_test.hello.greet.greet                                     .....   SUCCESS
tests.python.pants_test.base.abbreviate_target_ids                              .....   SUCCESS
tests.python.pants_test.base.address                                            .....   SUCCESS
tests.python.pants_test.base.build_file                                         .....   SUCCESS
tests.python.pants_test.base.build_invalidator                                  .....   SUCCESS
tests.python.pants_test.base.build_root                                         .....   SUCCESS
tests.python.pants_test.base.double_dag                                         .....   SUCCESS
tests.python.pants_test.base.generator                                          .....   SUCCESS
tests.python.pants_test.base.hash_utils                                         .....   SUCCESS
tests.python.pants_test.base.parse_context                                      .....   SUCCESS
tests.python.pants_test.base.revision                                           .....   SUCCESS
tests.python.pants_test.base.run_info                                           .....   SUCCESS
tests.python.pants_test.cache.cache                                             .....   SUCCESS
tests.python.pants_test.commands.test_goal                                      .....   SUCCESS
tests.python.pants_test.commands.test_setup_py                                  .....   SUCCESS
tests.python.pants_test.engine.test_engine                                      .....   SUCCESS
tests.python.pants_test.engine.test_group_engine                                .....   SUCCESS
tests.python.pants_test.fs.fs                                                   .....   SUCCESS
tests.python.pants_test.java.distribution.distribution                          .....   SUCCESS
tests.python.pants_test.java.jar.jar                                            .....   SUCCESS
tests.python.pants_test.net.http.http                                           .....   SUCCESS
tests.python.pants_test.process.process                                         .....   SUCCESS
tests.python.pants_test.python.test_antlr_builder                               .....   SUCCESS
tests.python.pants_test.python.test_resolver                                    .....   SUCCESS
tests.python.pants_test.python.test_thrift_builder                              .....   SUCCESS
tests.python.pants_test.python.test_thrift_namespace_packages                   .....   SUCCESS
tests.python.pants_test.reporting.reporting                                     .....   SUCCESS
tests.python.pants_test.scm.test_git                                            .....   SUCCESS
tests.python.pants_test.targets.artifact                                        .....   SUCCESS
tests.python.pants_test.targets.bundle                                          .....   SUCCESS
tests.python.pants_test.targets.exclusive                                       .....   SUCCESS
tests.python.pants_test.targets.internal                                        .....   SUCCESS
tests.python.pants_test.targets.jar_library                                     .....   SUCCESS
tests.python.pants_test.targets.pants_target                                    .....   SUCCESS
tests.python.pants_test.targets.python_binary                                   .....   SUCCESS
tests.python.pants_test.targets.python_target                                   .....   SUCCESS
tests.python.pants_test.targets.target                                          .....   SUCCESS
tests.python.pants_test.targets.util                                            .....   SUCCESS
tests.python.pants_test.tasks.binary_create                                     .....   SUCCESS
tests.python.pants_test.tasks.builddict                                         .....   SUCCESS
tests.python.pants_test.tasks.bundle_create                                     .....   SUCCESS
tests.python.pants_test.tasks.cache_manager                                     .....   SUCCESS
tests.python.pants_test.tasks.check_exclusives                                  .....   SUCCESS
tests.python.pants_test.tasks.check_published_deps                              .....   SUCCESS
tests.python.pants_test.tasks.config                                            .....   SUCCESS
tests.python.pants_test.tasks.console_task                                      .....   SUCCESS
tests.python.pants_test.tasks.context                                           .....   SUCCESS
tests.python.pants_test.tasks.dependees                                         .....   SUCCESS
tests.python.pants_test.tasks.dependencies                                      .....   SUCCESS
tests.python.pants_test.tasks.depmap                                            .....   SUCCESS
tests.python.pants_test.tasks.filemap                                           .....   SUCCESS
tests.python.pants_test.tasks.filter                                            .....   SUCCESS
tests.python.pants_test.tasks.ivy_utils                                         .....   SUCCESS
tests.python.pants_test.test_maven_layout                                       .....   SUCCESS
tests.python.pants_test.test_thrift_util                                        .....   SUCCESS
Local: yes!  Travis ci is broken

Added a java_sources for scala_lib example 

[tw-172-25-145-95 pants]$ git diff
diff --git a/src/scala/com/pants/example/BUILD b/src/scala/com/pants/example/BUILD
index 6b607d7..f0c5b2e 100644
--- a/src/scala/com/pants/example/BUILD
+++ b/src/scala/com/pants/example/BUILD
@@ -3,7 +3,11 @@
 
 scala_library(name = 'jvm-run-example-lib',
   dependencies = [],
-  sources = ['JvmRunExample.scala']
+  sources = ['JvmRunExample.scala'],
+  java_sources = [pants('src/java/com/pants/examples/pingpong/handler')],
+  provides=artifact(org='com.pants.common.example',
+                    name='scala_lib',
+                    repo = 'build-support/ivy:public',),
 )
 
 jvm_binary(name ='jvm-run-example',
[tw-172-25-145-95 pants]$ 

1. Run Publish

[tw-172-25-145-95 pants]$ PANTS_DEV=1 ./pants goal publish src/scala/com/pants/example:jvm-run-example-lib --publish-local=~tdesai/m2/ --no-publish-dryrun --no-ng-daemon
*** Running pants in dev mode from src/python/pants/bin/pants_exe.py ***
....
Changes for com.pants.common.example#scala_lib since 0.0.0 @ None:

commit ce644e165b503d920c81ca9f8d1f35966c9a252f
Author: Benjy <benjy@foursquare.com>
Date:   Wed Apr 2 23:59:20 2014 +0000

    Make ./pants run work for scala.
    
    Unclear how (or even if) this ever worked before.

 src/scala/com/pants/example/JvmRunExample.scala | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

Publish com.pants.common.example#scala_lib with revision 0.0.1-SNAPSHOT ? [y|N] y



00:06:26 00:02       [jar-publish]
               SUCCESS

2. Making sure, only scala_lib.jar is created and pingpong.jar is not created for the  java_sources target.
[tw-172-25-145-95 pants]$ ls ~tdesai/m2/com/pants/common/example/
scala_lib

3. Verify, scala_lib.jar has the PingHandler.class.
[tw-172-25-145-95 pants]$ unzip -l  ~tdesai/m2/com/pants/common/example/scala_lib/0.0.1-SNAPSHOT/scala_lib-0.0.1-SNAPSHOT.jar
Archive:  /Users/tdesai/m2/com/pants/common/example/scala_lib/0.0.1-SNAPSHOT/scala_lib-0.0.1-SNAPSHOT.jar
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  01-01-80 00:00   com/
        0  01-01-80 00:00   com/pants/
        0  01-01-80 00:00   com/pants/example/
     1098  04-15-14 00:06   com/pants/example/JvmRunExample$.class
      732  04-15-14 00:06   com/pants/example/JvmRunExample.class
        0  01-01-80 00:00   com/pants/examples/
        0  01-01-80 00:00   com/pants/examples/pingpong/
        0  01-01-80 00:00   com/pants/examples/pingpong/handler/
     2313  04-15-14 00:06   com/pants/examples/pingpong/handler/PingHandler.class
 --------                   -------
     4143                   9 files

4. Verified the generated pom.xml has dependency of java_sources target src/java/com/pants/examples/pingpong/handler/BUILD/ 
[tw-172-25-145-95 pants]$ cat src/java/com/pants/examples/pingpong/handler/BUILD 
# Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).


java_library(name = 'handler',
  dependencies = [
    pants('3rdparty:guava'),
    pants('3rdparty:guice'),
    pants('3rdparty/jvm/com/sun/jersey:jersey-core'),
    pants('3rdparty/jvm/com/twitter/common:base'),
    pants('3rdparty/jvm/com/twitter/common:stats'),
  ],
...
[tw-172-25-145-95 pants]$ 
[tw-172-25-145-95 pants]$ cat  ~tdesai/m2/com/pants/common/example/scala_lib/0.0.1-SNAPSHOT/scala_lib-0.0.1-SNAPSHOT.pom
<?xml version="1.0" encoding="UTF-8"?>

<!--
Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).
Licensed under the Apache License, Version 2.0 (see LICENSE).
-->

<!-- generated by pants! -->
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>
  <groupId>com.pants.common.example</groupId>
  <artifactId>scala_lib</artifactId>
  <packaging>jar</packaging>
  <version>0.0.1-SNAPSHOT</version>

  <licenses>
    <license>
      <name>The Apache Software License, Version 2.0</name>
      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
      <distribution>repo</distribution>
    </license>
  </licenses>

  <dependencies>
    <dependency>
      <groupId>com.google.guava</groupId>
      <artifactId>guava</artifactId>
      <version>15.0</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>com.google.code.findbugs</groupId>
      <artifactId>jsr305</artifactId>
      <version>1.3.9</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>javax.inject</groupId>
      <artifactId>javax.inject</artifactId>
      <version>1</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>com.google.inject</groupId>
      <artifactId>guice</artifactId>
      <version>3.0</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-core</artifactId>
      <version>1.12</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>com.twitter.common</groupId>
      <artifactId>base</artifactId>
      <version>0.0.85</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>com.twitter.common</groupId>
      <artifactId>stats</artifactId>
      <version>0.0.90</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>
</project>
Loading file attachments...

TE
JS
  1. Please don't merge until the red CI issue is fixed.
  2. src/python/pants/tasks/jar_create.py (Diff revision 2)
     
     
    In isolation this still creates 2 jars with java sources.  When combined with publish, there is a fail-fast because of the check you added.  So this is fine in practice, but a TODO here to only place the java sources in one jar or the other is in order.  In other words it would be best here to have a single combined classfile jar with java and scala classes, and a single combined sourcefile jar with java and scala code.  As it stands we get pairs of each when some runs ./pants goal jar ...
    1. Yes you are right. 
      Will add a TODO for to handle this in jar goal.
      
      Also i might have to change the ivy.mustache since AFAIR mapped={{.}} it was not working for publish. 
      So cant use this ship it. 
      
      
  3. 
      
TE
TE
TE
  1. 
      
  2. src/python/pants/tasks/jar_publish.py (Diff revision 4)
     
     
    Currently repo is set for local publish with confs "default" See line number 363.
    
    If i use the flag --publish-jar_create-sources, the conf still points to ["default"] hence, the source jar does not get published.
    
    Hence i added this. 
    Now that i read this loud, I also feel if --no-publish-jar_create-sources is specified, i should remove it from confs.
    
    What do you guys think?
     
    
    
  3. src/python/pants/tasks/scaladoc_gen.py (Diff revision 4)
     
     
    This is to make sure, the docs for java_sources are also included, 
    
  4. 
      
TE
BE
  1. Hey Tejal, have you been following the thread about jar_sources on pants-devel? What do you think about re-implementing it that way? Would that be feasible in a short amount of time? The issue with this change is that there's a lot of special-casing going on, and a lot of logic that we'll have to later undo. 
  2. 
      
TE
JS
  1. 
      
  2. src/python/pants/tasks/jar_publish.py (Diff revisions 2 - 5)
     
     
    s/set (/set(/
  3. please kill the print
  4. 
      
TE
TE
Review request changed

Status: Closed (submitted)

Change Summary:

merged
Loading...