Fixup minified_dependencies.

Review Request #1986 - Created March 25, 2015 and submitted

Information
John Sirois
pants
jsirois/issues/1126/fix_minified_dependencies
1126, 1326
0fd7e95...
Reviewers
pants-reviews
jinfeng, stuhood, wickman
This change fixes the core of the SetupPy minified_dependencies and adds
a hack dependency to each of pants 3 auxiliary sdist targets to produce
correct sdists with no repeated code and an exact set of setup_requires.

The SetupPy test is moved to its proper parallel test tree home and a
test for the pantsbuild.pants contrib sdist case is added that fails
before the fix.

A follow-on change will remove the need for the 3 hack deps by imposing
a well defined global ownership model for python targets that don't
provide - tracked here: https://github.com/pantsbuild/pants/issues/1329

 contrib/cpp/src/python/pants/contrib/cpp/BUILD                            |   3 +
 contrib/release_packages.sh                                               |   8 +--
 contrib/scrooge/src/python/pants/contrib/scrooge/BUILD                    |   3 +
 src/python/pants/BUILD                                                    |   3 +
 src/python/pants/backend/python/tasks/setup_py.py                         | 134 +++++++++++++++++++---------------------
 tests/python/pants_test/backend/python/tasks/BUILD                        |  26 +++++++-
 tests/python/pants_test/{python => backend/python/tasks}/test_setup_py.py | 166 ++++++++++++++++++++++++++++++++++----------------
 tests/python/pants_test/python/BUILD                                      |  19 ------
 8 files changed, 210 insertions(+), 152 deletions(-)

Manual testing included 1st verifying the auxialiary sdists now only
contained the source files they owned and not the transitive set of
pants files they used:

$ ./build-support/bin/release.sh -n
$ for tar in dist/*.tar.gz; do tar -xzf $tar -C dist/; done
$ cat dist/pantsbuild.pants.contrib.scrooge-0.0.31/src/pantsbuild.pants.contrib.scrooge.egg-info/namespace_packages.txt 
pants
pants.contrib
$ cat dist/pantsbuild.pants.contrib.scrooge-0.0.31/src/pantsbuild.pants.contrib.scrooge.egg-info/requires.txt 
pantsbuild.pants==0.0.31
twitter.common.collections>=0.3.1,<0.4
$ cat dist/pantsbuild.pants.contrib.scrooge-0.0.31/src/pantsbuild.pants.contrib.scrooge.egg-info/SOURCES.txt 
MANIFEST.in
setup.py
src/pants/__init__.py
src/pants/contrib/__init__.py
src/pants/contrib/scrooge/__init__.py
src/pants/contrib/scrooge/register.py
src/pants/contrib/scrooge/tasks/__init__.py
src/pants/contrib/scrooge/tasks/scrooge_gen.py
src/pants/contrib/scrooge/tasks/thrift_linter.py
src/pantsbuild.pants.contrib.scrooge.egg-info/PKG-INFO
src/pantsbuild.pants.contrib.scrooge.egg-info/SOURCES.txt
src/pantsbuild.pants.contrib.scrooge.egg-info/dependency_links.txt
src/pantsbuild.pants.contrib.scrooge.egg-info/namespace_packages.txt
src/pantsbuild.pants.contrib.scrooge.egg-info/requires.txt
src/pantsbuild.pants.contrib.scrooge.egg-info/top_level.txt
src/pantsbuild.pants.contrib.scrooge.egg-info/zip-safe

Same went for pantsbuild.pants.testinfra.

I also did the twitter/commons sdists release prep with the
./build-support/commons/scripts/build_py_commons.sh script before
and after and found all sdist diffs were 3rdparty dep additions like
so:

$ diff -rx *.tar.gz old/ new/
diff -rx '*.tar.gz' old/twitter.common.app-0.3.3/setup.py new/twitter.common.app-0.3.3/setup.py
13a14
>                             'twitter.common.lang==0.3.3',
14a16
>                             'twitter.common.options==0.3.3',
diff -rx '*.tar.gz' old/twitter.common.app-0.3.3/src/twitter.common.app.egg-info/requires.txt new/twitter.common.app-0.3.3/src/twitter.common.app.egg-info/requires.txt
1a2
> twitter.common.lang==0.3.3
2a4
> twitter.common.options==0.3.3
diff -rx '*.tar.gz' old/twitter.common.concurrent-0.3.3/setup.py new/twitter.common.concurrent-0.3.3/setup.py
14a15
>                             'twitter.common.lang==0.3.3',
diff -rx '*.tar.gz' old/twitter.common.concurrent-0.3.3/src/twitter.common.concurrent.egg-info/requires.txt new/twitter.common.concurrent-0.3.3/src/twitter.common.concurrent.egg-info/requires.txt
2a3
> twitter.common.lang==0.3.3
...

This makes sense because one change to the dep minification algorithm
was to acheive an exact set of setup_requires and not the old minified
set. Its only the local source files retained in the sdist that are
minified.

CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/55903062

John Sirois
Stu Hood
John Sirois
John Sirois
John Sirois
John Sirois
Review request changed

Status: Closed (submitted)

Loading...