Fixup doc publishing to work under pantsbuild/pants.

Review Request #305 — Created May 1, 2014 and submitted

jsirois
pants
docs/contributors
98
pants-reviews
benjyw, lahosken, patricklaw, tejal, travis
commit 0efa643dfdd347e74206629ab4fa483ec9033475
Author: John Sirois <jsirois@twitter.com>
Date:   Thu May 1 14:37:52 2014 -0600

    Fixup doc publishing to work under pantsbuild/pants.
    
    Introduce a central driver script to replace the manual process
    and leverage the pants venv for the needed 3rdparty packages.

 3rdparty/python/dev_only_requirements.txt |  6 ++++
 build-support/bin/publish_docs.sh         | 74 ++++++++++++++++++++++++++++++++++++++++++++++++
 pants => build-support/pants_venv         | 58 +++++++++++++------------------------
 pants                                     | 77 ++------------------------------------------------
 src/python/pants/docs/conf.py             | 10 -------
 src/python/pants/docs/docs.rst            | 50 ++++++++++++--------------------
 src/python/pants/docs/publish_via_git.sh  |  2 +-
 7 files changed, 121 insertions(+), 156 deletions(-)
Re-ran pants bootstrap vanilla and PANTS_DEV=1 to confirm that was not broken by the shuffle.

Then for the new stuff:


Usage
=====
$ ./build-support/bin/publish_docs.sh -h
Publishes the http://pantsbuild.github.io/ docs locally or remotely.

Usage: ./build-support/bin/publish_docs.sh (-h|-opd)
 -h           print out this help message
 -o           open the doc site locally
 -s           publish the doc site remotely
 -d           publish the site to a subdir at this path (useful for public previews)


Hermeticity
===========
# no sphinx - no pants
$ pip uninstall sphinx sphinx_rtd_theme
$ pip list | grep sphinx
$ git clean -fdx

# go
$ ./build-support/bin/publish_docs.sh -opd sirois-test-site
...
*** Running pants in dev mode from /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/bin/pants_exe.py ***

14:45:55 00:00 [main]
               (To run a reporting server: ./pants goal server)
14:45:55 00:00   [setup]
14:45:55 00:00     [bootstrap]
14:45:55 00:00     [parse]
14:45:55 00:00   [builddict]
14:45:55 00:00     [builddict]
                   Generating /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/goals_reference.rst
                   Generating /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/pants_ini_reference.rst
                   Generating /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/build_dictionary.rst
               SUCCESS
Copying '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/build_dictionary.rst' to '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs'
Copying '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/goals_reference.rst' to '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs'
Copying '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/pants_ini_reference.rst' to '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs'
Writing to file '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/targets.rst'
Writing to file '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/base.rst'
rm -rf _build/*
sphinx-build -b html -d _build/doctrees   . _build/html
Making output directory...
Running Sphinx v1.2.2
loading pickled environment... not yet created
building [html]: targets for 34 source files that are out of date
updating environment: 34 added, 0 changed, 0 removed
reading sources... [100%] with_intellij                                                                                                                                                                                                                                           
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:89: WARNING: Include file u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/java/com/pants/examples/pingpong/main/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:100: WARNING: Include file u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/java/com/pants/examples/pingpong/handler/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:119: WARNING: Include file u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/java/com/pants/examples/pingpong/handler/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:126: WARNING: Include file u'/home/jsirois/dev/3rdparty/3rdparty/jvm/com/sun/jersey/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/pants_ini_reference.rst:16: WARNING: malformed hyperlink target.
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/pants_ini_reference.rst:29: WARNING: malformed hyperlink target.
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/tshoot.rst:39: ERROR: Unexpected indentation.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] with_intellij                                                                                                                                                                                                                                            
writing additional files... (83 module code pages) _modules/index genindex py-modindex search
copying images... [100%] images/intellij-project-structure-project.png                                                                                                                                                                                                            
copying static files... WARNING: html_static_path entry u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/_static' does not exist
done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded, 8 warnings.

Build finished. The HTML pages are in _build/html.
To abort publishing these docs to http://pantsbuild.github.io/sirois-test-site press CTRL-C, otherwise press enter to continue.Created new window in existing browser session.
To abort publishing these docs to http://pantsbuild.github.io/sirois-test-site press CTRL-C, otherwise press enter to continue.
Cloning into 'pantsbuild.github.io'...
remote: Counting objects: 4374, done.
remote: Compressing objects: 100% (706/706), done.
remote: Total 4374 (delta 3938), reused 4024 (delta 3641)
Receiving objects: 100% (4374/4374), 7.60 MiB | 684.00 KiB/s, done.
Resolving deltas: 100% (3938/3938), done.
Checking connectivity... done.
[master 2045c2b] publish by jsirois
...
 create mode 100644 sirois-test-site/with_intellij.html
Counting objects: 10, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 50.88 KiB | 0 bytes/s, done.
Total 8 (delta 1), reused 0 (delta 0)
To git@github.com:pantsbuild/pantsbuild.github.io.git
   1ff2f17..2045c2b  master -> master
Created new window in existing browser session.



See the results here: http://pantsbuild.github.io/sirois-test-site/docs.html
  • 0
  • 0
  • 1
  • 0
  • 1
Description From Last Updated
JS
  1. 
      
  2. build-support/pants_venv (Diff revision 1)
     
     
    I'm not sure why this did not pick up as a move, but it is a stright-up move of ~90% of the pants script with s/${HERE}/${REPO_ROOT}/ in a few spots that need to reference paths stably.
  3. 
      
JS
LA
  1. When I look at go/pants , I think that this did not pick up our customized template file.
    
    The 'make html' command should use the file _templates/layout.html to do things like put our Travis-CI badge on the page.
    
    If that makes you say "oh, now that I know of the existence of this helper file, I just need to do this one quick thing to get it into the venv-magic-thingy", then great.
    
    If that makes you say "oh it will not be so straightforward", then... I think you should ship this anyhow. A sane doc-generation process that doesn't require mysterious installs would be sweeet.
    1. Still noodling around on this one.  Things haven't obviously changed from the manual process.
    2. I notice some churn out on the sphinx_rtd_theme repo.  This change uses latest - unconstrained versions - for sphinx* so I may just need to pin old.
      What do you have for this?
      $ pip list | grep -i sphinx
      Sphinx (1.2.2)
      sphinx-rtd-theme (0.1.6)
      
    3. oho
      
      $ pip list | grep -i sphinx
      Sphinx (1.2)
      sphinx-rtd-theme (0.1.5)
      
    4. Thanks you for that - gold! The 0.1.5 -> 0.1.6 is exactly what breaks the layout template.  So I've pinned that and constrained Sphinx in the 1.2 series for now.
      
      I just took the liberty of burning a publish with this fix in-hand since I did before.  You should note the main docsite is restored to travis-ci green badge glory.
  2. build-support/bin/publish_docs.sh (Diff revision 2)
     
     
    when I look a few lines down, I think this is -p
    1. Good catch - fixed.
  3. 
      
JS
JS
JS
  1. 
      
  2. build-support/bin/publish_docs.sh (Diff revision 4)
     
     
    no soup for you! s/$/&/
  3. 
      
JS
JS
  1. Patched into master - I assumed the shipit stood here.
  2. 
      
JS
Review request changed

Status: Closed (submitted)

Loading...