[engine] using memoized to avoid repeated keying and unpickling

Review Request #3597 - Created March 23, 2016 and discarded

Information
Peiyu Wang
pants
3066, 3078
Reviewers
pants-reviews
kwlzn, patricklaw, stuhood

This is yet another low hanging fruit but effective performance
improvement in Storage.

In my manual testing list 3rdparty::, improvement for cold cache is
about 50%, for fully warmed-up cache, it's sightly slower because of the
extra memoized check cost.

Using memoized is only for now, there are various lru_cache
implementations for py2 that we could use later, like [1] and [2]
or just switch to py3.

[1] https://github.com/repoze/repoze.lru
[2] https://pypi.python.org/pypi/functools32

http://jenkins.pantsbuild.org/job/pants_ci.unit_tests/140/ passed

Issues

  • 3
  • 0
  • 0
  • 3
Description From Last Updated
It should not be necessary to check the type in equality, because the digested blob includes the type (or does, ... Stu Hood Stu Hood
This method will key by the entire step, rather than keying by the keyable fields. But memoizing this method probably ... Stu Hood Stu Hood
Rather than using memoized here, you can catch a few more cases by doing this manually with a dict, and ... Stu Hood Stu Hood
Peiyu Wang
Stu Hood
Stu Hood
Peiyu Wang
Review request changed

Status: Discarded

Loading...