Remove Duplicates in File System tasks in v2 Engine

Review Request #4096 - Created July 19, 2016 and submitted

Yujie Chen
kwlzn, stuhood

Currently when performing file system tasks in v2 Engine (like glob matching), we keep duplicate paths.
This can cause errors sometimes. For example, if 2 glob patterns defined in "sources" field of a target both match a common scala file, this file will be given twice during compilation, thus triggering errors like "X is already defined as class X".
In fact, we do not have any use cases where keeping duplicate is a requirement, however we do have use cases like the above where removing duplicates can save us a lot of trouble. In addition, v1 engine does not have duplicate paths, which will make transition from v1 to v2 engine smoother.

In this review:
1. Enhanced logic in merge_paths ( to remove duplicates while preserving the order of paths.
2. Added a test case.

ci green:

Kris Wilson
Stu Hood
Ity Kaul
Kris Wilson
Yujie Chen
Review request changed

Status: Closed (submitted)

Change Summary:

thanks, Kris, Stu and Ity for reviewing!