[llvm-dev] ThinLTO caching & compression

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] ThinLTO caching & compression

Jonas Paulsson via llvm-dev

Hi,

 

I was wondering if there were plans to add compression to the ThinLTO cache?

 

If no, what would be a good solution? Use llvm/Support/Compression.h? Integrate any other compression algorithm? (it’d be good if it could work out-of-the-box)

Support HTTP to take advantage of the compression?

Allow for plugins to handle the cache streams? (that would be great, we wouldn’t need any post-processing steps to store the files in our object store)

 

Also side-question: I was wondering if cache files can be shared across a team? (given relative paths in the compilation cmd-lines)

 

Thanks!

Alex.


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] ThinLTO caching & compression

Jonas Paulsson via llvm-dev
Should be shareable - don't think anyone's working on compression, could probably be implemented separately in your distributed storage layer?

On Sat, Jan 4, 2020 at 9:44 AM Alexandre Ganea via llvm-dev <[hidden email]> wrote:

Hi,

 

I was wondering if there were plans to add compression to the ThinLTO cache?

 

If no, what would be a good solution? Use llvm/Support/Compression.h? Integrate any other compression algorithm? (it’d be good if it could work out-of-the-box)

Support HTTP to take advantage of the compression?

Allow for plugins to handle the cache streams? (that would be great, we wouldn’t need any post-processing steps to store the files in our object store)

 

Also side-question: I was wondering if cache files can be shared across a team? (given relative paths in the compilation cmd-lines)

 

Thanks!

Alex.

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] ThinLTO caching & compression

Jonas Paulsson via llvm-dev
+pcc who implemented the caching for questions on compression. I focus more on ThinLTO in distributed build mode, which doesn't utilize the built in caching.
But yes, the cache files should be shareable. 

On Sat, Jan 4, 2020 at 10:25 AM David Blaikie <[hidden email]> wrote:
Should be shareable - don't think anyone's working on compression, could probably be implemented separately in your distributed storage layer?

On Sat, Jan 4, 2020 at 9:44 AM Alexandre Ganea via llvm-dev <[hidden email]> wrote:

Hi,

 

I was wondering if there were plans to add compression to the ThinLTO cache?

 

If no, what would be a good solution? Use llvm/Support/Compression.h? Integrate any other compression algorithm? (it’d be good if it could work out-of-the-box)

Support HTTP to take advantage of the compression?

Allow for plugins to handle the cache streams? (that would be great, we wouldn’t need any post-processing steps to store the files in our object store)

 

Also side-question: I was wondering if cache files can be shared across a team? (given relative paths in the compilation cmd-lines)

 

Thanks!

Alex.

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


--
Teresa Johnson | Software Engineer | [hidden email] |

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev