[llvm-dev] Generating exported wasm functions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Generating exported wasm functions

Alberto Barbaro via llvm-dev
Using LLVM v5, my Cone compiler (https://github.com/jondgoodwin/cone) automatically directly generated all functions as named and exported for both wasm text and binary files (not using clang/lld etc, but using the API).

Upgrading to LLVM v7, generated wasm object files now fail, exhibiting markedly different behavior, behavior that varies between the text and binary files. For example, the binary wasm file (only) now automatically adds two import directives that were not there before. More importantly, function names, visible in the wat file, are stripped from the binary wasm file and are not exported. It seems as if LLVM webassembly codegen has acted as if the linker were at play and had defaulted to options like: -fvisibility=hidden, -W1, --import-memory etc.

Is there a way to programmatically (via the LLVM APIs) alter this default behavior, so that I can make public functions once again visible and exported?  Also, is there documentation somewhere that describes these and other breaking changes in the way that LLVM v7  now generates webassembly object files?

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Generating exported wasm functions

Alberto Barbaro via llvm-dev
Hi Jonathan,
A lot has changed with the wasm backend since LLVM v5! One of the reasons we remained an experimental target was that the interfaces (e.g. the text and binary formats, and the LLVM APIs) were not stable or well-documented. We have been investing a lot of work in these formats since then; for example LLVM's wasm object file format and lld support for static linking are now feature-complete and mostly documented (https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md). Our plan is to additionally document the C ABI and declare it (and the object file format) as "version 1" by the LLVM 8.0 release in hopes of keeping that version stable. LLVM's wasm text assembly format is currently under active development and I wouldn't consider the text format in either LLVM v7 or current trunk to be stable. We also hope to have some stability for it by LLVM 8.0.

With respect to the details of symbol visibility and exports, there are some details about that that we are currently hashing out, as we are currently designing and implementing dynamic linking to go with our static linking support. I think there wasm will always have some differences compared to the model used by e.g. ELF/COFF/MachO. One reason is that wasm includes the notion of having (a specified set of) symbols exported from a finally-linked binary; in wasm this is used to export a set of functions to the external environment (e.g. JavaScript). ELF has --export-dynamic but I think that's probably not enough control for the web use case.

We definitely do want to enable control over the visibility and export of public functions (at the source, IR, and linker level). For the IR I'd expect this would be reflected as attributes on functions or GlobalValues, which should be controllable via the LLVM APIs.

In any case we'd be interested in hearing about your feedback or use cases; most of the discussion about the conventions used by LLVM and related tools are on the GitHub repo we use for our docs: https://github.com/WebAssembly/tool-conventions/. Feel free to file an issue or chime in on an existing one.

Best,
-Derek

On Sat, Nov 17, 2018 at 2:04 AM Jonathan Goodwin via llvm-dev <[hidden email]> wrote:
Using LLVM v5, my Cone compiler (https://github.com/jondgoodwin/cone) automatically directly generated all functions as named and exported for both wasm text and binary files (not using clang/lld etc, but using the API).

Upgrading to LLVM v7, generated wasm object files now fail, exhibiting markedly different behavior, behavior that varies between the text and binary files. For example, the binary wasm file (only) now automatically adds two import directives that were not there before. More importantly, function names, visible in the wat file, are stripped from the binary wasm file and are not exported. It seems as if LLVM webassembly codegen has acted as if the linker were at play and had defaulted to options like: -fvisibility=hidden, -W1, --import-memory etc.

Is there a way to programmatically (via the LLVM APIs) alter this default behavior, so that I can make public functions once again visible and exported?  Also, is there documentation somewhere that describes these and other breaking changes in the way that LLVM v7  now generates webassembly object files?
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Generating exported wasm functions

Alberto Barbaro via llvm-dev
Derek:

Thank you for your detailed and helpful response. Since writing that e-mail, I have done more research about the work being done. I am very pleased by the excellent progress on the linker, the new wasm file conventions, and the emergent work on dynamic linking. I think these are all very positive developments.

A couple of days ago, I did open an issue on the  https://github.com/WebAssembly/tool-conventions/ repository to explore these issues in more depth:  https://github.com/WebAssembly/tool-conventions/issues/81. Sam responded and I have responded back to him. Although I am pleased with what's been accomplished, there are a few areas where I hope we can do better, both in terms of the architecture's features and the way we communicate them.

These are important issues, and I am willing to take the time to exchange perspectives and feedback in detail. I am also willing to write helpful documentation and/or contribute to LLVM code changes if that too would be helpful.

Cheers,
Jonathan

On Thu, Nov 29, 2018 at 5:43 AM Derek Schuff <[hidden email]> wrote:
Hi Jonathan,
A lot has changed with the wasm backend since LLVM v5! One of the reasons we remained an experimental target was that the interfaces (e.g. the text and binary formats, and the LLVM APIs) were not stable or well-documented. We have been investing a lot of work in these formats since then; for example LLVM's wasm object file format and lld support for static linking are now feature-complete and mostly documented (https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md). Our plan is to additionally document the C ABI and declare it (and the object file format) as "version 1" by the LLVM 8.0 release in hopes of keeping that version stable. LLVM's wasm text assembly format is currently under active development and I wouldn't consider the text format in either LLVM v7 or current trunk to be stable. We also hope to have some stability for it by LLVM 8.0.

With respect to the details of symbol visibility and exports, there are some details about that that we are currently hashing out, as we are currently designing and implementing dynamic linking to go with our static linking support. I think there wasm will always have some differences compared to the model used by e.g. ELF/COFF/MachO. One reason is that wasm includes the notion of having (a specified set of) symbols exported from a finally-linked binary; in wasm this is used to export a set of functions to the external environment (e.g. JavaScript). ELF has --export-dynamic but I think that's probably not enough control for the web use case.

We definitely do want to enable control over the visibility and export of public functions (at the source, IR, and linker level). For the IR I'd expect this would be reflected as attributes on functions or GlobalValues, which should be controllable via the LLVM APIs.

In any case we'd be interested in hearing about your feedback or use cases; most of the discussion about the conventions used by LLVM and related tools are on the GitHub repo we use for our docs: https://github.com/WebAssembly/tool-conventions/. Feel free to file an issue or chime in on an existing one.

Best,
-Derek

On Sat, Nov 17, 2018 at 2:04 AM Jonathan Goodwin via llvm-dev <[hidden email]> wrote:
Using LLVM v5, my Cone compiler (https://github.com/jondgoodwin/cone) automatically directly generated all functions as named and exported for both wasm text and binary files (not using clang/lld etc, but using the API).

Upgrading to LLVM v7, generated wasm object files now fail, exhibiting markedly different behavior, behavior that varies between the text and binary files. For example, the binary wasm file (only) now automatically adds two import directives that were not there before. More importantly, function names, visible in the wat file, are stripped from the binary wasm file and are not exported. It seems as if LLVM webassembly codegen has acted as if the linker were at play and had defaulted to options like: -fvisibility=hidden, -W1, --import-memory etc.

Is there a way to programmatically (via the LLVM APIs) alter this default behavior, so that I can make public functions once again visible and exported?  Also, is there documentation somewhere that describes these and other breaking changes in the way that LLVM v7  now generates webassembly object files?
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev