[llvm-dev] VC C++ demangler

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
Hi,

We have a demangler for the Itanium ABI, but looks like we don't have one for the MSVC-style symbols. Is there any good demangler we can import to LLVM?

If there's no suitable demangler, I'd like to write one. Currently, we are using `UnDecorateSymbolName` function, but the function is available only on Windows (which is problematic when you are doing a cross-build), and the function is not thread-safe. These two seem to be an enough reason to have our own demanler.

Rui

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
On Mon, Jun 19, 2017 at 10:53 AM, Rui Ueyama via llvm-dev
<[hidden email]> wrote:

> Hi,
>
> We have a demangler for the Itanium ABI, but looks like we don't have one
> for the MSVC-style symbols. Is there any good demangler we can import to
> LLVM?
>
> If there's no suitable demangler, I'd like to write one. Currently, we are
> using `UnDecorateSymbolName` function, but the function is available only on
> Windows (which is problematic when you are doing a cross-build), and the
> function is not thread-safe. These two seem to be an enough reason to have
> our own demanler.
>

I'm not aware of a suitable one, currently. I agree it would be very
useful to have.

--
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev

A long time ago, when I devised the grammar and structure of the Microsoft C++ name mangling scheme (decorated names), the document describing the object model and the name decoration scheme were made publically available.  Perhaps this is still available publically, or perhaps Microsoft might be willing to share an up to date definition of the name-decoration grammar, especially in light of the integration of CodeView debugging information into LLVM, which somewhat ties in with this.

 

This was expressed as a regular BNF grammar, so it should be possible to create a clean-room implementation of both the “mangler” and “de-mangler” from that BNF definition if it still exists in that form.  Does the recently added CodeView debug information not provide this description (I admit I haven’t looked)?

 

Certainly tools like ‘c++filt’ do not know about the Microsoft name decoration scheme, but LLVM does know how to mangle the names using the VC++ ABI, and since the mangling follows a regular grammar, the de-mangling should be relatively straight-forward to implement.

 

All the best,

 

            MartinO

 

-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Davide Italiano via llvm-dev
Sent: 19 June 2017 19:00
To: Rui Ueyama <[hidden email]>
Cc: llvm-dev <[hidden email]>
Subject: Re: [llvm-dev] VC C++ demangler

 

On Mon, Jun 19, 2017 at 10:53 AM, Rui Ueyama via llvm-dev <[hidden email]> wrote:

> Hi,

> 

> We have a demangler for the Itanium ABI, but looks like we don't have

> one for the MSVC-style symbols. Is there any good demangler we can

> import to LLVM?

> 

> If there's no suitable demangler, I'd like to write one. Currently, we

> are using `UnDecorateSymbolName` function, but the function is

> available only on Windows (which is problematic when you are doing a

> cross-build), and the function is not thread-safe. These two seem to

> be an enough reason to have our own demanler.

> 

 

I'm not aware of a suitable one, currently. I agree it would be very useful to have.

 

--

Davide

 

"There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare _______________________________________________

LLVM Developers mailing list

[hidden email]

http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
We have clang/lib/AST/MicrosoftMangle.cpp, so looks like what I should do is to write code that do the reverse of it. One thing I should be careful is to produce the exact same outputs as Microsoft's UnDecorateSymbolName function would output so that the behavior doesn't change between Windows and non-Windows platforms, but it probably shouldn't be hard.

On Mon, Jun 19, 2017 at 12:34 PM, Martin J. O'Riordan <[hidden email]> wrote:

A long time ago, when I devised the grammar and structure of the Microsoft C++ name mangling scheme (decorated names), the document describing the object model and the name decoration scheme were made publically available.  Perhaps this is still available publically, or perhaps Microsoft might be willing to share an up to date definition of the name-decoration grammar, especially in light of the integration of CodeView debugging information into LLVM, which somewhat ties in with this.

 

This was expressed as a regular BNF grammar, so it should be possible to create a clean-room implementation of both the “mangler” and “de-mangler” from that BNF definition if it still exists in that form.  Does the recently added CodeView debug information not provide this description (I admit I haven’t looked)?

 

Certainly tools like ‘c++filt’ do not know about the Microsoft name decoration scheme, but LLVM does know how to mangle the names using the VC++ ABI, and since the mangling follows a regular grammar, the de-mangling should be relatively straight-forward to implement.

 

All the best,

 

            MartinO

 

-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Davide Italiano via llvm-dev
Sent: 19 June 2017 19:00
To: Rui Ueyama <[hidden email]>
Cc: llvm-dev <[hidden email]>
Subject: Re: [llvm-dev] VC C++ demangler

 

On Mon, Jun 19, 2017 at 10:53 AM, Rui Ueyama via llvm-dev <[hidden email]> wrote:

> Hi,

> 

> We have a demangler for the Itanium ABI, but looks like we don't have

> one for the MSVC-style symbols. Is there any good demangler we can

> import to LLVM?

> 

> If there's no suitable demangler, I'd like to write one. Currently, we

> are using `UnDecorateSymbolName` function, but the function is

> available only on Windows (which is problematic when you are doing a

> cross-build), and the function is not thread-safe. These two seem to

> be an enough reason to have our own demanler.

> 

 

I'm not aware of a suitable one, currently. I agree it would be very useful to have.

 

--

Davide

 

"There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare _______________________________________________

LLVM Developers mailing list

[hidden email]

http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
On Mon, Jun 19, 2017 at 12:48 PM, Rui Ueyama <[hidden email]> wrote:
> We have clang/lib/AST/MicrosoftMangle.cpp, so looks like what I should do is
> to write code that do the reverse of it. One thing I should be careful is to
> produce the exact same outputs as Microsoft's UnDecorateSymbolName function
> would output so that the behavior doesn't change between Windows and
> non-Windows platforms, but it probably shouldn't be hard.
>

Something you may want to keep in mind while prototyping is that
writing a demangler in C++ (or any other unsafe language) is
particularly annoying, so you may want to keep fuzzers and sanitizers
always in your toolbox. The itanium demangler currently in tree
suffers from all kinds of bugs/vulnerabilities, FWIW.

--
Davide
_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev


2017-06-19 12:55 GMT-07:00 Davide Italiano via llvm-dev <[hidden email]>:
On Mon, Jun 19, 2017 at 12:48 PM, Rui Ueyama <[hidden email]> wrote:
> We have clang/lib/AST/MicrosoftMangle.cpp, so looks like what I should do is
> to write code that do the reverse of it. One thing I should be careful is to
> produce the exact same outputs as Microsoft's UnDecorateSymbolName function
> would output so that the behavior doesn't change between Windows and
> non-Windows platforms, but it probably shouldn't be hard.
>

Something you may want to keep in mind while prototyping is that
writing a demangler in C++ (or any other unsafe language) is
particularly annoying, so you may want to keep fuzzers and sanitizers
always in your toolbox. The itanium demangler currently in tree
suffers from all kinds of bugs/vulnerabilities, FWIW.

I thought these have been fixed after being fuzzed by Kostya? Are there still new ones that popped-up that haven't been fixed?

-- 
Mehdi


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
In reply to this post by ORiordan, Martin via llvm-dev


On Mon, Jun 19, 2017 at 12:49 PM Rui Ueyama via llvm-dev <[hidden email]> wrote:
We have clang/lib/AST/MicrosoftMangle.cpp, so looks like what I should do is to write code that do the reverse of it. One thing I should be careful is to produce the exact same outputs as Microsoft's UnDecorateSymbolName function would output so that the behavior doesn't change between Windows and non-Windows platforms, but it probably shouldn't be hard.

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.
 

On Mon, Jun 19, 2017 at 12:34 PM, Martin J. O'Riordan <[hidden email]> wrote:

A long time ago, when I devised the grammar and structure of the Microsoft C++ name mangling scheme (decorated names), the document describing the object model and the name decoration scheme were made publically available.  Perhaps this is still available publically, or perhaps Microsoft might be willing to share an up to date definition of the name-decoration grammar, especially in light of the integration of CodeView debugging information into LLVM, which somewhat ties in with this.

 

This was expressed as a regular BNF grammar, so it should be possible to create a clean-room implementation of both the “mangler” and “de-mangler” from that BNF definition if it still exists in that form.  Does the recently added CodeView debug information not provide this description (I admit I haven’t looked)?

 

Certainly tools like ‘c++filt’ do not know about the Microsoft name decoration scheme, but LLVM does know how to mangle the names using the VC++ ABI, and since the mangling follows a regular grammar, the de-mangling should be relatively straight-forward to implement.

 

All the best,

 

            MartinO

 

-----Original Message-----
From: llvm-dev [mailto:[hidden email]] On Behalf Of Davide Italiano via llvm-dev
Sent: 19 June 2017 19:00
To: Rui Ueyama <[hidden email]>
Cc: llvm-dev <[hidden email]>
Subject: Re: [llvm-dev] VC C++ demangler

 

On Mon, Jun 19, 2017 at 10:53 AM, Rui Ueyama via llvm-dev <[hidden email]> wrote:

> Hi,

> 

> We have a demangler for the Itanium ABI, but looks like we don't have

> one for the MSVC-style symbols. Is there any good demangler we can

> import to LLVM?

> 

> If there's no suitable demangler, I'd like to write one. Currently, we

> are using `UnDecorateSymbolName` function, but the function is

> available only on Windows (which is problematic when you are doing a

> cross-build), and the function is not thread-safe. These two seem to

> be an enough reason to have our own demanler.

> 

 

I'm not aware of a suitable one, currently. I agree it would be very useful to have.

 

--

Davide

 

"There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare _______________________________________________

LLVM Developers mailing list

[hidden email]

http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

--paulr

 


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.


That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.


That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
In reply to this post by ORiordan, Martin via llvm-dev
It has been a while since I have fought this stuff but, as I recall, there is some relationship between the display name of a function in the debug info and the result of demangling a symbol.
I think a good criteria is to have Clang's display name generation for CodeView and our implementation of the demangler agree. This way we have an explainable system and any discrepency we would want to correct would result us in fixing both Clang and our demangler.

Even better idea: use the demangler to generate the display names in the debug info. I am pretty sure this is what MSVC does.

On Tue, Jun 20, 2017 at 10:36 AM, Rui Ueyama via llvm-dev <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.


That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
In reply to this post by ORiordan, Martin via llvm-dev
On Tue, Jun 20, 2017 at 10:51 AM, Robinson, Paul <[hidden email]> wrote:

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.


I'll try to keep the difference only in whitespace.
 

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
FYI, I started writing a demangler. I think I can send an initial patch to review in a few days.

On Tue, Jun 20, 2017 at 11:07 AM, Rui Ueyama <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:51 AM, Robinson, Paul <[hidden email]> wrote:

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.


I'll try to keep the difference only in whitespace.
 

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.




_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
Please add me on reviews.  BTW, even differing in whitespace might cause problems, I know their tools have some builtin assumptions about whitespace in type names.  How deeply engrained this is is not clear though.

On Fri, Jun 23, 2017 at 10:10 AM Rui Ueyama via llvm-dev <[hidden email]> wrote:
FYI, I started writing a demangler. I think I can send an initial patch to review in a few days.

On Tue, Jun 20, 2017 at 11:07 AM, Rui Ueyama <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:51 AM, Robinson, Paul <[hidden email]> wrote:

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.


I'll try to keep the difference only in whitespace.
 

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
I uploaded a FYI patch (not intended for submission) as https://reviews.llvm.org/D34667. If you want to take a look and comment on its design, please do so. Thanks!

On Fri, Jun 23, 2017 at 5:25 PM, Zachary Turner <[hidden email]> wrote:
Please add me on reviews.  BTW, even differing in whitespace might cause problems, I know their tools have some builtin assumptions about whitespace in type names.  How deeply engrained this is is not clear though.

On Fri, Jun 23, 2017 at 10:10 AM Rui Ueyama via llvm-dev <[hidden email]> wrote:
FYI, I started writing a demangler. I think I can send an initial patch to review in a few days.

On Tue, Jun 20, 2017 at 11:07 AM, Rui Ueyama <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:51 AM, Robinson, Paul <[hidden email]> wrote:

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.


I'll try to keep the difference only in whitespace.
 

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
Sorry for didn't post earlier, has anyone checked http://mingw-w64.sourceforge.net/libmangle/ ?

Also the Wine project has some code which might be useful as reference / test case:





On Tue, Jun 27, 2017 at 3:53 PM, Rui Ueyama via llvm-dev <[hidden email]> wrote:
I uploaded a FYI patch (not intended for submission) as https://reviews.llvm.org/D34667. If you want to take a look and comment on its design, please do so. Thanks!

On Fri, Jun 23, 2017 at 5:25 PM, Zachary Turner <[hidden email]> wrote:
Please add me on reviews.  BTW, even differing in whitespace might cause problems, I know their tools have some builtin assumptions about whitespace in type names.  How deeply engrained this is is not clear though.

On Fri, Jun 23, 2017 at 10:10 AM Rui Ueyama via llvm-dev <[hidden email]> wrote:
FYI, I started writing a demangler. I think I can send an initial patch to review in a few days.

On Tue, Jun 20, 2017 at 11:07 AM, Rui Ueyama <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:51 AM, Robinson, Paul <[hidden email]> wrote:

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.


I'll try to keep the difference only in whitespace.
 

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




--
Regards,
Qian Hong

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] VC C++ demangler

ORiordan, Martin via llvm-dev
They are LGPL, so I think we cannot use them.

On Tue, Jun 27, 2017 at 1:17 AM, Qian Hong <[hidden email]> wrote:
Sorry for didn't post earlier, has anyone checked http://mingw-w64.sourceforge.net/libmangle/ ?

Also the Wine project has some code which might be useful as reference / test case:





On Tue, Jun 27, 2017 at 3:53 PM, Rui Ueyama via llvm-dev <[hidden email]> wrote:
I uploaded a FYI patch (not intended for submission) as https://reviews.llvm.org/D34667. If you want to take a look and comment on its design, please do so. Thanks!

On Fri, Jun 23, 2017 at 5:25 PM, Zachary Turner <[hidden email]> wrote:
Please add me on reviews.  BTW, even differing in whitespace might cause problems, I know their tools have some builtin assumptions about whitespace in type names.  How deeply engrained this is is not clear though.

On Fri, Jun 23, 2017 at 10:10 AM Rui Ueyama via llvm-dev <[hidden email]> wrote:
FYI, I started writing a demangler. I think I can send an initial patch to review in a few days.

On Tue, Jun 20, 2017 at 11:07 AM, Rui Ueyama <[hidden email]> wrote:
On Tue, Jun 20, 2017 at 10:51 AM, Robinson, Paul <[hidden email]> wrote:

If it's only whitespace differences, that's easy to accommodate.  If there are other cases that don't work, maybe don't use this tactic for those, if we have a good reason for being different.  As they say, don't throw the baby out with the bathwater.


I'll try to keep the difference only in whitespace.
 

--paulr

 

From: David Blaikie [mailto:[hidden email]]
Sent: Tuesday, June 20, 2017 10:39 AM
To: Rui Ueyama; Robinson, Paul
Cc: Martin J. O'Riordan; [hidden email]
Subject: Re: [llvm-dev] VC C++ demangler

 

Yeah, may well be the case - I don't /think/ LLVM quite matches the exact syntax of the GCC demangler either (I seem to recall constants as non-type template parameters were a bit different).

 

On Tue, Jun 20, 2017 at 10:36 AM Rui Ueyama <[hidden email]> wrote:

On Tue, Jun 20, 2017 at 10:17 AM, Robinson, Paul <[hidden email]> wrote:

Just to be clear - once LLVM has its own demangler, it should probably use it on all platforms, so there'd be no worry about different behavior between LLVM on Windows and LLVM elsewhere.

But that said, it's probably still important/worthwhile to make sure it's consistent with the platform demangler.

 

Personally I would be all for a unit test program that verified against the Windows API when run on Windows, and against canned output on non-Windows.

 

That was my preference too, but looks like getting the exact same results as the Windows API is not that easy nor worthwhile when it comes to arbitrary formatting rules. For example, IIRC, UnDecorateSymbolName generates not "int const* const* x" nor "int const * const * x" but "int const* const * x". This is simply odd, and I'd guess we don't want to mimic all these corner cases. So mixing our own demangler and the Windows demangler can cause unnecessary churn.



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




--
Regards,
Qian Hong


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev