optimizer clobber EFLAGS

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

optimizer clobber EFLAGS

Michael Hordijk

Using Clang/LLVM 3.6.0 we are observing a case where the optimizations
are clobbering EFLAGS on x86_64.  This is inconvenient when the status
of bit 9 (IF), which controls interrupts, changes.

Here's a simple test program.  Assume that the external function foo()
modifies the IF bit in EFLAGS.

---

#include <stdlib.h>
#include <stdbool.h>

void foo(void);
int a;

int bar(void)
{
         foo();

         bool const zero = a -= 1;

         asm volatile ("" : : : "cc");
         foo();

         if (zero) {
                 return EXIT_FAILURE;
         }

         foo();

         return EXIT_SUCCESS;
}

---

And it's compiled using the following command line:

---

$ clang -O2   -c -o clang-eflag.o clang-eflag.c

---

Produces this output:

---

$ objdump -S clang-eflag.o

clang-eflag.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <bar>:
    0:   53                      push   %rbx
    1:   e8 00 00 00 00          callq  6 <bar+0x6>
    6:   ff 0d 00 00 00 00       decl   0x0(%rip)        # c <bar+0xc>
    c:   9c                      pushfq
    d:   5b                      pop    %rbx
    e:   e8 00 00 00 00          callq  13 <bar+0x13>
   13:   b8 01 00 00 00          mov    $0x1,%eax
   18:   53                      push   %rbx
   19:   9d                      popfq
   1a:   75 07                   jne    23 <bar+0x23>
   1c:   e8 00 00 00 00          callq  21 <bar+0x21>
   21:   31 c0                   xor    %eax,%eax
   23:   5b                      pop    %rbx
   24:   c3                      retq

---

The critical bits here are that at 0xc/0xd, we save the value of EFLAGS
into %rbx.  We then call foo() (which changes bit 9 of EFLAGS).  We then
0x18/0x19 reload EFLAGS from the stale value in %rbx thus clobbering the
value of bit 9 leaving interrupts in an unexpected state.

You may notice that I've tried the asm volatile ("" : : : "cc")
constraint.  The LLVM IR has the what appears to be an appropriate
inline ASM directive:

---

entry:
   tail call void @foo() #2
   %0 = load i32* @a, align 4, !tbaa !1
   %sub = add nsw i32 %0, -1
   store i32 %sub, i32* @a, align 4, !tbaa !1
   %tobool = icmp eq i32 %sub, 0
   tail call void asm sideeffect "",
"~{cc},~{dirflag},~{fpsr},~{flags}"() #2, !srcloc !5
   tail call void @foo() #2
   br i1 %tobool, label %if.end, label %return

if.end:                                           ; preds = %entry
   tail call void @foo() #2
   br label %return

return:                                           ; preds = %entry, %if.end
   %retval.0 = phi i32 [ 0, %if.end ], [ 1, %entry ]
   ret i32 %retval.0

---

The constraint doesn't appear to do anything which is not totally
surprising.  I'm thinking that the "cc" constraint tells the optimizer
that "EFLAGS has been clobbered". What we need is a way to tell the
optimizer that "EFLAGS has been clobbered and the new value in EFLAGS
needs to be preserved (so don't clobber it)."

- michael

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: optimizer clobber EFLAGS

Reid Kleckner-2
I remember this bug. :) IMO, LLVM should never emit pushf / popf. I'm not sure this patch to fix it ever got committed:
http://reviews.llvm.org/D6629

On Wed, Jul 29, 2015 at 3:11 PM, Michael Hordijk <[hidden email]> wrote:

Using Clang/LLVM 3.6.0 we are observing a case where the optimizations are clobbering EFLAGS on x86_64.  This is inconvenient when the status of bit 9 (IF), which controls interrupts, changes.

Here's a simple test program.  Assume that the external function foo() modifies the IF bit in EFLAGS.

---

#include <stdlib.h>
#include <stdbool.h>

void foo(void);
int a;

int bar(void)
{
        foo();

        bool const zero = a -= 1;

        asm volatile ("" : : : "cc");
        foo();

        if (zero) {
                return EXIT_FAILURE;
        }

        foo();

        return EXIT_SUCCESS;
}

---

And it's compiled using the following command line:

---

$ clang -O2   -c -o clang-eflag.o clang-eflag.c

---

Produces this output:

---

$ objdump -S clang-eflag.o

clang-eflag.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <bar>:
   0:   53                      push   %rbx
   1:   e8 00 00 00 00          callq  6 <bar+0x6>
   6:   ff 0d 00 00 00 00       decl   0x0(%rip)        # c <bar+0xc>
   c:   9c                      pushfq
   d:   5b                      pop    %rbx
   e:   e8 00 00 00 00          callq  13 <bar+0x13>
  13:   b8 01 00 00 00          mov    $0x1,%eax
  18:   53                      push   %rbx
  19:   9d                      popfq
  1a:   75 07                   jne    23 <bar+0x23>
  1c:   e8 00 00 00 00          callq  21 <bar+0x21>
  21:   31 c0                   xor    %eax,%eax
  23:   5b                      pop    %rbx
  24:   c3                      retq

---

The critical bits here are that at 0xc/0xd, we save the value of EFLAGS into %rbx.  We then call foo() (which changes bit 9 of EFLAGS).  We then 0x18/0x19 reload EFLAGS from the stale value in %rbx thus clobbering the value of bit 9 leaving interrupts in an unexpected state.

You may notice that I've tried the asm volatile ("" : : : "cc") constraint.  The LLVM IR has the what appears to be an appropriate inline ASM directive:

---

entry:
  tail call void @foo() #2
  %0 = load i32* @a, align 4, !tbaa !1
  %sub = add nsw i32 %0, -1
  store i32 %sub, i32* @a, align 4, !tbaa !1
  %tobool = icmp eq i32 %sub, 0
  tail call void asm sideeffect "", "~{cc},~{dirflag},~{fpsr},~{flags}"() #2, !srcloc !5
  tail call void @foo() #2
  br i1 %tobool, label %if.end, label %return

if.end:                                           ; preds = %entry
  tail call void @foo() #2
  br label %return

return:                                           ; preds = %entry, %if.end
  %retval.0 = phi i32 [ 0, %if.end ], [ 1, %entry ]
  ret i32 %retval.0

---

The constraint doesn't appear to do anything which is not totally surprising.  I'm thinking that the "cc" constraint tells the optimizer that "EFLAGS has been clobbered". What we need is a way to tell the optimizer that "EFLAGS has been clobbered and the new value in EFLAGS needs to be preserved (so don't clobber it)."

- michael

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: optimizer clobber EFLAGS

JF Bastien
Agreed, never emit pushf/popf. Sorry I never committed the patch, the cmov issue got hairy and I never got to debugging it :-)
I can get back to it if there's interest!

On Wed, Jul 29, 2015 at 4:12 PM, Reid Kleckner <[hidden email]> wrote:
I remember this bug. :) IMO, LLVM should never emit pushf / popf. I'm not sure this patch to fix it ever got committed:
http://reviews.llvm.org/D6629


On Wed, Jul 29, 2015 at 3:11 PM, Michael Hordijk <[hidden email]> wrote:

Using Clang/LLVM 3.6.0 we are observing a case where the optimizations are clobbering EFLAGS on x86_64.  This is inconvenient when the status of bit 9 (IF), which controls interrupts, changes.

Here's a simple test program.  Assume that the external function foo() modifies the IF bit in EFLAGS.

---

#include <stdlib.h>
#include <stdbool.h>

void foo(void);
int a;

int bar(void)
{
        foo();

        bool const zero = a -= 1;

        asm volatile ("" : : : "cc");
        foo();

        if (zero) {
                return EXIT_FAILURE;
        }

        foo();

        return EXIT_SUCCESS;
}

---

And it's compiled using the following command line:

---

$ clang -O2   -c -o clang-eflag.o clang-eflag.c

---

Produces this output:

---

$ objdump -S clang-eflag.o

clang-eflag.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <bar>:
   0:   53                      push   %rbx
   1:   e8 00 00 00 00          callq  6 <bar+0x6>
   6:   ff 0d 00 00 00 00       decl   0x0(%rip)        # c <bar+0xc>
   c:   9c                      pushfq
   d:   5b                      pop    %rbx
   e:   e8 00 00 00 00          callq  13 <bar+0x13>
  13:   b8 01 00 00 00          mov    $0x1,%eax
  18:   53                      push   %rbx
  19:   9d                      popfq
  1a:   75 07                   jne    23 <bar+0x23>
  1c:   e8 00 00 00 00          callq  21 <bar+0x21>
  21:   31 c0                   xor    %eax,%eax
  23:   5b                      pop    %rbx
  24:   c3                      retq

---

The critical bits here are that at 0xc/0xd, we save the value of EFLAGS into %rbx.  We then call foo() (which changes bit 9 of EFLAGS).  We then 0x18/0x19 reload EFLAGS from the stale value in %rbx thus clobbering the value of bit 9 leaving interrupts in an unexpected state.

You may notice that I've tried the asm volatile ("" : : : "cc") constraint.  The LLVM IR has the what appears to be an appropriate inline ASM directive:

---

entry:
  tail call void @foo() #2
  %0 = load i32* @a, align 4, !tbaa !1
  %sub = add nsw i32 %0, -1
  store i32 %sub, i32* @a, align 4, !tbaa !1
  %tobool = icmp eq i32 %sub, 0
  tail call void asm sideeffect "", "~{cc},~{dirflag},~{fpsr},~{flags}"() #2, !srcloc !5
  tail call void @foo() #2
  br i1 %tobool, label %if.end, label %return

if.end:                                           ; preds = %entry
  tail call void @foo() #2
  br label %return

return:                                           ; preds = %entry, %if.end
  %retval.0 = phi i32 [ 0, %if.end ], [ 1, %entry ]
  ret i32 %retval.0

---

The constraint doesn't appear to do anything which is not totally surprising.  I'm thinking that the "cc" constraint tells the optimizer that "EFLAGS has been clobbered". What we need is a way to tell the optimizer that "EFLAGS has been clobbered and the new value in EFLAGS needs to be preserved (so don't clobber it)."

- michael

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: optimizer clobber EFLAGS

Michael Hordijk-2
On 7/29/15 18:35, JF Bastien wrote:
> Agreed, never emit pushf/popf. Sorry I never committed the patch, the
> cmov issue got hairy and I never got to debugging it :-)
> I can get back to it if there's interest!

You've definitely got some interest here.  I've been looking at your
patch on http://reviews.llvm.org/D6629 and I think I'm up to speed on
where it's stuck.  I'll shoot you an email directly unless you want to
keep the effort here on the list.

- michael


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: optimizer clobber EFLAGS

JF Bastien
On Fri, Jul 31, 2015 at 2:29 PM, Michael Hordijk <[hidden email]> wrote:
On 7/29/15 18:35, JF Bastien wrote:
Agreed, never emit pushf/popf. Sorry I never committed the patch, the
cmov issue got hairy and I never got to debugging it :-)
I can get back to it if there's interest!

You've definitely got some interest here.  I've been looking at your patch on http://reviews.llvm.org/D6629 and I think I'm up to speed on where it's stuck.  I'll shoot you an email directly unless you want to keep the effort here on the list.

Off-list is fine, we'll circle back when we come to a conclusion. 

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev