[llvm-dev] Using LLD to create a .lib from a .def

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev
On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev <[hidden email]> wrote:
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib file for x86-64. Windows uses different name mangling scheme for x86 and x86-64, so if you mix the two, it could result in an "undefined symbol" error.
 
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module definition files. You might be able to use that. See llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev


On Wed, Jun 14, 2017 at 7:37 PM, Rui Ueyama <[hidden email]> wrote:
On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev <[hidden email]> wrote:
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib file for x86-64. Windows uses different name mangling scheme for x86 and x86-64, so if you mix the two, it could result in an "undefined symbol" error.

I tested this by setting the machine in the config directly to AMD64.
 
 
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module definition files. You might be able to use that. See llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

OK, thanks. I'll get a build going with latest trunk and see if I can get it to work.
 

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev




_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev
When I build llvm-project from source (revision 53a3cb539f7e92e70d4eef8d1b219cbef7c8ba1c from https://github.com/llvm-project/llvm-project-20170507)

The build and install seemed to work correctly. But then when I run `llvm-config --libfiles`, I get this:

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a


The library got built: ./llvm/build/lib/libLLVMTestingSupport.a
But `make install` didn't copy it to the prefix path.


On Thu, Jun 15, 2017 at 10:33 AM, Andrew Kelley <[hidden email]> wrote:


On Wed, Jun 14, 2017 at 7:37 PM, Rui Ueyama <[hidden email]> wrote:
On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev <[hidden email]> wrote:
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib file for x86-64. Windows uses different name mangling scheme for x86 and x86-64, so if you mix the two, it could result in an "undefined symbol" error.

I tested this by setting the machine in the config directly to AMD64.
 
 
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module definition files. You might be able to use that. See llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

OK, thanks. I'll get a build going with latest trunk and see if I can get it to work.
 

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev





_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev
I saw this change come in yesterday:

commit 572ad839e2f66eaa82ffc71b1061eb3d06a4d126
Author: Saleem Abdulrasool <[hidden email]>
Date:   Thu Jun 15 20:39:58 2017 +0000

    COFF: add support for lib mode usage

    When link is invoked with `/def:` and no input files, it behaves as if
    `lib.exe` was invoked.  Emulate this behaviour, generating the import
    library from the def file that was passed.  Because there is no input to
    actually generate the dll, we simply process the def file early and exit
    once we have created the import library.


Thank you for this, Saleem.

I still have this problem though, with latest master of llvm-project, which is bd8af0ee30bc1a308bf286089b172f2ae87f3de5

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a

I'm pretty sure it's not just me, anyone should be able to reproduce this issue with a clean build.

Regards,
Andrew

On Thu, Jun 15, 2017 at 12:29 PM, Andrew Kelley <[hidden email]> wrote:
When I build llvm-project from source (revision 53a3cb539f7e92e70d4eef8d1b219cbef7c8ba1c from https://github.com/llvm-project/llvm-project-20170507)

The build and install seemed to work correctly. But then when I run `llvm-config --libfiles`, I get this:

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a


The library got built: ./llvm/build/lib/libLLVMTestingSupport.a
But `make install` didn't copy it to the prefix path.


On Thu, Jun 15, 2017 at 10:33 AM, Andrew Kelley <[hidden email]> wrote:


On Wed, Jun 14, 2017 at 7:37 PM, Rui Ueyama <[hidden email]> wrote:
On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev <[hidden email]> wrote:
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib file for x86-64. Windows uses different name mangling scheme for x86 and x86-64, so if you mix the two, it could result in an "undefined symbol" error.

I tested this by setting the machine in the config directly to AMD64.
 
 
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module definition files. You might be able to use that. See llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

OK, thanks. I'll get a build going with latest trunk and see if I can get it to work.
 

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev






_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev
It looks like it is an unrelated build issue (probably related to LLVM_BUILD_LLVM_DYLIB), no?

On Fri, Jun 16, 2017 at 11:53 AM, Andrew Kelley <[hidden email]> wrote:
I saw this change come in yesterday:

commit 572ad839e2f66eaa82ffc71b1061eb3d06a4d126
Author: Saleem Abdulrasool <[hidden email]>
Date:   Thu Jun 15 20:39:58 2017 +0000

    COFF: add support for lib mode usage

    When link is invoked with `/def:` and no input files, it behaves as if
    `lib.exe` was invoked.  Emulate this behaviour, generating the import
    library from the def file that was passed.  Because there is no input to
    actually generate the dll, we simply process the def file early and exit
    once we have created the import library.


Thank you for this, Saleem.

I still have this problem though, with latest master of llvm-project, which is bd8af0ee30bc1a308bf286089b172f2ae87f3de5

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a

I'm pretty sure it's not just me, anyone should be able to reproduce this issue with a clean build.

Regards,
Andrew

On Thu, Jun 15, 2017 at 12:29 PM, Andrew Kelley <[hidden email]> wrote:
When I build llvm-project from source (revision 53a3cb539f7e92e70d4eef8d1b219cbef7c8ba1c from https://github.com/llvm-project/llvm-project-20170507)

The build and install seemed to work correctly. But then when I run `llvm-config --libfiles`, I get this:

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a


The library got built: ./llvm/build/lib/libLLVMTestingSupport.a
But `make install` didn't copy it to the prefix path.


On Thu, Jun 15, 2017 at 10:33 AM, Andrew Kelley <[hidden email]> wrote:


On Wed, Jun 14, 2017 at 7:37 PM, Rui Ueyama <[hidden email]> wrote:
On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev <[hidden email]> wrote:
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib file for x86-64. Windows uses different name mangling scheme for x86 and x86-64, so if you mix the two, it could result in an "undefined symbol" error.

I tested this by setting the machine in the config directly to AMD64.
 
 
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module definition files. You might be able to use that. See llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

OK, thanks. I'll get a build going with latest trunk and see if I can get it to work.
 

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev







_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Using LLD to create a .lib from a .def

ORiordan, Martin via llvm-dev
It's a pretty standard build, all I did is change the prefix path:

cmake .. -DCMAKE_INSTALL_PREFIX=/home/andy/local -DCMAKE_PREFIX_PATH=/home/andy/local -DCMAKE_BUILD_TYPE=Debug

On Fri, Jun 16, 2017 at 4:29 PM, Rui Ueyama <[hidden email]> wrote:
It looks like it is an unrelated build issue (probably related to LLVM_BUILD_LLVM_DYLIB), no?

On Fri, Jun 16, 2017 at 11:53 AM, Andrew Kelley <[hidden email]> wrote:
I saw this change come in yesterday:

commit 572ad839e2f66eaa82ffc71b1061eb3d06a4d126
Author: Saleem Abdulrasool <[hidden email]>
Date:   Thu Jun 15 20:39:58 2017 +0000

    COFF: add support for lib mode usage

    When link is invoked with `/def:` and no input files, it behaves as if
    `lib.exe` was invoked.  Emulate this behaviour, generating the import
    library from the def file that was passed.  Because there is no input to
    actually generate the dll, we simply process the def file early and exit
    once we have created the import library.


Thank you for this, Saleem.

I still have this problem though, with latest master of llvm-project, which is bd8af0ee30bc1a308bf286089b172f2ae87f3de5

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a

I'm pretty sure it's not just me, anyone should be able to reproduce this issue with a clean build.

Regards,
Andrew

On Thu, Jun 15, 2017 at 12:29 PM, Andrew Kelley <[hidden email]> wrote:
When I build llvm-project from source (revision 53a3cb539f7e92e70d4eef8d1b219cbef7c8ba1c from https://github.com/llvm-project/llvm-project-20170507)

The build and install seemed to work correctly. But then when I run `llvm-config --libfiles`, I get this:

llvm-config: error: component libraries and shared library

llvm-config: error: missing: /home/andy/local/lib/libLLVMTestingSupport.a


The library got built: ./llvm/build/lib/libLLVMTestingSupport.a
But `make install` didn't copy it to the prefix path.


On Thu, Jun 15, 2017 at 10:33 AM, Andrew Kelley <[hidden email]> wrote:


On Wed, Jun 14, 2017 at 7:37 PM, Rui Ueyama <[hidden email]> wrote:
On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev <[hidden email]> wrote:
I'm copying some LLD code into my codebase like this:

// workaround for LLD not exposing ability to convert .def to .lib

#include <set>

namespace lld {
namespace coff {

class SymbolBody;
class StringChunk;
struct Symbol;

struct Export {
  StringRef Name;       // N in /export:N or /export:E=N
  StringRef ExtName;    // E in /export:E=N
  SymbolBody *Sym = nullptr;
  uint16_t Ordinal = 0;
  bool Noname = false;
  bool Data = false;
  bool Private = false;

  // If an export is a form of /export:foo=dllname.bar, that means
  // that foo should be exported as an alias to bar in the DLL.
  // ForwardTo is set to "dllname.bar" part. Usually empty.
  StringRef ForwardTo;
  StringChunk *ForwardChunk = nullptr;

  // True if this /export option was in .drectves section.
  bool Directives = false;
  StringRef SymbolName;
  StringRef ExportName; // Name in DLL

  bool operator==(const Export &E) {
    return (Name == E.Name && ExtName == E.ExtName &&
            Ordinal == E.Ordinal && Noname == E.Noname &&
            Data == E.Data && Private == E.Private);
  }
};

enum class DebugType {
  None  = 0x0,
  CV    = 0x1,  /// CodeView
  PData = 0x2,  /// Procedure Data
  Fixup = 0x4,  /// Relocation Table
};

struct Configuration {
  enum ManifestKind { SideBySide, Embed, No };
  llvm::COFF::MachineTypes Machine = llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN;
  bool Verbose = false;
  llvm::COFF::WindowsSubsystem Subsystem = llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
  SymbolBody *Entry = nullptr;
  bool NoEntry = false;
  std::string OutputFile;
  bool DoGC = true;
  bool DoICF = true;
  bool Relocatable = true;
  bool Force = false;
  bool Debug = false;
  bool WriteSymtab = true;
  unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
  StringRef PDBPath;

  // Symbols in this set are considered as live by the garbage collector.
  std::set<SymbolBody *> GCRoot;

  std::set<StringRef> NoDefaultLibs;
  bool NoDefaultLibAll = false;

  // True if we are creating a DLL.
  bool DLL = false;
  StringRef Implib;
  std::vector<Export> Exports;
  std::set<std::string> DelayLoads;
  std::map<std::string, int> DLLOrder;
  SymbolBody *DelayLoadHelper = nullptr;

  // Used for SafeSEH.
  Symbol *SEHTable = nullptr;
  Symbol *SEHCount = nullptr;

  // Used for /opt:lldlto=N
  unsigned LTOOptLevel = 2;

  // Used for /opt:lldltojobs=N
  unsigned LTOJobs = 1;

  // Used for /merge:from=to (e.g. /merge:.rdata=.text)
  std::map<StringRef, StringRef> Merge;

  // Used for /section=.name,{DEKPRSW} to set section attributes.
  std::map<StringRef, uint32_t> Section;

  // Options for manifest files.
  ManifestKind Manifest = SideBySide;
  int ManifestID = 1;
  StringRef ManifestDependency;
  bool ManifestUAC = true;
  std::vector<std::string> ManifestInput;
  StringRef ManifestLevel = "'asInvoker'";
  StringRef ManifestUIAccess = "'false'";
  StringRef ManifestFile;

  // Used for /failifmismatch.
  std::map<StringRef, StringRef> MustMatch;

  // Used for /alternatename.
  std::map<StringRef, StringRef> AlternateNames;

  uint64_t ImageBase = -1;
  uint64_t StackReserve = 1024 * 1024;
  uint64_t StackCommit = 4096;
  uint64_t HeapReserve = 1024 * 1024;
  uint64_t HeapCommit = 4096;
  uint32_t MajorImageVersion = 0;
  uint32_t MinorImageVersion = 0;
  uint32_t MajorOSVersion = 6;
  uint32_t MinorOSVersion = 0;
  bool DynamicBase = true;
  bool AllowBind = true;
  bool NxCompat = true;
  bool AllowIsolation = true;
  bool TerminalServerAware = true;
  bool LargeAddressAware = false;
  bool HighEntropyVA = false;

  // This is for debugging.
  bool DebugPdb = false;
  bool DumpPdb = false;
};

extern Configuration *Config;

void writeImportLibrary();
void parseModuleDefs(MemoryBufferRef MB);

} // namespace coff
} // namespace lld

//=========================================================================




This is so that I can write the following user code:



// writes the output to dll_path with .dll replaced with .lib
void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
    lld::coff::Config = new lld::coff::Configuration;
    auto mem_buf = MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
    MemoryBufferRef mbref(*mem_buf);
    lld::coff::parseModuleDefs(mbref);
    lld::coff::Config->OutputFile = buf_ptr(dll_path);
    lld::coff::writeImportLibrary();
}


Then I give it def_contents that looks like:
LIBRARY kernel32
EXPORTS
ExitProcess
GetConsoleMode
GetStdHandle
GetFileInformationByHandleEx
WriteFile
GetLastError


with dll_path set to ./zig-cache/all.dll. This generates ./zig-cache/all.lib.

The generated LLVM IR looks like:

; Function Attrs: noreturn nounwind
declare void @ExitProcess(i32) #6
; Function Attrs: nounwind
declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull) #3
; Function Attrs: nounwind
declare i8* @GetStdHandle(i32) #3
; Function Attrs: nounwind
declare i1 @GetFileInformationByHandleEx(i8* nonnull, i32, i8* nonnull, i32) #3
; Function Attrs: nounwind
declare i1 @WriteFile(i8* nonnull, i8* nonnull readonly, i32, i32*, %OVERLAPPED*) #3
; Function Attrs: nounwind
declare i32 @GetLastError() #3

...with code you would expect to call these functions.

Then I link with 
lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB -ENTRY:_start ./zig-cache/hello.obj ./zig-cache/builtin.obj ./zig-cache/compiler_rt.obj ./zig-cache/all.lib


and I get the following errors:
./zig-cache/hello.obj: undefined symbol: ExitProcess
./zig-cache/hello.obj: undefined symbol: GetConsoleMode
./zig-cache/hello.obj: undefined symbol: GetStdHandle
./zig-cache/hello.obj: undefined symbol: GetFileInformationByHandleEx
./zig-cache/hello.obj: undefined symbol: GetLastError
./zig-cache/hello.obj: undefined symbol: WriteFile
error: link failed



1. Is there something else I need to be doing to make this work correctly?

The first thing I would check is to make sure that you created your .lib file for x86-64. Windows uses different name mangling scheme for x86 and x86-64, so if you mix the two, it could result in an "undefined symbol" error.

I tested this by setting the machine in the config directly to AMD64.
 
 
(Note: I already tried renaming "all" to "kernel32". Ideally I could generate 1 .lib file for all the DLL calls needed.)
2. Can LLD expose this ability directly so I don't have to copy paste a bunch of LLD internal stuff?

Martell recently factored out the code to parse and generate module definition files. You might be able to use that. See llvm/Object/COFFModuleDefinition.h.

The location at where LLD uses COFFModuleDefinition.h is in parseModuleDefs and createImportLibrary in COFF/Driver.cpp.

OK, thanks. I'll get a build going with latest trunk and see if I can get it to work.
 

Regards,
Andrew Kelley


_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev








_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev