Win64 LLP64 ABI support with cross-compiler tooling#630
Win64 LLP64 ABI support with cross-compiler tooling#630xdqi wants to merge 3 commits intolkl:masterfrom
Conversation
Cross-compiler wrapper (tools/lkl/bin/): - x86_64-w64-mingw32-gcc: routes kernel code (-D__KERNEL__) to x86_64-pc-cygwin-gcc (LP64), user-space to real mingw-gcc (LLP64) with a specs file to override linker path - x86_64-w64-mingw32-cc: symlink to gcc wrapper - x86_64-w64-mingw32-ld: patched ld supporting PE --image-base - x86_64-w64-mingw32-objcopy: patched objcopy for PE format Custom runtime: - lib/jmp_buf.c: setjmp/longjmp for x86_64 MS ABI (10 callee-saved XMM registers + different GPR set vs SysV)
PE64 default image base is 0x140000000. Weak symbols resolve to 0, causing rel32 relocation overflow (displacement > 2GB). Setting --image-base=0x10000 keeps all addresses in the low 32-bit range.
On Win64 (MinGW-w64 LLP64), 'long' is 32-bit while the LKL kernel (compiled with Cygwin LP64 GCC) uses 64-bit 'long'. This causes ABI mismatches in syscall parameters, UAPI struct layouts, and pointer casts between user-space and kernel. Introduce __lkl_long_t / __lkl_ulong_t conditional typedefs (arch/lkl/include/uapi/asm/lkl_long.h): - LLP64 (MinGW-w64 x64): expands to 'long long' (64-bit) - LP64 (Linux/Cygwin/macOS): expands to 'long' (no change) headers_install.py replaces 'long' with __lkl_long_t in installed UAPI headers and auto-inserts #include <lkl/asm/lkl_long.h>. Source files updated to use __lkl_long_t for syscall parameter arrays, ioctl pointer casts, and return values. Also fix case-sensitive includes in virtio_net_wintap.c for Linux cross-compilation.
| #define _ASM_UAPI_LKL_LONG_H | ||
|
|
||
| /* | ||
| * __lkl_long_t / __lkl_ulong_t: pointer-width integer types for the |
There was a problem hiding this comment.
Why not lkl_long_t instead of __lkl_long_t?
There was a problem hiding this comment.
I did this following the convention of __lkl_kernel_long_t.
There was a problem hiding this comment.
Since this is exposed to user APIs, I think it is better to use lkl_long_t instead. __lkl__kernel are typically used for internal kernel structures.
|
|
||
| define nt_host | ||
| $(call set_autoconf_var,NT,y) | ||
| $(call set_kernel_config,INIT_STACK_ALL_ZERO,n) |
There was a problem hiding this comment.
It's used to make my old Cygwin GCC 11 happy. We can remove it if we maintain our own newer Cygwin cross compiler.
There was a problem hiding this comment.
Thanks for the weak externals fix in xdqi/lkl-binutils@64f5155!
I think it is time to move away from comitting binaries. We could use a script that downloads, patches and builds the tools similar with tools/lkl/scripts/dpdk-sdk-build.sh.
There was a problem hiding this comment.
When I worked on this, I wondered why we are stuck on Binutils 2.25.
MSYS2 Binutils had already removed the patches years ago.
The {i686,x86_64}-w64-ming32-{ld,objcopy} on Debian 13 doesn't work so I have to use this old version.
|
Just to make the LKL run under vanilla 64-bit Win32 environment instaed of Cygwin. |
2 participants
lkl: Win64 LLP64 ABI support with cross-compiler tooling
Summary
This PR enables LKL to be compiled for Win64 (PE) targets using a hybrid LP64/LLP64 toolchain. The core challenge is that Win64 follows the LLP64 data model (
long=32-bit) while the LKL kernel internally uses LP64 (long=64-bit). These patches bridge that gap with compile-time typedef remapping, cross-compiler shims, and a custom setjmp/longjmp for the MS ABI.Changes
1.
__lkl_long_ttypedef for cross-ABI compatibilityarch/lkl/include/uapi/asm/lkl_long.hdefines__lkl_long_t/__lkl_ulong_t:long long(64-bit)long(unchanged)headers_install.pyauto-replaceslongwith__lkl_long_tin installed UAPI headers and inserts the required include2. Cross-compiler wrapper scripts (
tools/lkl/bin/)x86_64-w64-mingw32-gcc: routes kernel code (-D__KERNEL__) tox86_64-pc-cygwin-gcc(LP64), user-space to real MinGW-w64 gcc (LLP64), with generated specs to a patched linkerx86_64-w64-mingw32-cc: symlink to the gcc wrapperx86_64-w64-mingw32-ld: supports PE--image-base=0x10000x86_64-w64-mingw32-objcopy: PE format support3. Custom setjmp/longjmp for x86_64 MS ABI
setjmp.hwith raw register save/restore inline assemblyRtlUnwindExwhich cannot handle cross-stack jumps used by LKL's cooperative context switching4. Patched binutils: fix weak externals in PE COFF after
ld -rWhen
ld -rcreates a relocatable object (e.g.vmlinux), thetagndxfield in PE COFF weak external aux entries is not relocated — it retains stale symbol indices from the original object. All 352C_NT_WEAKsymbols in the LKLvmlinuxhave incorrecttagndxvalues: some point to non-code sections (e.g..data), others to completely unrelated code symbols (e.g.bpf_arena_get_user_vm_start→cmd_line_append).The existing binutils code only fell back to name-based lookup when
tagndxpointed to a non-code section, missing the case where it points to a valid-but-wrong code symbol. This causedobjcopy -Gto resolve weak externals to incorrect addresses, leading to runtime crashes (e.g.ktime_getdereferencing NULL throughtk_core->clock).The fix in
coff_nt_weak_to_local()(lkl-binutils@64f5155) always searches for the.weak.<name>.<tag>implementation by name first — PE COFF weak external naming is deterministic and reliable. Only falls back totagndxwhen no name match is found.Repositoriy: xdqi/lkl-binutils (lkl branch, based on binutils-2.25.1)
Toolchain
The build environment is Debian 13 (trixie):
gcc-mingw-w64-x86-64,binutils-mingw-w64-x86-64)x86_64-pc-cygwin-gccproduces LP64 PE/COFF output directly--image-basesupport (lkl-binutils)The kernel is compiled with Cygwin GCC (LP64, direct PE output), while user-space libs and tests use MinGW-w64 GCC (LLP64). The
__lkl_long_ttypedef bridges the ABI between these two worlds.Disclaimer
This pull request is mostly done by Claude Code, but I audited every line of code :).