Can a C compiler generate a 64-bit executable where pointers are 32-bits?
Most programs fit well in address space of less than 4GB, but in some cases the program may need to use the new processor features/instructions that are only available in the x64 architecture. Unfortunately the normal procedure would be to generate a 64-bit executable and earning the cost of pointers being twice the size.
There are specific applications where most data structures have much of their elements composed of pointer. Large pointers will produce cache miss(en) more often and if you are not going to use memory beyond the 4GB, 64-bit pointers are pure waste.
Are there compilers where I can use x64-specific registers and instructions, but preserving the 32-bit pointers to save memory? On which platforms?
Is it possible to do this transparently in legacy code? There is a switch/flag to turn this on compiler?
If it is not possible to work with legacy code, what do you need to change in the code to get the 64-bit features, while maintaining 32-bit pointers?
I know this is only advantageous if this memory saving is really important.
2 answers
Yes, that is possible.
However understand that this is not as simple a change as it seems. 32-bit libraries expect you to pass arguments through the records in a certain format, and that Pointers are 4 bytes. 64-bits use other incompatible format and 8-byte pointers. If you're going to use 4-byte pointers and all the rest of the features the same as it would be in 64-bit, neither version of the library will be compatible with your program. That means basically that you will need to compile all the libraries that you are going to use in this format.
Another problem is in the kernel. When making a system call, arguments are also passed in a differentiated way. Then you will also need special kernel support to run such a program.
This is implemented as a new architecture, the x32:
To compile a program in x32 mode, you will need a linux environment reasonably modern:
- Linux 64-bit version 3.4 compiled with the CONFIG_X86_X32 option.
- GLIBC 2.16
- GCC 4.7
- Binutils 2.22
My default ubuntu installation comes with these requirements. Compiling a program in x32 mode is simple: add the flag -mx32 to the gcc. I was able to run the compiled programs without any additional procedures and observing the generated assembly it becomes clear that 64-bit loggers are used, even with 32-bit pointers.
As it is a relatively new feature (integrated into the kernel at the end of 2011) and little used (i.e. little tested in production), it is possible that minor problems are encountered.
If you don't want to change architecture and prefer something more "stable", another option would be to swap all pointers for uint32_t and do a conversion whenever necessary. There is no loss of performance since this conversion does not generate instructions. However there is the problem of the pointer pointing to something beyond the 4GB. This will happen, for example, for all stack pointers, since the stack starts at the end of the virtual memory of the process.
I don't know if it helps, but in Visual C++ there is the link flag /LARGEADDRESSAWARE, which allows a 32bit executable to access up to 3 gigs of RAM, instead of 2, when it runs on a 64-bit version of Windows.
I use this at work as we have a program that often needs to allocate more than 2 gigs.
According to the documentation of the flag, it is possible to pass /LARGEADDRESSAWARE:NO to 64 programs, which limits these programs to 2 giga of RAM. It is possible that this will cause pointers stay also in 32 bits, but I did not check.