Alternative object file backend

[philipc]

I have an initial implementation of COFF object file support currently living in an object-write repository. This is an alternative implementation to faerie, and it presents a lower level API than faerie. This API requires the user to define sections, symbols, and relocations, rather than functions and data objects. Currently object-write also supports writing ELF, but not yet Mach-O.

I also have an initial cranelift module Backend implementation that uses object-write. Compared to the faerie module backend, I think this implementation is still fairly straight forward, and the lower level API is more flexible.

I also have investigated changing rustc_codegen_cranelift to use this, and it can successfully compile and link a simple program using the MSVC linker. It appears that at least the wrong calling convention is being used though, so it currently segfaults. This can also use the ELF support in object-write, and that passes the current rustc_codegen_cranelift tests.

This is all still in development, but I'm interested in early feedback about the direction this is going. I wrote object-write for other purposes, but I think it's a good fit for cranelift too, and may be useful for #511.

[bjorn3]

It appears that at least the wrong calling convention is being used though, so it currently segfaults.

That's correct. The abi is always set to SystemV: https://github.com/bjorn3/rustc_codegen_cranelift/blob/e998997f98e936bfa7445272df1aaa43448c736a/src/abi.rs#L81.

[philipc]

Changing all the SystemV to WindowsFastcall gets the mini_core_hello_world test running.

[sunfishcode]

Cool! I found this issue on how faerie and object-write relate: philipc/object-write#1

The part about the abstraction being leaky is interesting. Faerie has a more abstract interface to relocations, however Cranelift ends up bypassing it, using link_with anyway, because it already knows the relocations it wants.

I expect that as Cranelift grows in its role as a low-level compiler, it will increasingly need to be aware of how object files are set up, including things like what sections things are in and what kinds of symbols and relocations it's using. So a low-level API seems like it could fit well for Cranelift's needs.

cranelift-faerie has active users, so it's not going to go away any time soon in any case, but I do think it makes sense to experiment with different abstraction levels.

Would it make sense to make Object::write be pub fn write<T: Write>(&self, format: Format, output: T), rather than just returning a Vec?

Also, here's a random comment on the ELF backend:

/// Set the section names expected by the linker.
fn finalize_elf_section_names(&mut self) {
for section in &mut self.sections {
if section.name.is_empty() || section.name[0] != b'.' {
let base = match section.kind {
SectionKind::Text => &b".text"[..],
SectionKind::Data => &b".data"[..],
SectionKind::ReadOnlyData | SectionKind::ReadOnlyString => &b".rodata"[..],
_ => continue,
};
let mut name = base.to_vec();
if !section.name.is_empty() {
name.push(b'.');
name.extend(&section.name);
}

This behavior of prepending ".text" to section names that don't start with "." is not familiar to me. The GCC __section__ attribute allows users to create identifiers that don't get ".text" or other things prepended to them, for example.

[philipc]

Faerie has a more abstract interface to relocations, however Cranelift ends up bypassing it, using link_with anyway, because it already knows the relocations it wants.

Yep, but doing it that way also makes it hard for faerie to get things like m4b/faerie#80 right, since faerie is missing the information it needs to make the decision.

Would it make sense to make Object::write be pub fn write<T: Write>(&self, format: Format, output: T), rather than just returning a Vec?

Maybe! Vecs are more convenient to write to, but it would be doable. This is basically to just reduce memory usage? Along those lines, I wonder if all the section data should be Cow.

This behavior of prepending ".text" to section names that don't start with "." is not familiar to me.

Yeah, this is a bit of a hack to get things working. The MSVC linker seems to have different naming conventions, so I wanted a way to choose the correct name automatically. Probably should do this as a helper method that the caller explicitly uses instead.

[sunfishcode]

Maybe! Vecs are more convenient to write to, but it would be doable. This is basically to just reduce memory usage?

My idea was just that Vec implements Write, so it should be straightforward. And if write just takes a thing which impelements Write, someone could use it with things other than Vec, for example it might be possible to write straight to a File.

Along those lines, I wonder if all the section data should be Cow.

It's not immediately obvious how Cow fits the use case here. When is section data shared?

[philipc]

My idea was just that Vec implements Write, so it should be straightforward.

I do have some minor usage of Vec methods for convenience, but yes it would be doable. There's also some debug asserts that use Vec::len which I would be sad to lose, so that would require Seek too.

for example it might be possible to write straight to a File.

Sure, I was just curious if that is a significant advantage, or only a nicety.

It's not immediately obvious how Cow fits the use case here. When is section data shared?

Probably not for cranelift's use case, but it may be useful for reading and writing existing files, which is where I am using it.