[z88dk-dev] z80asm .reloc for runtime binary relocation

Bridge to the z88dk-developers mailing list
Post Reply
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

[z88dk-dev] z80asm .reloc for runtime binary relocation

Post by alvin »

After some discussion on this, it turns out the reloc file is inadequate as-is for relocating most binaries.

Right now, it is only a list of binary offsets where a 16-bit adjustment would have to be made if the binary is moved from its original ORG. This is obviously inadequate for instances were the upper or lower 8 bits of an address is used in computations which occurs frequently in (eg) sdcc-generated code.

Code: Select all

 0109 DD 7E FC      [19]  343         ld      a,-4 (ix)
010C C6r00         [ 7]  344         add     a, #<(_dos_filenum)
010E DD 77 FC      [19]  345         ld      -4 (ix),a
0111 DD 7E FD      [19]  346         ld      a,-3 (ix)
0114 CEs00         [ 7]  347         adc     a, #>(_dos_filenum)
0116 DD 77 FD      [19]  348         ld      -3 (ix),a
Here the address "_dos_filenum" is divided into 8-bit parts.

What I think will work is to have the reloc file consist of three parts. The first part is a 16-bit count (number of offsets) followed by a list of 16-bit offsets where a 16-bit quantity needs adjustment by the code movement (this is what is done now for .reloc). The second part is a 16-bit count followed by a list of 16-bit offsets where an 8-bit quantity needs adjustment by the LSB of the code movement. The last part is a 16-bit count followed by a list of 16-bit offsets where an 8-bit quantity needs adjustment by the MSB of the code movement.

Any thoughts on this? Will this finally put to bed the runtime relocation issue and if so how easy would it be to put into z80asm?



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Bodo Wenzel

Post by Bodo Wenzel »

Any thoughts on this?
Well, it seems that the format of the object files is changing. How
about having a hard cut?

Just a proposal: When I was thinking about writing a relocating
assembler many years ago I found the standard IEEE 695 (MUFOM), I hope I
remember the number correctly. It defines a lot of records that might
solve any requirements you're facing today. It was used by Zilog for
their Z8 assembler, but I never found any other usage. I loved the idea
of two formats, one being readable by humans in text form, the other
being binary for dense code.

Cheers, Bodo

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

Well, it seems that the format of the object files is changing. How
about having a hard cut?

Just a proposal: When I was thinking about writing a relocating
assembler many years ago I found the standard IEEE 695 (MUFOM), I hope I
remember the number correctly. It defines a lot of records that might
solve any requirements you're facing today. It was used by Zilog for
their Z8 assembler, but I never found any other usage. I loved the idea
of two formats, one being readable by humans in text form, the other
being binary for dense code.
The linker has changed quite a bit, primarily to introduce sections. Paulo has decided to undertake a re-write since z80asm's present codebase is not conducive to introducing some features that we'd like to have; maybe he'll have a look.

But this post is about generating a relocation file intended for the z80 itself so that it can patch binaries at runtime. There are several projects in need of this: fuzix (unix-type os on z80), symbos (windows-like os on z80), contiki and several other smaller projects getting off the ground. These environments need to load the z80 binary into available memory space and then patch it to run at its assigned location. We don't want to replicate z80asm's linking code, only supply enough information that the z80 can quickly patch an existing binary on its own after loading.

There are a few attempts for the z80, all flawed as far as I can see.

* zasm : this assembler mentions runtime relocation but explicitly states it cannot generate information for relocating 8-bit quantities. That means code like this cannot be patched with info delivered by zasm:

Code: Select all

0109 DD 7E FC      [19]  343         ld      a,-4 (ix)
010C C6r00         [ 7]  344         add     a, #<(_dos_filenum)
010E DD 77 FC      [19]  345         ld      -4 (ix),a
0111 DD 7E FD      [19]  346         ld      a,-3 (ix)
0114 CEs00         [ 7]  347         adc     a, #>(_dos_filenum)
0116 DD 77 FD      [19]  348         ld      -3 (ix),a
This is very common in sdcc output so such a patcher will almost never work for sdcc-compiled programs.

* symbos : symbos's loader isn't documented very well but it sounds like it also only patches 16-bit addresses like zasm. So the above cannot be patched. The new idea that symbos's loader introduces is a method for relocating portions of the output binary to different addresses in memory.

* contiki : Kevin Thacker patched sdld to generate relocation information. It can do 16-bit address patching, 8-bit LSB patching and 8-bit MSB patching. So the example above can be patched but his method is bugged. An address needing patching is always 16-bit even when it is split into its MSB / LSB halves. For some relocation offsets, the address plus offset can cause a carry into the MSB. This means sometimes the MSB value needs the MSB of the offset added and sometimes it needs the MSB of the offset + 1 added. Whether the +1 is added can only be determined by examining the LSB of the relocation address.

So I think the best we can do for z80 runtime relocation, which should almost always work except for perverted cases, is to supply a .reloc file associated with the output binary containing three sections:

* offsets to 16-bit adjustment
2 byte count indicating number of offsets needing adjustment
2 bytes per offset into the output binary where a 16-bit quantity needs adjustment

* offsets to 8-bit LSB adjustment
2 byte count indicating number of offsets needing adjustment
2 bytes per offset into the output binary where an 8-bit quantity needs adjustment by the LSB amount

* offsets to 8-bit MSB adjustment
2 byte count indicating number of offsets needing adjustment
3 bytes per offset into the output binary where an 8-bit quantity needs adjustment by the MSB amount. The first two bytes are the offset, the third is the LSB of the relevant relocatable address

This is simple-minded on purpose so that someone can easily process this into a form needed for the target.

If a binary is being relocated by X bytes (this is the displacement from compile-time ORG), the naive procedure taken by the z80 would be:

* for each offset in the 16-bit adjustment group: read address of next adjustment (2 bytes), add X to 16-bit number stored there
* for each offset in the 8-bit LSB adjustment group: read address of next adjustment (2 bytes), add X%256 tot he 8-bit number stored there
* for each offset in the 8-bit MSB adjustment group: read address of next adjustment (2 bytes), add X/256 + carryof(LSB+X%256) where LSB is the LSB component of the relocate address associated with the MSB.

Code: Select all

ld      a,-4 (ix)
add     a, #<(_dos_filenum)    ;; LSB(_dos_filenum)  ;; entry for this goes in the 8-bit LSB group
;; translated to z80asm this would appear as "add a,+(_dos_filenum) & 0xff" but the linker should be prepared for other operations applied to this quantity
ld      -4 (ix),a
ld      a,-3 (ix)
adc     a, #>(_dos_filenum)   ;; MSB(_dos_filenum)  ;; entry for this goes in the 8-bit MSB group and the associated LSB is LSB(_dos_filenum)
;; translated to z80asm this would appear as "adc a,+(_dos_filenum)/256" but the linker should be prepared for other operations applied to this quantity
ld      -3 (ix),a
This is naive because you could come up with ways to preprocess the .reloc file, depending on the needs of the loader, that would make the file smaller. Also it's possible that expressions for the 16-bit quantities could be much more complicated -- the linker should be able to compute the overall 16-bit values to determine the MSB/LSB components for the .reloc file.

An example target is fuzix. They propose to only allow relocation in 256-byte increments. This solves a number of problems: first, code written to rely on 256-byte aligned data will still work after relocation and second, the patching only has to be done to the MSB of addresses. To generate a fuzix-compatible relocation file from the proposed .reloc above, you'd just make a list of all the addresses of the MSB portion of the 16-bit adjustments and add the list of all the addresses of the 8-bit MSB adjustments (no LSB partner required).

I think the above will work to relocate almost all binary files at runtime and is about as simple as can be made. The current .reloc file cannot be successfully used to relocate most programs.



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Post Reply