[z88dk-dev] What needs to be done on the sdcc side for the use of sd

[Philipp Klaus Krause]

There was a plan to use sdcc instead of scc in z88dk. Some work towards
that goal has already been done. However, there probably are some things
left to do. So:

What needs to be done on the sdcc side?
What is the most important (and thus should be done first)?

Philipp

------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/

[stefano]

>What needs to be done on the sdcc side?
>What is the most important (and thus should be done first)?

First thing coming to my mind is to reverse the function parameters list order.. is it feasible ?
Also the _FASTCALL_ and _CALLEE_ style calling methods would be welcome (and it is something SDCC could take benefit of, in any case)..

An easy thing, if not yet fixed, is to complete the z80asm compatibility.. some parameter is fixed here:
http://www.z88dk.org/forum/viewtopic.php?id=4587



------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

[Philipp Klaus Krause]

Am 12.03.2012 14:11, schrieb Stefano Bodrato (stefano_bodrato@...):

>What needs to be done on the sdcc side?
>What is the most important (and thus should be done first)?

Isn't this the __smallc calling convention that was implemented last
spring or so? We don't want to make it the default, since it causes
problems with varargs though.

First thing coming to my mind is to reverse the function parameters list order.. is it feasible ?
Also the _FASTCALL_ and _CALLEE_ style calling methods would be welcome (and it is something SDCC could take benefit of, in any case)..

Is this for calling existing functions in your library you want to use
from sdcc?

An easy thing, if not yet fixed, is to complete the z80asm compatibility.. some parameter is fixed here:
http://www.z88dk.org/forum/viewtopic.php?id=4587

While sdcc was meant to offer such flexibility, it doesn't work well
with our current peephole optimizer. As a temporary solution: Why not
just use the assembler that comes with sdcc for the asm generated by
sdcc, and use another assembler for other asm code, then link everything
together using whatever linker you prefer?

Philipp

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

[alvin]

Isn't this the __smallc calling convention that was implemented last
spring or so? We don't want to make it the default, since it causes
problems with varargs though.

yes it is the small c thing. z88dk wants to do left to right pushing of parameters which complicates varargs and the compiler has to supply extra information (in the A register) on function call to communicate the number of arguments to vararg functions. But the right way to do it is the conventional C linkage that sdcc is doing -- right to left means no special steps need to be taken with varargs functions. But you say there is a smallc linkage directive supplied, that may help in transition too.

Also the _FASTCALL_ and _CALLEE_ style calling methods would be welcome (and it is something SDCC could take benefit of, in any case)..

Is this for calling existing functions in your library you want to use
from sdcc?

This is actually very important -- it is an efficient means of calling library functions (or user functions) written in assembler.

To first approximation, the best way to get the best performance out of a C compiler for a small cpu is to make sure the libraries are hand coded in asm so that most cycles are spend in handcrafted code rather than compiler generated code. The place that generated code for the z80 is now is still far from optimally hand coded stuff. So a chunk of the effort in z88dk's dev has been in hand coded libraries. Further, the idea is to make sure z88dk is not just a C development environment but also an asm level dev environment. This means the preference is to have these hand coded libraries efficiently accessible from both C and asm.

For this kind of thing to be accessible from sdcc there has to be a convention for calling asm routines and passing parameters to them efficiently.

Right now in z88dk there are three calling conventions:

1. the smallc one with the left to right pushing of params on stack and the caller expected to clean up the stack after the function returns. this is z88dk's equivalent of the right to left convention in sdcc. An example bit of generated code:

extern int test(int a, b);

...

ld hl,(_aloc) ; push params
push hl
ld hl,(_bloc)
push hl
call _test
pop bc ; clean up stack
pop bc





2. fastcall where a single parameter is passed by register in (DE)HL. this is meant for asm libraries or user supplied asm. An example bit of code generated by the caller might look like this:

extern int __FASTCALL__ test(int d);

...

ld hl,(paramlocation)
call _test




3. callee where the target function is expected to clean up the stack. this is also meant for asm libraries or user supplied asm. asm functions pop params off the stack as they read them, effectively cleaning up the stack. this can lead to savings of hundreds of bytes in compiled programs since the compiler doesn't have to generate stack cleanup code each time the function is called. Example:


extern int __CALLEE__ test(int a, int b);

...

ld hl,(aloc) ; push params
push hl
ld hl,(bloc)
push hl
call _test ; stack cleanup performed at target function


Three different linkage conventions can mess with calls through function pointers unless the compiler can determine what the linkage is. z88dk is not strongly enough typed to do this. eg, you can do this:

void *f;
f = strcpy;
(f)(d, s);

No way can the compiler know if strcpy is called CALLEE or C convention. So in z88dk, now, all calls through function pointers are done with C linkage and every CALLEE function has a second entry point associated with it using C linkage. This second entry point does not get linked into the binary unless the program actually needs it. Some cpp macro magic makes sure function pointer assignments like "f = strcpy" get the C linkage stub whereas stuff like "strcpy(d,s)" gets the CALLEE linkage.




With sdcc there is an additional problem -- it reserves an index register for stack frame and can have registers with important values that shouldn't be modified in functions called. This forces the function called to save all registers it needs to use on stack. This actually interferes with the speed of code esp for developers using asm and especially for library code that calls other library code. There once was a suggestion on the sdcc list to have a way to specify what registers an asm function would modify so the compiler would insert pushes and pops around the function call. In addition, the compiler may be able to use that information to place important values in untouched registers or to better allocate values in registers the function requires. This sounds like it may be a bit difficult.

The use of index registers to access the stack frame is also quite often sub-optimal, although I know it makes it easier on the compiler author. I would personally prefer the compiler to investigate alternatives to using index registers for stack frames. You see index registers used very rarely in that fashion in hand coded z80 so that's an indicator this is a non-starter.

In library code, index registers are often used as function pointers.



------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d

[Philipp Klaus Krause]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 13.03.2012 22:15, alvin (alvin_albrecht@...) wrote:

The use of index registers to access the stack frame is also quite
often sub-optimal, although I know it makes it easier on the compiler
author. I would personally prefer the compiler to investigate
alternatives to using index registers for stack frames. You see
index registers used very rarely in that fashion in hand coded z80 so
that's an indicator this is a non-starter.

In library code, index registers are often used as function
pointers.

Well, sdcc has --fomit-frame-pointer, which makes sdcc not use a frame
pointer (though it doesn't use ix for anything else either). When there
is no frame pointer, sdcc uses alternative mechanisms, such as

ld hl, #n
add hl, sp
ld r, (hl)

and (more recently) push / pop / ex (sp), hl

they typically are a bit more expensive than an access through ix, and
we sometimes loose hl that could beused otherwise, but we save the
initialization and restoring of ix. Doing some testing, I found that
sdcc does better with a frame pointer for functions that have many stack
accesses, but does better without it when there are few, so we now
(since a few weeks ago) have a heuristic deciding on the use of a frame
pointer depending on register pressure and number of parameter accesses.
This has helped with code size for small functions quite a bit.

Philipp

P.S.: Example:

extern void cv_set_read_vram_address(unsigned short int);
extern unsigned char cv_vinb(void);

unsigned char cvu_vinb(const unsigned short int dest)
{
cv_set_read_vram_address(dest);
return(cv_vinb());
}

results in 18 bytes of code for z88dk 1.8, 26 bytes for HITECH-C
7.80PL2, 28 bytes for ADC 5.02, 24 bytes for older sdcc. For current
sdcc trunk it is 12 bytes:

_cvu_vinb:
pop bc
pop hl
push hl
push bc
push hl
call _cv_set_read_vram_address
pop af
jp _cv_vinb

I'll probably make the small change to use ex (sp), hl instead of the
pop / push pair here, saving another byte, before the 3.2.0 release.
Yes, passing arguments in registers would save even more for this
example, but that's a more complicated issue in general.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9h9UgACgkQbtUV+xsoLpp41gCeLZKXqTXtduKCJQLeUhLP9dXf
ls4AnRsO3i33ZJhQIhXIZQTIL6/zgHRh
=a2nJ
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

[Philipp Klaus Krause]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 13.03.2012 22:15, alvin (alvin_albrecht@...) wrote:

Isn't this the __smallc calling convention that was implemented
last spring or so? We don't want to make it the default, since
it causes problems with varargs though.

yes it is the small c thing. z88dk wants to do left to right
pushing of parameters which complicates varargs and the compiler
has to supply extra information (in the A register) on function
call to communicate the number of arguments to vararg functions.
But the right way to do it is the conventional C linkage that sdcc
is doing -- right to left means no special steps need to be taken
with varargs functions. But you say there is a smallc linkage
directive supplied, that may help in transition too.

Also the _FASTCALL_ and _CALLEE_ style calling methods would
be welcome (and it is something SDCC could take benefit of, in
any case)..

Is this for calling existing functions in your library you want
to use from sdcc?

This is actually very important -- it is an efficient means of
calling library functions (or user functions) written in
assembler.

To first approximation, the best way to get the best performance
out of a C compiler for a small cpu is to make sure the libraries
are hand coded in asm so that most cycles are spend in handcrafted
code rather than compiler generated code. The place that generated
code for the z80 is now is still far from optimally hand coded
stuff. So a chunk of the effort in z88dk's dev has been in hand
coded libraries. Further, the idea is to make sure z88dk is not
just a C development environment but also an asm level dev
environment. This means the preference is to have these hand coded
libraries efficiently accessible from both C and asm.

For this kind of thing to be accessible from sdcc there has to be
a convention for calling asm routines and passing parameters to
them efficiently.

Right now in z88dk there are three calling conventions:

1. the smallc one with the left to right pushing of params on
stack and the caller expected to clean up the stack after the
function returns. this is z88dk's equivalent of the right to left
convention in sdcc. An example bit of generated code:

extern int test(int a, b);

...

ld hl,(_aloc) ; push params push hl ld hl,(_bloc) push hl call
_test pop bc ; clean up stack pop bc

sdcc supports this one thorugh __smallc (partially: sdcc can call such
functions as long as they do not have a variable number of parameters,
but sdcc cannot generate functions to be called this way).

2. fastcall where a single parameter is passed by register in
(DE)HL. this is meant for asm libraries or user supplied asm. An
example bit of code generated by the caller might look like this:

extern int __FASTCALL__ test(int d);

...

ld hl,(paramlocation) call _test




3. callee where the target function is expected to clean up the
stack. this is also meant for asm libraries or user supplied asm.
asm functions pop params off the stack as they read them,
effectively cleaning up the stack. this can lead to savings of
hundreds of bytes in compiled programs since the compiler doesn't
have to generate stack cleanup code each time the function is
called. Example:


extern int __CALLEE__ test(int a, int b);

...

ld hl,(aloc) ; push params push hl ld hl,(bloc) push hl call
_test ; stack cleanup performed at target function

How many existing functions you would want to use are there with
fastcall and callee calling conventions? Would it help to have the
kind of partial support sdcc has for __smallc for these two?

Three different linkage conventions can mess with calls through
function pointers unless the compiler can determine what the
linkage is. z88dk is not strongly enough typed to do this. eg,
you can do this:

void *f; f = strcpy; (f)(d, s);

No way can the compiler know if strcpy is called CALLEE or C
convention. So in z88dk, now, all calls through function pointers
are done with C linkage and every CALLEE function has a second
entry point associated with it using C linkage. This second entry
point does not get linked into the binary unless the program
actually needs it. Some cpp macro magic makes sure function
pointer assignments like "f = strcpy" get the C linkage stub
whereas stuff like "strcpy(d,s)" gets the CALLEE linkage.




With sdcc there is an additional problem -- it reserves an index
register for stack frame and can have registers with important
values that shouldn't be modified in functions called. This forces
the function called to save all registers it needs to use on stack.
This actually interferes with the speed of code esp for developers
using asm and especially for library code that calls other library
code. There once was a suggestion on the sdcc list to have a way to
specify what registers an asm function would modify so the compiler
would insert pushes and pops around the function call. In
addition, the compiler may be able to use that information to place
important values in untouched registers or to better allocate
values in registers the function requires. This sounds like it may
be a bit difficult.

All those calling convention stuff might be able to save a byte or a
few cycles here or there at a function call. That might be important
for small functions that do a lot of function calls. However, compared
to other compilers, sdcc already seems to do quite well for those (see
table at
https://sourceforge.net/apps/trac/sdcc/wiki/Philipp%27s%20TODO%20list)?.
where sdcc is worse than other compilers, and where there is more
potential for improvement are the big, complex functions. IMO,
improving the performance of the register allocator and improving the
common subexpression elimination will yield the biggest gains in code
size and speed for sdcc this year.

Philipp

One notable exception is the following function implementing a galois
lfsr:

unsigned char galois_lfsr(void)
{
static unsigned char reg = 1;
reg = ((reg & 1) ? (reg >> 1)^0xe1 : (reg >> 1));
return(reg);
}

sdcc code size for this is currently 27 bytes (compared to 54 bytes
for z88dk 1.8, 62 bytes for ADC 5.02, 20 bytes for HITECH-C 7.80PL2
and 13 bytes for hand-coded asm). Old sdcc 2.70 generated 46 bytes of
code.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9iLxAACgkQbtUV+xsoLpqm6wCg9e/OiwfcrT0HJhUJlGcPn88d
+qEAoOavflsnyOiiBeKogS00VLi5Wcx7
=uPKR
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

[stefano]

Philip,
excuse me if I'll sound a bit elusive with this answer but I'd like to explain a point of my earlier answer: having z80asm valid code was not intended to push you to that direction only but to try to get sdcc+z88dk sor of in working condiftion in a very minimal situation (i.e. the enigma.c example) with no need of code recompilations.
This was a comfortable/relaxed starting point for further analisys.. always keeping in mind that SDCC *is* a very good compiler. Perhaps it is not enough, but in any case the last time I had to fix those z80asm declaration items.
On the other side, you are right a thing which might help is to just make z80asm compatible with the SDCC-mode object code.



------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

[alvin]

Well, sdcc has --fomit-frame-pointer, which makes sdcc not use a frame
pointer (though it doesn't use ix for anything else either). When there
is no frame pointer, sdcc uses alternative mechanisms, such as

ld hl, #n
add hl, sp
ld r, (hl)

and (more recently) push / pop / ex (sp), hl

they typically are a bit more expensive than an access through ix, and
we sometimes loose hl that could beused otherwise, but we save the
initialization and restoring of ix. Doing some testing, I found that
sdcc does better with a frame pointer for functions that have many stack
accesses, but does better without it when there are few, so we now
(since a few weeks ago) have a heuristic deciding on the use of a frame
pointer depending on register pressure and number of parameter accesses.
This has helped with code size for small functions quite a bit.

That is good to hear.. it is a good direction to head toward.

I am not suggesting the compiler should be disallowed from using index registers for stack frames but that it should investigate all alternatives in terms of speed and/or code size. Functions with many parameters that are randomly accessed are more likely to need a data structure like an indexed frame but most functions are not like that which is why it is so rare in hand coded programs. When it's hand-coded it's first place params in registers, then in less accessible or slower registers (eg the EXX set or IXL/IXH), then temporary locations on the stack in PUSH/POP order. No z80 compiler has ever (as far as I know) been written to investigate alternative stack frame strategies or even alternative code sequences for common operations that may fit better with register allocation or stack frame usage. That's where I would like to see things go but I digress

RE: CALLEE LINKAGE

How many existing functions you would want to use are there with
fastcall and callee calling conventions? Would it help to have the
kind of partial support sdcc has for __smallc for these two?

Several hundred functions if not close to a thousand use callee or fastcall linkage.

Here is an example so you know how things are currently being done with asm library functions:

Code: Select all

; char *strrchr(const char *s, int c)
;
; return ptr to last occurrence of c in s, NUL considered part of s

XLIB strrchr_callee
XDEF asm_strrchr, strrchr


strrchr:

pop af
pop de
pop hl
push hl
push de
push af

jp asm_strrchr

strrchr_callee:

pop hl
pop de
ex (sp),hl

asm_strrchr:

; enter :  e = char c
;         hl = char *s
;
; exit  : hl = ptr to last occurrence of c if found and Z flag set
;         hl = 0 if not found and NZ flag set
;          a = char c
;         de = strlen(s)
;
; uses  : af, bc, de, hl

; original version scanned forward once but this turns out to be
; slower than finding end of string and then searching backward
; because of cpir and cpdr instructions, except for very short strings
; uses more registers as a result but worth it I guess

; first find end of string and strlen

xor a
ld c,a
ld b,a
cpir
dec hl

ld a,e                       ; a = char c
ex de,hl                     ; de = ptr to NUL at end of s

ld hl,$ffff
sbc hl,bc
ld c,l
ld b,h                       ; bc = strlen(s)

; bc = strlen(s)
; de = ptr to NUL at end of s
;  a = char c

; next look for match to char from end of s

ex de,hl
cpdr
inc hl
ret z

ld l,c
ld h,b
ret

You'll note three entry points: strrchr is the left to right C-linkage with caller expected to clear up stack, strrchr_callee is callee linkage with the function called expected to clear up the stack, and asm_strrchr is the asm entry point with register set up.

strrchr_callee is the entry point used from C except for function pointers which use the strrchr entry point.

All those calling convention stuff might be able to save a byte or a
few cycles here or there at a function call. That might be important
for small functions that do a lot of function calls. However, compared
to other compilers, sdcc already seems to do quite well for those (see
table at
https://sourceforge.net/apps/trac/sdcc/ ? ODO%20list)?.

Well we are talking about large programs There are dozens of games written using z88dk, mainly on the ZX Spectrum platform, and callee linkage by itself can save a few hundred bytes in the final compiled code in a 30k program. But the approach of z88dk has been different :- we know that, to first order, the best results in terms of speed and compactness is gained from asm libraries so effort has been placed into that. Accordingly, z88dk programs see best performance and benefit from using library code wherever possible. The comparisons you are using are testing the compiler's ability to translate C code (something we know sdcc does better and z88dk's C translator would need a virtual rewrite to take it to the next step and no one's been willing to do that as it represents a lot of work).

Whatever happens, C programmers would prefer not to see penalties associated with having asm library code; the preference would be for CALLER-SAVE calls (the compiler saves what it needs to rather than the function saving everything it alters), the register allocator smart enough to use information about what registers the target function alters, and maybe the C compiler able to use other methods of delivering results (eg, an asm implementation if isspace() returns with a flag set appropriately to indicate whether a char is a space). Some of this might be difficult.

Anyway without callee or fastcall linkage at minimum, the sdcc version of strrchr() would look like this:

Code: Select all

strrchr:

exx

pop af
pop hl
pop de
push de
push hl
push af

call strrchr

exx
ret

asm_strrchr:

(as before)

ret

Note I've used exx as I am not sure what registers sdcc uses. Had strrchr used IX, I would have had to push that as well. Larger library functions need to use the EXX set as well so the collection of parameters from the stack would get very ugly. Some platform hardware reserves certain registers as well; the library code may not be able to use the EXX set to save registers the compiler uses and that also makes things ugly.



------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

[alvin]

It seems I pasted the version that was not done I just want to point out the difficulties of using asm libraries (or user supplied asm code) in sdcc now.

The last example showed use of EXX to save registers. If that is not possible, then we can do this:

Code: Select all

strrchr:

push ix   ; save frame pointer if asm lib code uses ix
push bc  ; i recall sdcc may reserve bc from a long time ago?

ld hl,6
add hl,sp  ; sdcc is right to left param push, point at 'char *s'
ld e,(hl)
inc hl
ld d,(hl)
inc hl
ld l,(hl)
ex de,hl

; e = char c
; hl = char *s

call asm_strrchr

pop bc
pop ix
ret

Compare this to z88dk's callee linkage:

Code: Select all

strrchr_callee:

pop hl
pop de
ex (sp),hl

.... fall through to asm lib code

If we only have to worry about saving IX, the sdcc version could be like this:


**** FRAGMENT A:

Code: Select all

strrchr:

pop af
pop hl
pop de
push de
push hl
push af

push ix   ; if asm lib code uses it
call asm_strrchr
pop ix
ret

If the asm lib code doesn't use ix and doesn't have to save other registers, the sdcc linkage can be equivalent to the z88dk C linkage:

Code: Select all

strrchr:

pop af
pop hl
pop de
push de
push hl
push af

...
fall through to asm lib code

Consider that a large program may have 50-100 separate calls to library functions and multiply all the pushing and popping of params the compiler inserts around the library call as well as the stack gymnastics each lib function needs to perform. There are several hundred bytes wasted there.

Then consider the execution cycles wasted with each lib call involving all the stack gymnastics just to interface with the C compiler; these two things added together do add up in comparison with hand coded programs and it is that which we need to strive to compete with.

Using IX as frame ptr (although as I said you may not want to rule out completely) makes things a *lot* slower and bigger because index register instructions involve *at least* one extra byte in the opcode. It is why z80 programmers avoid this sort of thing. z88dk does not reserve any registers at all which allows very clean interfacing with asm code and that is something I think sdcc will need at some point; the costs of not doing so are not trivial. If IX cannot be abandoned, the compiler should insert push ix and pop ix around the call to lib functions that modify IX (but only those that modify IX; and other important registers) in order to avoid the nastiness of something like FRAGMENTA above. This will mean 4 bytes of penalty per lib function call but I think that is preferable to the execution time penalty of all that extra decorator code.

Anyway I hope that has helped identify one of the problems from asm programmers.



------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

[Philipp Klaus Krause]

On 26.03.2012 09:20, alvin (alvin_albrecht@...) wrote:

No z80 compiler
has ever (as far as I know) been written to investigate alternative
stack frame strategies or even alternative code sequences for common
operations that may fit better with register allocation or stack
frame usage. That's where I would like to see things go but I
digress

Do you mean things like this:
For multiplication we can generate code sequence A or B. A is shorter by
1 byte, but needs an additional free register, so the compiler chooses
between the two to get the minimal total code size, depending on if the
register could be used to hold a value to reduce code size further
elsewhere?

Philipp


------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev