Future of the libraries

Other misc things
User avatar
dom
Well known member
Posts: 1702
Joined: Sun Jul 15, 2007 10:01 pm

Future of the libraries

Post by dom »

The direction of movement is pulling bits of newlib into classic rather than the other way round.

It’s a slow process but I’ll get there eventually.

Keyboard reading is easily done with input.h and direct screen printing is pretty simple so I don’t think there’s much need to add anything.
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Simple print string at screen location?

Post by nuc1e0n »

That direction of movement is the opposite of what I'd expect.

So what are the purposes of newlib and the classic c lib considered to be at present?

Is the newlib considered to be an unstable testbed for new approaches that shouldn't be used for end-user code unless testing out the lib itself or is it considered to be the future library that everyone should be using for preference? newlib is contained in the "_DEVELOPMENT" folder in libsrc right?

My understanding is newlib's stdio functionality is more similar to that of larger and more complex C compilers and that the aim is to make it more compliant with the C11 standard.

However, I can see that a lot of beneficial newlib code has now been 'backported' to the classic lib.
derekfountain
Member
Posts: 114
Joined: Mon Mar 26, 2018 1:49 pm

Re: Simple print string at screen location?

Post by derekfountain »

nuc1e0n wrote: Tue Feb 15, 2022 6:09 pm That direction of movement is the opposite of what I'd expect.
So what are the purposes of newlib and the classic c lib considered to be at present?
Yes, I always thought the clue was in the names. :) Classic library stuff would be moved into the new libraries in due course.

I thought the standards compliant future would be SDCC and newlib. I'm not sure that's the case any more.
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Simple print string at screen location?

Post by nuc1e0n »

I say its fine if the roadmap has changed (porting all the classic libs for all platforms would be a huge job I think if that has been the goal), but personally I'd like to better understand what the current situation is.
User avatar
dom
Well known member
Posts: 1702
Joined: Sun Jul 15, 2007 10:01 pm

Re: Simple print string at screen location?

Post by dom »

It’s probably worth looking at the points here: https://github.com/z88dk/z88dk/wiki/Cla ... ew-library

Classic has, I think, already absorbed all of the newlib c library with the exception of stdio and some target specific libraries such as Bifrost.

In terms of stdio I think the key differences from a user perspective are errno handling, file support and some esoteric format handling for printf/scanf. Given the lack of fileio in newlib, this thread(!), that a lot has changed since newlib was created and no one has taken advantage of the potential of the newlib stdio I have a hunch that those features aren’t used.

The other major thing is of course that Alvin is now working on the ZXNext and has no time for z88dk and I find it much easier to bring up new targets with classic.

So in a way, newlib has been a proving ground for a bunch of concepts that have been absorbed and applied to a 100+ targets.

Realistically, given the available maintainer resource, new features added to newlib will either be ported from classic or added to classic at the same time.
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Simple print string at screen location?

Post by nuc1e0n »

So @derekfountain, at this point does it make much practical difference if you switch your work to use the classic lib if most of the benefits of newlib have been absorbed into classic already?

I must admit that when I look at the z88dk codebase as a whole, I find it much easier to work with the classic lib stuff myself as well.
Timmy
Well known member
Posts: 276
Joined: Sat Mar 10, 2012 4:18 pm

Re: Simple print string at screen location?

Post by Timmy »

nuc1e0n wrote: Tue Feb 15, 2022 9:58 pm So @derekfountain, at this point does it make much practical difference if you switch your work to use the classic lib if most of the benefits of newlib have been absorbed into classic already?
That is a really good question. Are you suggesting that one of the libs is completely useless?
I must admit that when I look at the z88dk codebase as a whole, I find it much easier to work with the classic lib stuff myself as well.
I'm glad you've admiited that. I must say your comment has possibly a lot of value.
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Simple print string at screen location?

Post by nuc1e0n »

No I'm not. I'm looking to see some resolution for derek's frustrations, not to mention my own frustrations around the newlib/classic dichotomy.

What's your view on this Timmy?
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Simple print string at screen location?

Post by nuc1e0n »

Come to think of it, as frustrating as this may be, projects needing to change direction due to team members moving on is a fact of life. The info on this thread is good to know.
User avatar
jorgegv
Member
Posts: 149
Joined: Wed Nov 18, 2020 5:08 pm

Re: Simple print string at screen location?

Post by jorgegv »

Indeed it is good to know! I started coding RAGE1 with classic and then moved on early to newlib for the (presumed) better support for really minimal CRTs, but now I'm considering going back to classic due to banking support.

If newlib is indeed some kind of beta area, and most of it is already in classic (and it is intended to be this way in the future), I would switch immediately to Classic+SDCC. Direct banking support would allow me to remove some complex code from my library (I developed my own banking system based on SDCC+NEWLIB)

So waiting for confirmation from the Z88dk masters...
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Simple print string at screen location?

Post by megatronuk »

Personally I was looking at the wiki docs a few months ago when my latest development idea was getting formed in my mind and thought from the description that newlib was the direction of travel.

The distinct lack of fileio support meant that I had instead chose classic as the target for the things I wanted to do (ZX, MSX-DOS banking, CPC), to make the multi-platform nature of what I was wanting to do a little easier (as long as I have open, read and close, I'm pretty much golden for cross platform data use).

It's good to hear that classic is actually the intended route forwards and that elements of newlib are instead making their way back in.
Timmy
Well known member
Posts: 276
Joined: Sat Mar 10, 2012 4:18 pm

Re: Simple print string at screen location?

Post by Timmy »

I was here when the newlib got introduced, and I think it was stated very quickly that it was still under development. In fact the whole newlib was created under the DEVELOPMENT directory. It was made very clear here in the forums that it was still unfinished, and that the "new" in newlib was just a prefix that described that it was created at a later moment than the classic.

The newlib tried a lot of things, and I think it was said that it might be a replacement for the old one when it might become mature one day.
In the end, this never really happened. Probably because a lot of old code still used classic and this was what most old open-source z88dk code used. Some people didn't switch to the new one, because a lot of code changes would have to be applied to their code. (Backward compatibility.) And perhaps also, is that the newlib only was working on very few platforms (if I remember correctly). Lastly, classic is just much easier to support but newlib has more new libraries added.

So in the end, newlib was innovative but classic was just more stable, has more platforms supported, and has more backward compatibility.
This has also been stated many times here in the forums if I remember correctly.

But I'm glad that we're all on the same point now and that we can all set on classic and this would make it easier to share code from now on. :)
Timmy
Well known member
Posts: 276
Joined: Sat Mar 10, 2012 4:18 pm

Re: Simple print string at screen location?

Post by Timmy »

It seems I can't already edit my post so I will just add that:
This is purely my observations as a long time user, and might not be the same as the the thoughts from the developers.

What I do like about classic is that I could take my 10 year old code and still can compile it (with several modifications, see another thread), and I could even port some of it to other platforms like MSX (Heart Stealer 2 that I've released last year for MSXDev was a port from my Spectrum code), and without much change in the game logic parts of my code.

I might add that I do like the idea of newlib; it was really innovative and it added a lot of features that just weren't available on classic for a long time.

Note also that I'm still using z88scc (I forgot the name, but the not sdcc version), with classic. It's not as efficient as sdcc but it works for me.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Simple print string at screen location?

Post by megatronuk »

Timmy wrote: Wed Feb 16, 2022 3:42 pm.

But I'm glad that we're all on the same point now and that we can all set on classic and this would make it easier to share code from now on. :)
Definitely!
User avatar
dom
Well known member
Posts: 1702
Joined: Sun Jul 15, 2007 10:01 pm

Re: Simple print string at screen location?

Post by dom »

Thank you all for the comments, I think I said pretty much the same last year: viewtopic.php?p=19605#p19605

To stop any panic, newlib won't be going anywhere, similar to Timmy's comment there's many projects built using it and removing it would cause much breakage. However, I personally have got no plans to add new targets or add features solely to it though I'm happy to accept contributions that do.

Thinking about things further, the end-point should be a state where you don't know/don't care what library you're using. There's a really good example of this with the rc2014 target. The 8085 subtype actually uses the classic library with the newlib crt0.

However, we're not there yet, so thinking about use-cases. I suppose there's the following (and I'll put names on them if you don't mind):

- derekfountain: Used newlib for years, happy with what's there -> carry on with newlib (though please let me know any crt0 features you depend on)
- jorgegv: Starting up a new library, want to take advantage of new features, maybe want some more things in z88dk out of box -> use classic (sorry!)
- Timmy: Targetting multiple platforms -> use classic
- nuc1e0n: Need file operations + multiple targets -> use classic
- a.n.other: Starting out with z88dk no existing codebase -> use classic
- a.n.other II: Want to target CP/M -> use classic
- a.n.other III: targeting a naked cpu -> use newlib (+z80/+z180 targets)

Another use-case is targeting ZXNext, and I don't think I have a good answer for that one yet.
Timmy
Well known member
Posts: 276
Joined: Sat Mar 10, 2012 4:18 pm

Re: Simple print string at screen location?

Post by Timmy »

dom, Thanks for the clarification!

I feel like that this discussion was pretty important to let it become hidden under this thread.

Can I suggest we make another thread to indicate we had a discussion between newlib and classic in some more visible part of the forum, and then include a link to this thread?

(I could probably make an announcement in the Announcements part of the forum but I don't know if I have the permission to do it.)

I have the feeling this would clear out many misunderstandings for other people who might have missed this discussion.
derekfountain
Member
Posts: 114
Joined: Mon Mar 26, 2018 1:49 pm

Re: Simple print string at screen location?

Post by derekfountain »

dom wrote: Wed Feb 16, 2022 10:45 pm - derekfountain: Used newlib for years, happy with what's there -> carry on with newlib (though please let me know any crt0 features you depend on)
Well, not really to be honest.

I settled on newlib for my ZX tutorial series because the situation with dual compilers and dual libraries was complicated and I wanted to simplify for the new starter (which I myself was at the time). Newlib, as the name implies, was the future, so that's the one I went with. But things have worked out differently. Classic has a lot more features than newlib. This discussion started with my wanting a simple way of printing a character to the screen. I've also wanted the simple sprite routine in classic, which isn't in newlib. And some of the graphics primitives. I saw the other day that classic has a IM2 interrupt setup routine, a task which I've been doing manually from newlib.

My assumption has always been that these things, and plenty more besides, would eventually be ported into newlib, but that now seems to not be the case. In fact, things are going the other direction. As dom's list earlier in the thread shows, there seems to be little reason to stick with newlib now, and my next project will almost certainly use classic.

None of which is criticism. Stuff happens, projects change direction, we all know that. I glad we've got some clarification. :)
User avatar
jorgegv
Member
Posts: 149
Joined: Wed Nov 18, 2020 5:08 pm

Re: Simple print string at screen location?

Post by jorgegv »

dom wrote: Wed Feb 16, 2022 10:45 pm - jorgegv: Starting up a new library, want to take advantage of new features, maybe want some more things in z88dk out of box -> use classic (sorry!)
...
Another use-case is targeting ZXNext, and I don't think I have a good answer for that one yet.
Well, thanks for the confirmation. I'm not starting a new library now, but I'll happily migrate RAGE1 to classic now that the roadmap is clear. I'll have some immediate benefits and now we've confirmed that the future is classic.

Regarding ZX Next, what are the issues with it? I'm close to receiving my own ZX Next hardware, and I intend to port my RAGE1 lib to the Next sometime in the near future, so any advise about the better direction for that would be very nuch appreciated...

Thanks a lot again for all the info.
User avatar
dom
Well known member
Posts: 1702
Joined: Sun Jul 15, 2007 10:01 pm

Re: Simple print string at screen location?

Post by dom »

jorgegv wrote: Thu Feb 17, 2022 10:16 amRegarding ZX Next, what are the issues with it? I'm close to receiving my own ZX Next hardware, and I intend to port my RAGE1 lib to the Next sometime in the near future, so any advise about the better direction for that would be very nuch appreciated...
Although zxn support in classic is already at a good level, I've not got a general answer (yet) for a few reasons:

1. I think there's some parts of the library that I've not yet pulled over
2. I'm not certain that classic can or correctly create dotn or dotx files
3. I believe AA gives quite a lot of support on zxn usage in other places

However, as is/should be well known, I'm always up for improving the functionality of a target, so any missing things can be added to classic as you need them - I won't leave you hanging!
User avatar
jorgegv
Member
Posts: 149
Joined: Wed Nov 18, 2020 5:08 pm

Re: Simple print string at screen location?

Post by jorgegv »

dom wrote: Thu Feb 17, 2022 12:03 pm However, as is/should be well known, I'm always up for improving the functionality of a target, so any missing things can be added to classic as you need them - I won't leave you hanging!
Note taken, thanks in advance! :-)
derekfountain
Member
Posts: 114
Joined: Mon Mar 26, 2018 1:49 pm

Re: Simple print string at screen location?

Post by derekfountain »

dom wrote: Thu Feb 17, 2022 12:03 pm 3. I believe AA gives quite a lot of support on zxn usage in other places
Oh, not wanting to hijack the thread, but can you say where? I have a Next arriving, erm, last August, so it should be here soon!

I've made a point of not looking at z88dk for the Next until I have all the kit but I'm itching to start learning.
User avatar
dom
Well known member
Posts: 1702
Joined: Sun Jul 15, 2007 10:01 pm

Re: Simple print string at screen location?

Post by dom »

derekfountain wrote: Thu Feb 17, 2022 12:37 pmOh, not wanting to hijack the thread, but can you say where? I have a Next arriving, erm, last August, so it should be here soon!
I don't know for certain, some issues get raised from time-to-time. I presume it's via Facebook or discord.
User avatar
feilipu
Member
Posts: 33
Joined: Tue Nov 15, 2016 5:02 am

Re: Future of the libraries

Post by feilipu »

Thinking about things further, the end-point should be a state where you don't know/don't care what library you're using. There's a really good example of this with the rc2014 target. The 8085 subtype actually uses the classic library with the newlib crt0.
Just to comment on this and the newlib targets that I've been curating.

The newlib stdio is very advanced and TBH I don't completely understand how it works. So although the rest of the newlib infrastructure was fairly straight forward to incorporate into the rc2014_8085 subtypes, it was just so much easier to use the classic style stdio implementation. Particularly, given this target needs to use the 8085 capabilities of sccz80.

For the newlib targets (including hbios, rc2014, scz180, yaz180) there is some imho useful driver code that could be moved into a combined library at some stage. The hbios interface is for ROMWBW native applications. The scz180 has code for native SPI (SD) interfaces, and the yaz180 has an interrupt driven far memory interface to the Am9511A (fairly proud of that one, though its is useless generally). And all targets have the 82c55 and diskio interface code for FatFS.

Something that needs to be fixed in newlib is the presence of floating point maths libraries that are polluting the target directory. Somewhat responsible for this mess, but I'm not sure how to fix it. Essentially the am9511, math32, math16, and m (math48) libraries need to disappear into the infrastructure somehow. It wasn't a problem when there was just the m (math48) library, but now it is just confusing.

Another thing I'd like to see is a better integration path for external libraries. Currently I maintain FreeRTOS and ChaN FatFS code outside of z88dk (and others), and install compiled libraries using z88dk-lib. There are some drawbacks (limitations) with this approach, such as only one header file being installable, but it is what is available now. The installation path makes it a little complex for anyone to use these well developed and supported projects. I feel it could be easier, and therefore more accessible to all.

My 2c.
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Future of the libraries

Post by nuc1e0n »

The hbios interface is for ROMWBW native applications. The scz180 has code for native SPI (SD) interfaces, and the yaz180 has an interrupt driven far memory interface to the Am9511A (fairly proud of that one, though its is useless generally). And all targets have the 82c55 and diskio interface code for FatFS.
So for my own understanding, these libraries enable the following things right?

1. Generic low level access to SD cards (SPI)
2. Generic high level access to FAT filesystems. Potentially using the generic low level SD card access code to access files on an SD card?
3. Hardware floating point math support using a AM9511A chip?

So with the sd card and fat code, a device could be made to plug in under any z80 and give it the ability to read and write files on an SD card with a program compiled using z88dk?

I for one would like to know more about that possibility generally.
Essentially the am9511, math32, math16, and m (math48) libraries need to disappear into the infrastructure somehow
This may not be relevant but I can see in z88dk/lib/clibs/ there are a bunch of .lib files, including math12.lib, math32.lib, math48.lib, mzx.lib. I believe these contain the implementations of math functionality that can be linked to right? I think these libs can be linked to using a zcc command line parameter such as "-lmzx" for the zx spectrum rom math functionality for example. I can see a similar set of .lib files in z88dk/libsrc/_DEVELOPMENT/lib/sccz80/ also. In addition there are the folders z88dk/libsrc/math/ and z88dk/libsrc/_DEVELOPMENT/math/float/ that seem to contain different options for math on classic and newlib respectively.

I'm not sure what you mean by "disappear into the infrastructure somehow"?
Another thing I'd like to see is a better integration path for external libraries.
This is something I'd also like to see myself. I can see in the MSX target on classic there's a really useful function named "AsmCall" which enables calling arbitrary memory locations while setting and retrieving the register values for you. I'd like to be able to use this on other targets, but it seems unclear to me whether I can or not.

Also, the c128 target on classic has some really neat functionality to read black and white .pcx images as part of something called "sgtools". But there again I'm not sure whether these functions would be useable from other targets.

A more general means of making z88dk's libraries less monolithic may already exist, but I'm not aware of it if does.
User avatar
feilipu
Member
Posts: 33
Joined: Tue Nov 15, 2016 5:02 am

Re: Future of the libraries

Post by feilipu »

So for my own understanding, these libraries enable the following things right?
1. Generic low level access to SD cards (SPI)
2. Generic high level access to FAT filesystems. Potentially using the generic low level SD card access code to access files on an SD card?
3. Hardware floating point math support using a AM9511A chip?
Yes. That's right. AFAIK, the newlib is supposed to be assembly only, so all of the high level things (in C) are supposed to be in external libraries that are installed using z88dk-lib. So that's the way they're implemented.
1. Generic low level access to SD cards (SPI).
The scz180 has access to a SD card via SPI using its CSIO interface. Potentially this could be extended to other z180 devices.
Any of the targets (eg rc2014, yaz180) that implement a PIO 82c55 device can access IDE (SSD, DOM, CF) devices.

Both of these are low level support for the IDE drivers to the diskio sub-system necessary to support reading FAT file systems.
2. Generic high level access to FAT filesystems. Potentially using the generic low level SD card access code to access files on an SD card?
Once the diskio routines are available, then the standard ChaN FatFS library can be used to provide a very robust disk system. I have it built for z80 and 8085 in my external libraries, and configure them for installation using z88dk-lib.
3. Hardware floating point math support using a AM9511A chip?
Support for the Am9511A (i8231) APU is available for the rc2014 and yaz180. The Am9511A is very configurable, and can be accessed in 4 different modes. For the yaz180 I implemented the interrupt mode, and allowed the driver to access all the z180 1MByte memory space. For the rc2014 I implemented the polling mode, and this is much more useful generally.

I had hoped that the rc2014 drivers could be used generally with other platforms, but I fell at the first hurdle with the DAI as it uses a memory mapped APU rather than I/O mapped access.
Essentially the am9511, math32, math16, and m (math48) libraries need to disappear into the infrastructure somehow.
The issue is that newlib uses the target directory to configure the build of libraries. The configuration of targets in newlib is very powerful, and something I appreciate. So even though the library code is in the "_DEVELOPMENT/math/float" directory, the build instructions appear both in the newlib target directory and in the classic libraries directory. The info appears twice, and files are listed twice.
A more general means of making z88dk's libraries less monolithic may already exist, but I'm not aware of it if does.
Using z88dk-lib is a fairly good way of building and maintaining a range of different libraries. The one constraint I have is that only a single header file can be automatically installed but I can live with that.

It is a good way to support C libraries that take an age to compile (FatFS takes about 10 minutes to compile, for example, and multiply that by target and library sccz80, sdcc_ix, sdcc_iy) and so are impractical to include as standard. And the user will want to configure the functions included themselves anyway.

But we're still a bit limited by the fact that we can't separate C files with multiple functions into individual sections, so that the linker can discard unused functions. So you have to guess which functions are needed in a library and compile it specific to the need. This makes things a bit unweildy.

2c more.
Post Reply