Developing for MSX-DOS2 and >64kb

megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Developing for MSX-DOS2 and >64kb

Post by megatronuk »

I have read through the example files for the SMS/zx/CPC/MSX that show the function banking features... And I think I understand how they can be made use of for a ROM target, however the game I am developing is intended for a disk target; I get that I will need to structure my application so that I have, at minimum 1 bank permanently swapped in containing essential, non swappable global data/code, with the other banks swapped in from their pragma defined bank numbers (with care in structuring the application so that I don't need mor banks swapping in than I have free at any given time - obvious candidates for being excluded from the unswappable banks would include, I assume, graphics assets and/or audio)

Are there any hooks in z88dk at present to support additional 16kb code, data or mixed code/data banks for an MSX-DOS2 target?

I haven't read nearly as much MSX technical docs as I should have, but there is reference to memory mapper functions in DOS2, but from what I can make out, this is intended for runtime allocation of additional ram. Is there something as simple as the banking example code for non-ROM targets?
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

I too have never read that section of the MSXDOS documentation either, I've had a quick scan through and can see that it's not as trivial to do banking in a DOS environment as in a ROM environment but still very possible.

As you've spotted, banking is achieved by a few things:

a) A set of defined sections: https://github.com/z88dk/z88dk/blob/mas ... egarom.asm
b) The banked_call routine: https://github.com/z88dk/z88dk/blob/mas ... m.asm#L141
c) A loader (for non ROM targets): https://github.com/z88dk/z88dk/blob/mas ... 0.asm#L148 (that's for +zx)

The bank numbers are set at link/compile time, however DOS will give back a segment number that corresponds to free memory so you'd need to maintain a mapping between the binary bank number and the DOS segment number which is used by the banked_call code.

It all sounds very achievable, it might take me a few weeks to get round to doing it myself, so feel free to give it a go yourself.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Yes, it appears the DOS2 mapper functions were introduced to try to bring some level of consistency to the various mappers that were available on the MSX platform, i.e. since the RAM/ROM mapper implementations seem to have created quite a mess on the platform - DOS seems to do the interrogation of the available mappers on boot, and present them in a consistent way.

There are a couple of DOS2 function calls that make accessing the extra blocks easy, plus a couple of calls that read and write bytes directly.to those banks (withhout being swapped in, from my understanding). It sounds ideal for loading additional data into those areas, a bit like the old extended memory calls on MS-DOS, but perhaps not quite as easy to use for additional code areas as th ROM banking methods.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

I had a tinker with this last night, but ran into the issue that the primary mapper couldn't give me any segments which makes me think I've misunderstood something: I presume the primary mapper is responsible for the onboard memory beyond 64k? Or did it create a RAM disc to use that memory up and I need to find a new emulator?
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

My understanding is that MSX-DOS2 required 128kb of RAM, and that it was primarily MSX2 machines that satisfy that criteria (and only a subset of those - since there are quite a few non-European models with only 64kb). I guess that's one of the problems with the MSX1|2|2+ being a loose collection of standards rather than a single manufacturer produced machine.

I don't know if the primary mapper on 128kb base machines (i.e. unexpanded via cart) can support segments above the built-in 128kb, but it should certainly be able to use them in the 64-128kb region. I don't know if MSX1 models have that functionality, but it should certainly be part of any MSX2 with 128kb from new.

On my (real) Sony HB-FX1DJ which has an internal expansion to 256KB, MSX-DOS2 (or, more accurately; Nextor - which I understand is DOS2 compliant) correctly detects all of that ram, without the use of an addon cart with it's own mapper (like my Megaflashrom, for example).... and it's definitely the built-in mapper doing that, rather than a cart. Plug in the MFR and it detects 768KB via the additional 512KB mapper in the cart.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

Ah right, if DOS2 requires 128kb, then I've got no free memory on a 128kb emulator. That matches with my observations at code and debug level.

So, I need to find a new emulator to go any further.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

In case you want to go further whilst I'm playing around with emulators I've attached the msxdos.asm I've been working on. It's got a lot of logging in so I can see where it breaks.

The basic idea is to allocate a segment, load the bank into it and keep track of the mapping between the z88dk bank id and the MSX segment. This mapping is then used in banked_call code.

appmake has been updated to write banks to disc, so the banked example should be good if you change the MSX cflags to -subtype=msxdos2
msxdos.asm.gz
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

I'd love to say I could look at this, alas I'm purely a C guy; the only assembly I have experience of was a m68k introductory course some 25 years ago at university... And it has long been pushed out of my brain by mundane trivia about which TV shows my kids like or how to operate the washing machine! :-D

It definitely sounds like you're making progress, however.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

Well, after a couple of tweaks it looks like it works:
Screenshot 2022-02-02 at 22.18.52.png
The code does need a cleanup/tune etc, but I'm having trouble finding the combination of working emulated hardware and a debugger, so I'm leaving it as a working proof of concept.

The configuration provides a 32kb home page (from $100 to 32767) and 32768->49151 is banked.

I've pushed the change, to use it you'll need to compile with the flags: -subtype=msxdos2 -pragma-define:CRT_DISABLELOADER=0
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Wow! That was fast work. I'll pull down the latest version and recompile this weekend.
I'm putting the meat on the bones of my Sinclair QL target of my rpg engine, and am starting to design the MSX implementation on paper right now ... The extra capacity afforded by dedicated vram will allow me to do some proper animation and the banking support should let me swap in and out an audio player as well as extra effects that just wouldn't be possible in an unbanked implementation (and for me, with lots of on-disk assets, a ROM is not the way to go). This looks really promising.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

In theory a ROM could be up to 4MB in size which is the same size as I've allocated for the DOS banking.

However, we don't, and I've not come across, a portable ROM filesystem that supports compression

Let me know how it goes.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Okay, pulled down the latest commit and rebuilt this morning. However, like you, I'm struggling to find a combination of hardware models and extensions in OpenMSX that I can get this working on :(

On my real MSX2+, with (256kb onboard + 512kb Megaflashrom ram - including Nextor DOS), it doesn't get as far as printing the "Printing from main memory bank" message, the (real) floppy churns for a second or two after calling bank.com, but then hangs:
hbf1xdj.png
The memory arrangement on this machine is:
hbf1xdj_slots.png
You do not have the required permissions to view the files attached to this post.
Last edited by megatronuk on Sat Feb 05, 2022 10:31 am, edited 1 time in total.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

In OpenMSX I've tried the following:

Config 1. NMS-8255 (MSX2, 128kb onboard ram, added a Nextor 2.11 ASCII16 rom)
nms8255.png
... hangs... and then after about 60 seconds prints a several screens worth of this:
nms8255_garbage.png
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

I also tried this config,

Config 2. NMS-8255 (MSX2, 128kb onboard ram, added the ASCII MSX-DOS2 cartridge.

But it seems to have the same behaviour as my real MSX2+, it just hangs indefinitely:
nms8255_1.png
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Another result;

Config 3. OpenMSX, Sony HB-F1XDJ (64kb base, +512kb mapper option, +Nextor ASCII16 standalone rom)
hbf1xdj_omsx_nextor.png
This one actually does print the "Printing from main memory bank" message, but then does nothing more... a few seconds later OpenMSX shows the crash message seen in the screengrab.
You do not have the required permissions to view the files attached to this post.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

I’m not convinced that the disc layout is correct for openmsx which would explain what you’re seeing.

Without a working debugger it’s tricky to fix though.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Developing for MSX-DOS2 and >64kb

Post by dom »

I should probably also restore the log messages to give a bit more clue as to what’s going on.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Okay, I think I've got a working configuration for OpenMSX....

Sony MSX2+ HB-F1XDJ (64kb base + 512kb mapper + DOS2 rom (both the downloaded Nextor rom and the bundled msxdos2 extensions work))
hbf1xdj_omsx_nextor_freebanks.png
Philips NMS-8255 (128kb base + NO mapper + DOS2 rom (both Nextor rom and the msxdos2 extensions work))
nms8255_2_free_bank.png
As expected, there aren't enough free banks remaining in a base 128kb configuration - although there's enough for one, it seems; that at least gets us 52kb + 16kb working memory, which is more than you would normally be able to get on a MSX-DOS2 system with the minimum amount of installed memory.

I'm still trying to get it to run on my real hardware... I am wondering if the Megaflashrom mapper is working differently to the 512kb mapper 'cart' that OpenMSX uses. I've tried replicating the Megaflashrom inside OpenMSX, but I can't get it to add it's own memory to that of the machine.
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

BTW, the above is with copying bank.com and the bank files to a MSX-DOS2 formatted bootable floppy, not bank.img as created from the example directory.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Okay, good news. The modified disk works on a real machine!
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Booting from my Megaflashrom / hard drive image on SD card, and then loading the test banking code from a real formatted MSX-DOS2 floppy:
real_msx2_banking.png
https://www.youtube.com/watch?v=7Ctat4j8sqQ

I don't follow the reason for the original failures - perhaps it was disk geometry or something weird like that, but that's now 3 for 3 on the two OpenMSX systems and this one real one.
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

I have expanded the banked example to 8 banks (which in theory should work on a MSX-DOS2 machine with 256kb or more - though possibly 192kb in practice), functions bank1 and bank2 are identical to the original code. bank3 through bank7 call the next bank in turn with a value, as bank3 did in the original code, e.g.

Code: Select all

#define BANK 3
#pragma bank 3
#endif

int func_bank3(int value) {
    printf("Printing from bank%d - passed value %d\n",BANK,value);
    return(func_bank4(3));
}
(and so on, in turn, so each function should print out the id of the function that called it in turn)

Then the final func_bank8 returns and should collapse the call stack back to main(), just like the original:

Code: Select all

#if __CPC__
#define BANK 7
#pragma bank 7
#else
#define BANK 8
#pragma bank 8
#endif

int func_bank8(int value) {
    // printf is in common code
    printf("Printing from bank%d - passed value %d\n",BANK,value);
    return(8);
}
Each bank function gets passed the ID of the function that called it (func_bank4 gets called with the value 3, func_bank5 with 4, and so on)

It compiles, but instead of the func_bank3 then calling the function from bank4 and onwards, OpenMSX crashes with a 'DI' halt error when func_bank4 is (or is about to be) called:
bank_calls.png
You do not have the required permissions to view the files attached to this post.
megatronuk
Member
Posts: 34
Joined: Fri Jan 28, 2022 11:39 pm

Re: Developing for MSX-DOS2 and >64kb

Post by megatronuk »

Also, an interesting difference in behaviour on a real system when attempting to use 8 banked functions as above - rather than hanging like OpenMSX does, it appears that bank.com terminates and simply returns to command2.com:
bank_calls_8banks_real.png
It's definitely terminating abnormally, as it doesn't return to main() to print the "Returned value from bank calls is..." string. It should be doing the nested call sequence all the way to func_bank8().
You do not have the required permissions to view the files attached to this post.
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Developing for MSX-DOS2 and >64kb

Post by nuc1e0n »

Using ASMLIB that's been built into Z88DK for a couple of years now banking in the MSX can be performed already. This banking also works on MSX1 computers if the MU.COM TSR has been loaded on MSXDOS1 (https://www.msx.org/downloads/mu-v105-w ... translated). ASMLIB hasn't really been documented much on the wiki however so I've converted it's documentation to markdown format and linked to it from the MSX platform page (https://github.com/z88dk/z88dk/wiki/Platform---MSX).

Here's some sample code of mine that does banking with ASMLIB:

Code: Select all

#include <arch/z80.h> /* for asmlib */

#define ALL_SEG 0x00  /* Allocate a 16K segment */
#define FRE_SEG 0x03  /* Free a 16K segment */
#define RD_SEG  0x06  /* Read 1 byte from the segment specified by A at the address pointed to by HL */
#define WR_SEG  0x09  /* Write 1 byte from E to the segment specified by A at the address pointed to by HL */
#define CAL_SEG 0x0c  /* Inter-segment call */
#define CALLS   0x0f  /* Inter-segment call */
#define PUT_PH  0x12  /* Put a segment on the bank specified by HL */
#define GET_PH  0x15  /* Get the segment number of the bank specified by HL */
#define PUT_P0  0x18  /* Put a segment on bank 0 */
#define GET_P0  0x1b  /* Get the segment number of bank 0 (0000h~3FFFh) */
#define PUT_P1  0x1e  /* Put a segment on bank 1 */
#define GET_P1  0x21  /* Get the segment number of bank 1 (4000h~7FFFh) */
#define PUT_P2  0x24  /* Put a segment on bank 2 */
#define GET_P2  0x27  /* Get the segment number of bank 2 (8000h~CFFFh) */
#define PUT_P3  0x2a  /* No effect */
#define GET_P3  0x2d  /* Get the segment number of bank 3 (C000h~FFFFh) */

extern char hasMapper;
extern unsigned char loadPageStatus;
extern unsigned char defaultBank;
extern int mapperJumpTable;
extern Z80_registers regs;

void dosload(int pageNumber) __z88dk_fastcall {
  const char * filename = "prefix00.ovl";

  static int temp;

  if(hasMapper) {
    if(loadPageStatus == 0xff) {
      /* no ram block has been allocated to this segment. try to allocate one now */
      regs.Bytes.A = 0;   /* allocate user segment */
      regs.Bytes.B = 0;   /* allocate from primary mapper */
      AsmCall(mapperJumpTable + ALL_SEG, &regs, REGS_MAIN, REGS_AF);
      regs.Bytes.B = 0;

      if(regs.Flags.C) {
        /* allocation failed. Switch to the default bank to load the code into that */
        regs.Bytes.A = defaultBank;
      }
      else {
        /* allocation suceeded. switch to the bank number that was returned */
        loadPageStatus = regs.Bytes.A;
      }

      AsmCall(mapperJumpTable + PUT_P1, &regs, REGS_AF, REGS_NONE);
    }
    else {
      /* a ram block has already been allocated to this segment. just switch to it and don't load anything */
      regs.Bytes.A = loadPageStatus;
      AsmCall(mapperJumpTable + PUT_P1, &regs, REGS_AF, REGS_NONE);
      return;
    }
  }

  sprintf(filename + 6, "%02d", pageNumber);
  filename[8] = '.';

  if((temp = open(filename, O_RDONLY, 0)) == -1) {
    fputs_z80("Couldn't find ", stderr);
    fputs_z80(filename, stderr);
    exit(EXIT_FAILURE);
  }

  read(temp, (void *)16384, 16384);
  close(temp);
}
I also have some assembler code that's run on startup that detects whether the memory banking extension is available (based on information gleaned from https://www.msx.org/wiki/Extended_Bios). This assembly code could be ported to C fairly easily with the bdos function :

Code: Select all


_initMapper: ; detect if a msx2 compatible mem mapper is present
  ; call CPM_VER. msx computers always return 0x22 but still implement more bdos calls than real cp/m 2.2 does
  ld c, 0x0c
  call 0x0005
  cp 0x22
  jr nz, noMapper

  ; test for whether this code is running on an MSX computer by calling MSX_DOSVER
  ld a, 1
  ld c, 0x6f
  call 0x0005
  or a
  jr nz, noMapper

  ; test for presence of extended bios
  ld a, (HOKVLD)
  bit 0, a
  jr z, noMapper  ; no extended bios

  ; call GET_VARTAB to test for msx2 mapper support
  xor a
  ld de, 0x0401
  call EXTBIOS
  or a
  jr z, noMapper  ; no mapper support if a = 0

  ; call GET_JMPTAB and store the resultant address
  ld de, 0x0402
  call EXTBIOS
  ld (_mapperJumpTable), hl

  ; store that the mapper was detected
  ld a, 1
  ld (_hasMapper), a

  ;store the default page segment number
  ld de, 0x0021 ; get the segment number selected on second page (GET_P1)
  xor a
  add hl, de
  call l_jphl
  ld (_defaultBank), a

noMapper:
  ret
 
code to detect the width and height of the console on MSXDOS 1 and 2:

Code: Select all

#include <cpm.h>
#define MSX_DOSVER 0x6f

...

if(
    bdos(CPM_VERS, 0) == 0x22 && /* MSX computers return 2.2 as their CP/M version */
    bdos(MSX_DOSVER, 0) == 0 && /* only MSX computers return 0 for this call */
    /* MSX computers store the console size directly */
    (width = *((unsigned char *)0xF3B0)) != 0 &&
    (height = *((unsigned char *)0xF3B1)) != 0
  ) {isMSX = 1; }
nuc1e0n
Member
Posts: 49
Joined: Wed Jun 10, 2020 12:34 am

Re: Developing for MSX-DOS2 and >64kb

Post by nuc1e0n »

Whoops. I forgot to include definitions for EXTBIOS and HOKVLD:

Code: Select all

EXTBIOS equ 0xFFCA
HOKVLD equ 0xFB20
Post Reply