CPMDISK again

[stefano]

While trying (once again unsuccessfully) to port a northstar emulator on Windows, I bumped into a couple of tools for the Microbee.
They are were named "alien" because they were able to exchange files working on several disk formats, one of the two even addressed the MSDOS/Apricot, NorthStar etc.
What impressed me was the cp/m related one, it supported a big number of disk formats and worked without modifying the machine's disk configuration.
I extracted the configuration table and I'm decoding it.

https://github.com/z88dk/techdocs/blob/ ... _alien.txt

The fields I was able to identify should be enough to complete what it is easy to spot with a disk tool like IMDU, but obviously you must have a disk image on file to analyse.
Hope I'll spot something else useful, like the skew tables and the sector sizes.

It confirmed the format parameters I got already for the Knight 2000 (aussie byte) and alienc.com picks the z88dk created file correctly running on the Microbee emulator. Still it's not good enough for MAME being able to load it correctly. There seems to be a small difference between the two disk sides on the original disk images, which I couldn't totally identify and recreate.
By the way I the new disk format is now available.

[stefano]

The Don Maslin archive includes the files for the GNAT System 10 computers.
I used them, partially unsuccessfully to test this new method of merging the results gathered from the IMDU convert log and the ones listed in my file (data obtained from ALIENC.COM).
It's not enough to build a valid disk image we can provide off the shelf, but it helps in extracting files from the disk dumps.

# Gnat System 10, IMDU RAW extraction (original format: 10 sectors, 2 sides)
# The 80 tracks version is a direct extension of the 40 track)
diskdef gnat
seclen 512
heads 1
tracks 80
sectrk 20
blocksize 2048
maxdir 128
os 2.2
boottrk 1
skewtab 0,1,6,7,12,13,18,19,4,5,10,11,16,17,2,3,8,9,14,15
end

The GNAT had a 9512 APU math coprocessor, and this disk includes a couple of diagnostic programs, t9512.com and tapu.com.

[stefano]

I'm pulling in a new option, side2_sector_numbering, useful to deal with those disk formats requiring a progressive sector numbering between the two disk sides. The Kaypro 4/10 double side disk format is a perfect test case (https://forum.vcfed.org/index.php?threa ... ost-696390).

Experimenting with appmake on new cpmdisk formats also helped me in spotting a limit on the D88 format, it had a fixed sector numbering offset (+1).

[dom]

I've been wondering if we should add the capability of reading the diskdefs file from cpmtools. There should mostly be a one-to-one mapping between > the parameters in that file and our structure.

cpmtools is many times used to read or edit diskette dumps rather than creating them, most of the definitions miss relevant details needed in a diskette dump to make a valid image file.
There are several versions of cpmtools around, loosely compatible each other which brings in further confusion (e.g. the version distributed in the "MicroBEE" website is very handy for the Windows users because it directly support many containers, including IMD, but IIRC misses some configuration trick).
In example, I don't think it has a parameter taking care of the sector numbering like the one I just introduced, it is just reading/writing the sector from/to the expected position. Nor it is dealing with the stepping mode or the transfer rate speed.

The "alienc" tool includes all the required information, it was able to format and fully edit alien diskette formats, but I decoded it only partially, enough to be a good alternative to the DPB.COM tool plus some extra detail.



I usually gather the word/byte addressing mode from the diskette dump, it's easy but I'm sure there are better ways to identify the directory type.. Looking online I think cpmtools is using the same standard CP/M logic to determine it: "The operating system determines whether to use 8 or 16-bit allocation entries by looking at the number of blocks on the disk. If it's less than 256 then it uses 8-bit allocation blocks. If it's more than 256 then it uses 16-bit allocation"

I'm not totally trusting this rule, but I admit I can't remember exceptions I met while dealing with disk images. Perhaps we could make appmake smarter and guess some of those parameters which are not specified.

EDIT2:
The Einstein format is a good example of an exception: "By telling cpmtools there are 103 tracks on the disk it will calculate there are 257 blocks and use 16-bit allocation block numbers. This works fine for us reading data off a disk as there can't possibly be blocks pointing that high up onto the (non-existent) parts of the disk."


EDIT:
There are even more subtile cases, like the Aussie Byte Knight2000 format where the files I'm generating are not "perfect" and the current versions of MAME can't read it properly. The emulator loads it and the DIRectory is properly shown, but CP/M hangs while loading a program. It is not the byte/word block addressing mode, I suspect that the two disk sides have a different stepping or similar.
The MicroBee's ALIENC.COM gets the file properly and once copied the program runs as expected, I can't say whether the real hardware would hang.

[stefano]

With the new "side2_sector_numbering" option I got a valid configuration for the GNAT disk format. At the moment it is a totally lost/forgotten system, but it passes the check with ALIENC.COM on an emulated microbee

[stefano]

We now have also a 40 tracks floppy mode for the microbee.
It required different space for the boot records and a different starting position in the sector skew sequence which is odd (the skew factor is always 3). Moreover both the 40 and 80 tracks images need to keep the sectors shuffled over a sequential sector numbering!

[stefano]

Yet another weird format, the 'Amust'.
Having the boot tracks formatted differently was more common on the 8" floppy drives, but when it happened on different media formats everybody had its personal way to go.
There's a good probability that keeping 5 big sectors on the first track (both the sides) would work for data disks, but to be 100% sure it should be tested on the real hardware or on a working emulator. The cpmtools in such cases will require 2 different definitions to skip the boot tracks, if a disk container is used it will be straightforward, but a raw dump will need an offset on the first bytes.



0/0 250 kbps DD 16x256
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
D D D D D D D D D D D D D D D D
0/1 D D D D D D D D D D D DE5 DE5 DE5 DE5 DE5
1/0 250 kbps DD 5x1024
1 2 3 4 5
D D D DE5 D
1/1 DE5 DE5 DE5 DE5 DE5
2/0 D D D D D
2/1 D D D D D
3/0 D D D D D
3/1 D D D D D
4/0 D D D D D
4/1 D D D D D
5/0 D D D D D
5/1 D D D D D
(...)

[stefano]

Those CP/M disks are even more different one each other than expected !

One of the biggest obstacle to support some rare format is the sector numbering.

Here
https://forums.debian.net/viewtopic.php?t=112244
--are mentioned 5 different methods supported by 22disk.

Another less tricky but totally uncovered case is the earlier Sharp MZ models, the diskdumps boot on the emulators but the data is recorded totally with inverted bits, and currently the common tools are hardly able to manage those disk images.
I'm close to a valid output, for those ones, inverting the output data on appmake and setting the parameters like the ones on ALIENC, I can launch DIR and see the filename listed on the MZ80B (which reads also the MZ80A disks), but the program does not run yet.
I know, there's still a lot to do before finding the right sector order, also because the sector number themselves were recorded with inverted bits!

mz80.png

[stefano]

I opened a last minute pull request for the Cortex disk format.
This one was relative to the Visual 1050, I couldn't test it directly but the disk format was in ALIENC.COM

As expected it is very similar to the Visual 1050 one, simply it was adjusted for 80 tracks, while the Cortex had 40 tracks disks, always single sided.

[stefano]

>> There are even more subtile cases, like the Aussie Byte Knight2000 format where the files I'm generating are not "perfect"
>> and the current versions of MAME can't read it properly. The emulator loads it and the DIRectory is properly shown,
>> but CP/M hangs while loading a program. It is not the byte/word block addressing mode,
>> The MicroBee's ALIENC.COM gets the file properly and once copied the program runs as expected, I can't say whether the real hardware would hang.

This could be a bug in appmake though, I'm trying to recreate one of the Sharp MZ3500 disk formats (CP/M, 256x16) using DISKDEF on the MZ-2500 simulator to test it, apparently appmake is doing weird things, the directory is apparently allocating too much space on the disk tracks (alternate sides ), and I'm sure the parameters are very close to the correct ones.
A bug involves the boot tracks count being halved when "alternate sides" is enabled, obvioulsy we have a workaround (to double the parameter), but I'd like to spot the culript.


// Sharp MZ-3500 - 320K
static disc_spec mz3500_spec = {
.name = "MZ3500",
.disk_mode = MFM250,
.sectors_per_track = 16,
.tracks = 40,
.sides = 2,
.sector_size = 256,
.gap3_length = 0x17,
.filler_byte = 0xe5,
.boottracks = 3,
.directory_entries = 128,
.extent_size = 2048,
.byte_size_extents = 1,
.first_sector_offset = 1,
.alternate_sides = 1,
.has_skew = 1,
.skew_track_start = 0,
.skew_tab = { 0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15 }
};


The MZ-3500 had other disk formats, the EOS is a potentially good candidate if I ever find a way to boot the emulator !

[stefano]

I'm halfway in decoding this MZ-3500 format.
It deploys like a single sided 32 tracks disk, with skew set to 4.
This figure is not yet supported by appmake, which obviously can still get a single sided disk configuration to produce a raw image file

[stefano]

They look like two different "bugs" preventing this configuration to succeed.
I apologise for the micro changes applied in such a short term, I'm attempting to remove some redundancy, some of which I'm responsible of.
I get a strangely big gap applied after the directory extents, apparently they are just two extents of 2048 bytes but they take a whole track on both the sides and put the files in the next free track couple, no matter if I'm enabling the skew or not.
I will probably redo the maths and insist on debugging. No matter whether this odd way to apply the skew is supported or not, I must first set the right distance between the directory and the program extents.

[stefano]

It was very simple indeed, this disk format assumes the two sides as being part of a single virtual track (swapping the active head on any disc rotation).

In the final write loop appmake takes a single track and if necessary swaps the sectors in it. This format overflows:
>>> 0, 4, 8, 12, 16, 20, 24, 28, 1, 5, 9, 13, 17, 21, 25, 29, <<<||>>> 2, 6, 10, 14, 18, 22, 26, 30, 3, 7, 11, 15, 19, 23, 27, 31 <<<

The resulting patterns showed an apparent bug, but appmake couldn't be blamed for it.

Please note that the MZ-3500 computers family had different models, I got this format from the Mz-2500 DISKDEFS.COM tool.
In the EOS CP/M for the german models I found the following capabilities for the "QUAD minifloppies":

FDOS/CPM: 256 Bytes pro Sektor, 292 KBytes (NON-EOS disk format, I think it includes both the Microtech and the Sharp CP/M disk mode)
MZ-80B : 512 Bytes pro Sektor, 336 KBytes
MZ-3500 : 1024 Bytes pro Sektor, 386 KBytes (the EOS standard format. $INIT5.COM deals with it)
QUAD : 1024 Bytes pro Sektor, 784 KBytes (elswhere I found documentation about PC-3201 when used with the DiCOS operating system, I don't know if it is equivalent to QUAD or had to do with a different EOS version).

The format I worked on creates 320K disk images, possibly it refers to "FDOS/CPM"

A second tool (for a different 80 tracks disk drive type) lists the following formats (sizes could refer to single/double sided disks):

128 Bytes FM pro Sektor, 241/482 KBytes ("A common format for data exchange")
256 Bytes MFM pro Sektor, 482/964 KBytes ("The double-sided version is compatible with the HAYAC - 2900 computer")
512 Bytes MFM pro Sektor, 556/1120 KBytes
1024 Bytes MFM pro Sektor, 592/1208 KBytes ("Compatible with the EOS format of the IMS-8000 computer family")

On VCFED I found diskette dumps basically in "368K" mode but with a missing track #0 on one of the disk sides, I don't know if the tool used to extract the disk image failed in reading it or it simply was never formatted. SAMDISK and CPMTOOLS get in trouble, but IMDU can extract a raw file which finally can be parsed with CPMTOOLS. I believe I found a correct configuration this mess:

Code: Select all

# MZ-3500 eos3: 1024 Bytes/sec, "386 KBytes" data area
# (byte sized extents)
diskdef eos3500b
  seclen 1024
  tracks 80
  heads 2
  sectrk 5
  blocksize 2048
  maxdir 64
  skew 2
  boottrk 1
  os 2.2
end