CD-i Bits

JNMS and Maxi-MMC updates

2011-11-07T01:26:00.002+01:00

This weekend I fixed some disc emulation issues for the JNMS and Maxi-MMC boards. I had previously erroneously identified these two boards but they are different.

The JNMS board is the one in the CDI 180 player (also called the JNMS player). It is not used in any other player and contains a CDIC (CD Interface Controller) chip but no SLAVE processor.

The Maxi-MMC board is the one in the CDI 601 and 602 players. From the emulation point of view it is virtually identical to the Mini-MMC board used by the CDI 605 player, but it has a different CDIC chip version. Both boards contain a SLAVE processor.

The link between the JNMS and Maxi-MMC boards is the CDIC chip: both turn out to have the same older CDIC chip version that differs in a few details from the version used on the Mini-MMC and Mono-I boards players (I described these differences in the earlier “CD-i 180 disc playing” post).

I noticed the JNMS / Maxi-MMC link from the CD-i player type table in the July 1996 issue of The Interactive Engineer (it’s on the ICDIA site); turns out I had misinterpreted the Board column on page 4 (there’s also an error there: the 601/602 certainly do not have the 180 board!).

After noticing this I did some testing and it turns out that the CDIC modifications needed for the 180 also work for the 601, including the TOC reading problem.

I have yet to find a way to get chip version information from the CDIC chip itself, so for the time being I’ve keyed the differences on the SLAVE software version. The 180 has no such chip, the 601 has version 1.x where the 605 has version 3.x. For now I’ve assumed that version 2.x also uses the older CDIC chip, but that may be wrong (the 602 or 604 might be interesting test cases).

Having done that, I did some more digging into the TOC read issue. It turns out that the 601 ROM performs CRC validation on the subcode Q data from the lead-in area (which is where the TOC is stored), and CD-i Emulator didn’t provide a valid CRC (no other ROMs I’ve seen so far validate this in software). The ROM even has compensation for shifts of between 1 and 7 bits in the incoming subcode Q data, probably because of some hardware timing issue.

I also noticed a bug in the ROM: it always uses the first sector buffer because it takes the current buffer bit from the wrong memory location. Not that this really matters because the TOC data is repeated multiple times; half of the repetitions will always land in the first buffer anyway. The bug is fixed in the 605 ROM.

Generating a valid CRC turned out to be straightforward (it’s just a simple CRC-CCITT calculation), but the ROM wouldn’t recognize it! After some head scratching I focused on the ROXL instruction (Rotate Left with Extend) used in the validation code. It is quite an esoteric instruction; could it be that there was an emulation bug here? It turns out that there was indeed; during rotation the contents of the X flag where put in the wrong bit. After fixing this the ROM properly recognized the data and the TOC reads became just as quick as other player models.

In search of version information for the CDIC chip I looked at the emulations and found one potential point of interest: the release number displayed by the service shell. This is a special GUI shell that performs some service functions; you can get to it by inserting a specially wired test plug into input port 1.

After some digging I found that the service shell obtains this number from the SLAVE processor, so it probably does not directly correspond to a CDIC version. The number does appear to differ from other version numbers, though, at least on my two 605 players.

The service shell obtains this number using two special I$SetStt calls into the CDIC driver; extending CD-i Link to remotely perform these same calls was easy. The new -cds[tatus] option can now be used to make the special calls. Here's some representative output of the -cds A3 option:

CD status A3000000 -> A3320000

Extending CD-i Link with remote OS9 calls is actually a fairly easy way to perform some information and tracing actions; I will probably use it for sorting out other dangling issues in the near future. When possible, this technique avoids the problems of writing a fullblown memory-resident trace module.

A new public beta release of CD-i Emulator that has full JNMS and Maxi-MMC support (among other things) is scheduled before the end of this year; there are still a few other issues that need sorting out first. This release should also have better support for the PCDI board used by several portable players, including the CD-i 370.

The major player holes still remaining are the Sony IVO-10/11 players, the Kyocera player, the Bang&Olufsen TV/player combi and of course the I2M board. There is some perspective for all of these but they are not high priority; except for the latter I expect all of them to be minor hardware variations of existing boards.

The I2M board has the interesting feature that it has multiple "ROMs" downloaded from the PC software (which is available for download from ICDIA); it also has a very different way of reading from CD as this is handled by the PC. As a consequence of this, audio is probably also handled differently. I have this board booting to a blue screen where it hangs on host communication.

SCSI support and a big surprise

2011-10-11T00:57:00.008+02:00

Last week I added SCSI disk support for the CD-i 60x extension board to CD-i Emulator. It took somewhat longer then I expected, though. This was mostly because the DP5380 SCSI controller chip exposes most low-level details of the SCSI protocol to the driver which means that all of these details have to be emulated.

The emulation ended up to be a more-or-less complete software implementation of the parallel SCSI-2 protocol, including most of the low-level signaling on the BSY, SEL, ATN, MSG, C/D-, I/O-, REQ and ACK lines. This is all implemented by the new CScsiBus class representing the SCSI bus that connects up to 16 instances of the CScsiPort class that each represent a single SCSI-2 bus interface. I was able to mostly avoid per-byte signaling of REQ and ACK if the target device implementation supports block transfers, a big performance win.

The new CCdiScsiDevice class emulates the DP5380 controller chip, working in conjunction with the CCdiScsiRamDevice and CCdiScsiDmaDevice classes that emulate the 32 KB of local extension SRAM and the discrete DMA logic around it that are included on the CD-i 60x extension board.

The CD-i 182 extension uses a compatible SCSI controller chip but a different DMA controller and has no local extension SRAM. I have not yet emulated these because I have almost no software to test it.

The new CScsiDevice class implements a generic SCSI device emulating minimal versions of the four SCSI commands that are mandatory for all SCSI device types: TEST UNIT READY, REQUEST SENSE, INQUIRY and SEND DIAGNOSTIC. It implements most of the boiler-plate of low-level SCSI signaling for target devices and the full command and status phases of SCSI command processing, allowing subclasses to focus on implementing the content aspects of the data transfer phase.

The CScsiFile class emulates a SCSI device backed by a file on the host PC; it includes facilities for managing the SCSI block size and the transfer of block-sized data to and from the backing file.

The CScsiDisk and CScsiTape classes emulate a SCSI disk and tape device, respectively, currently supporting a block size of 512 bytes only. Instances of these classes are connected to the SCSI bus by using the new
-s[csi]d[isk][0-7] FILE and -s[csi]t[ape][0-7] FILE options of CD-i Emulator.

The CD-i 60x extension board normally uses SCSI id 5; the built-in ROM device descriptors for SCSI disks use SCSI ids starting at zero (/h0 /h1 /h2) while the built-in device descriptor for a SCSI tape uses SCSI id 4 (/mt0). This means that the useful options with the 60x are -scsidisk0, -scsidisk1, -scsidisk2 and -scsitape 4.

I've added the new dsk subdirectory to contain disk images; tape images have no standard location as they are mostly intended for bulk-transfer purposes (see below).

Inside the CD-i player this leads to the following response to the built-in inquire command:

$ inquire -i=0
vendor identification:"CDIFAN  CDIEMU SCSIDISK "
 
$ inquire -i=4
vendor identification:"CDIFAN  CDIEMU SCSITAPE "

where the "CDIFAN " part is the vendor name and the "CDIEMU SCSIXXXX " part is the product name.

In the previous post I described a 450 MB OS-9 hard disk image that I found on the Internet. After mounting it with
-scsidisk0 mw.dsk I got the following output:

$ free /h0
"MediaWorkshop" created on: Feb 17, 1994
Capacity: 1015812 sectors (512-byte sectors, 32-sector clusters)
674144 free sectors, largest block 655552 sectors
345161728 of 520095744 bytes (329.17 of 496.00 Mb) free on media (66%)
335642624 bytes (320.09 Mb) in largest free block
 
$ dir -d /h0

Directory of /h0 23:49:36
ASU/            AUDIO/          CDI_BASECASE/   CINERGY/        CMDS/
COPY/           CURSORS/        DEFS/           DEMOS/          ENET/
ETC/            FDRAW/          FONTS/          FontExample/    ISP/
LIB/            MAUI/           MAUIDEMO/       MENU/           MWOS/
NFS/            README_CIN      README_MWS      SCRIPT/         SHARE/
SHIP/           SYS/            T2D_RUNTIME/    TEMP/           TEMPMARK/
TEST/           USR/            VIDEO/          abstract.txt    bibliographic.txt
bkgd.c8         bkgd.d          cdb             cdb1            cdb2
cdi_opt_install chris_test      cin             copyright.mws   copyright.txt
csd_605         custominits_cin delme           dos/            file
font8x8         get             globs.mod       go              go.mkfont
inetdb          ipstat          kick1a_f.c8     kick2a_f.c8     mtitle
mws             net             new_shell       new_shell.stb   scratch
screen          startup_cin     thelist

You can see why thought it was a MediaWorkshop disc, but on closer inspection this turned out to something quite different. Some basic scrutiny lead to the hypothesis that this is probably a disk backup of someone from Microware working on early development of the DAVID (Digital Audio Video Interactive Decoder) platform. There are various surprises on the disk which I will describe below.

Anyway, I wanted to transfer the contents to the PC as a tar archive, similar to the procedure I used for my CD-i floppy collection. After starting CD-i Emulator with a -scsitape4 mw.tar option this was simply a matter of typing the following into the terminal window:

tar cb 1/h0

This command runs the "tape archiver" program to create a tape with the contents of the /h0 directory, using a tape blocking size of 1 (necessary because my SCSI tape emulation doesn't yet support larger block sizes). The resulting mw.tar file on the PC is only 130 MB, not 450 MB which indicates that the disk is mostly empty. At some point I might use an OS-9 "undelete" program to find out if there are additional surprises.

Extracting the mw.tar file was now a simple matter of running the PC command

tar xvf mv.tar

This produced an exact copy of the OS-9 directory structure and files on the PC.

Many of the directories on the hard disk are clearly copies of various distribution media (e.g. CDI_BASECASE, CINERGY, CURSORS, ENET, FONTS, ISP, MWOS, NFS). The contents of the ENET, ISP and NFS directories at first appear to match some of my floppies, including version numbers, but on closer inspection the binaries are different. Running some of them produces "Illegal instruction" errors so I suspect that these are 68020 binaries.

The SHIP directory contains some prerelease RTNFM software; the readme talks about PES which is a type of MPEG-2 stream (Packetized Elementary Stream). Various asset directories contain versions of a "DAVID" logo.

The CMDS directory contains working versions of the Microware C compiler, identical to the ones I already had and also many other programs. It also contains some "cdb" files (configuration database?) that mention the 68340 processor.

The contents of the CMDS/BOOTOBJS directory produced a first surprise: it contains a subdirectory JNMS containing among others files named "rb1793" and "scsijnms". Could this be floppy and SCSI drivers for the CD-i 182 extension, as it contains with a 1793 floppy drive controller (the CD-i 60x uses a different one) and the player has a "JNMS" serial number?

Well, yes and no. Disassembly of the scsijnms file proved it to be compiled C code using an interface different from OS-9 2.4 drivers, so I suspect this is an OS-9 3.x driver. In any case, I cannot use it with the stock CD-i 180 player ROMs. Bummer...

And now for the big surprise: deeply hidden in a directory structure inside the innocently named COPY directory is the complete assembly source for the VMPEG video driver module "fmvdrv". At first glance it looked very familiar from my disassembly exercises on the identically-named Gate Array 2 MPEG driver module "fmvdrv", which is as expected because I had already noticed the large similarity between these two hardware generations.

The source calls the VMPEG hardware the "IC3" implementation, which matches CD-i digital video history as I know it. The Gate Array MPEG hardware would be "IC2" and the original prototype hardware would be "IC1". Furthermore, the sources contain three source files named fmvbugs1.a to fmvbugs3.a whose source file titles are "FMV first silicon bugs routines" to "FMV third silicon bugs routines". The supplied makefile currently uses only fmvbugs3.a as is to be expected for a VMPEG driver.

The fmvbugs1.a source contains some of the picture buffer manipulation logic that I've so far carefully avoided triggering because I couldn't understand it from my disassemblies, and this is now perfectly understandable: they are workarounds for hardware bugs!

As of two hours ago, I have verified that with a little tweaking and reconstruction of a single missing constants library file these sources produce the exact "fmvdrv" driver module contained in the vmpega.rom file directly obtained from my VMPEG cartridge.

In general these sources are very heavily commented, including numerous change management comments. They also include a full set of hardware register and bit names, although no comments directly describing the hardware. This should be of great help in finally getting the digital video emulation completely working.

All of the comments are English, although a few stray words and developer initials lead me to believe that the programmers were either Dutch or Belgian.

Disassembly comparisons lead me to the conclusion that careful undoing of numerous changes should result in exact sources for the GMPEGA2 driver module "fmvdrv" as well. I might even do it at some point, although this is not high priority for me.

The disk image containing all of these surprises is publicly available on the Internet since at least 2009, which is probably someone's mistake but one for which I'm very grateful at this point!

CD-i floppy inventory

2011-10-05T00:37:00.003+02:00

Last weekend I future-proofed my CD-i floppy collection. A bit to my surprise, all floppies except one turned out to be perfectly readable (nearly twenty years after they were last written!). Luckily, the one exception was a backup copy so I didn’t lose any contents.

I had originally intended to use the borrowed CDI 182 unit for this (it has two floppy drives). The primary motivation for this was that my unstowed CDI 605 could not read beyond track zero of any floppy, but after giving the matter some thought I decided to try my other CDI 605 first, the primary motivation for this being speed (see below). It turned out that this 605 could read the floppies perfectly, including the three 38U0 format ones that gave problems on the 182 unit. Microware has defined a number of OS-9 disk formats for floppies, the 38U0 one supposedly being the “universal” 3.5" format (there is also a 58U0 “universal” 5¼" format).

The problem with the “universal” formats is that track zero can be (and on my floppies, is) in a different density which makes it a bad fit for most tools, both on CD-i and PC. It also means that only 79 tracks are used for data storage, giving a raw capacity of 79 × 2 × 16 × 256 = 632 KB. The 3803 format used by all my other CD-i floppies uses all 80 tracks and consequently has 8 KB more of raw storage for a total of 640 KB (these are both double-density, double-side formats (DS, DD) with 16 sectors of 256 bytes per track like nearly all OS-9 disk formats).

Before unstowing my other CDI 605 (it was nearly at the bottom of a 150 cm stowed equipment stack) I tried reading the floppies with my trusty old Windows 98 machine which still has floppy drives. I could not quickly find a DOS tool that handled the 256 byte sectors (not even raread and friends), although I suspect that Sydex’s TELEDISK product would have handled it just fine. I also tried Reischke’s OS9MAX which should handle all OS-9 formats under the sun according to its documentation. The demo version ran under MS-DOS and gave me working directory listings, even for the 38U0 floppies, but it does not support actually reading the files and I am somewhat doubtful about the current availability of the paid-for full version (even apart from cost concerns).

Why did I decide to use the 605? It was not a question of reading the disks (the 182 did this mostly fine) but of handling the data thus read. The 182 unit has a SCSI connector but I have no drivers for it (yet) and dumping my full floppy collection over the serial port did not really appeal to me for speed and reliability reasons (it could have been done, of course).

The 605 player has a SCSI connector and includes drivers for it so I could have just connected it to the SCSI disk in my E1 emulator and copied the floppies to hard disk (I would still have needed to transfer them to my laptop which would have been a two-step process via the Windows 98 PC as I have no SCSI connection on my laptop).

Instead I used the BNC network connector of the 605 to directly transfer floppy images to my laptop (it needs a network switch supporting both a BNC connector and the modern RJ45 connectors, but luckily I have two of those, even if they are only 10 Mbit/s). Starting up the network environment of the 605 took only two OS-9 commands at the command shell prompt:

ispmode /le0 addr=10.0.0.120
mbinstall

After this I could just ftp in to my laptop where I ran ftpdmin, a very minimal ftp server program, and transfer floppy disk images directly:

ftp 10.0.0.110
bin
put /d0@ floppy.dsk

(where /d0@ is the raw floppy device, for 38U0 I used /d0uv@, both are built-in for the 605).

The transfers ran at the maximum speed of the floppy drive (way below the 10 Mbit/s network speed), and the resulting .dsk files are perfectly readable using the –v option (virtual disk) of Carey Bloodworth’s os9.exe program even though that program was originally written for Tandy Color Computer OS9/6809 floppies (the floppy disk format was not changed for OS-9/68000 which is at the core of CD-i’s CD-RTOS operating system).

For easy access I also created a “tar” format archive of each floppy on a RAM disk:

chd /d0
tar cvf /r768/floppy.tar .

and ftp’d those to my laptop as well (the /r768 device is a 768 KB variation of the /r512 built-in 512 KB RAM disk device of the 605 player).

I ended up with the following collection of unique floppy disk images:

605h3 - 605 H3 Driver Update (1 floppy)
605upd - 605 Driver Update (1 floppy)
bcase - Basecase Tests (1 floppy)
eboot41 - Emulation Boot Diskette (1 floppy)
eburn41 - Emulation and CDD 521 Boot Diskette (1 floppy)
inet - CD-I Internet Installation Disk - V1.3 (1 floppy)
nfs - OS-9/68000 Network File System V.1.0 (1 floppy)
os9sys - OS-9 System Diskette (1 floppy)
pubsoft - OptImage Public Domain Software (2 floppies)
pvpak - OptImage Preview Pak Installation Disk (1 floppy)
ubridge - OS-9 UniBridge Resident Utilities (3 floppies)

The 605* and eb* floppies are mostly interesting for CD-i 605 or E1 emulator owners, but the bcase floppy contains a set of CD-i standard conformance test programs that.

The inet and nfs floppies contain a full set of Internet software including Telnet and FTP servers and clients and an NFS client (all except the latter are also in the 605 ROMs).

The os9sys floppy contains a full set of Professional OS-9 programs and is my original source for most of the OS-9 CD-i disc that I described earlier (most of these are not in ROM on any CD-i player that I’ve seen so far).

The pubsoft floppies contain miscellanous utilities such as bfed, du, kermit, umacs and vi, most of which can be obtained elsewhere, some CD-i specific utilities such as da (CD-i disk analyzer) and iffinfo (CD-i IFF file dumper) as well as library source files for the CD-i IFF file library.

The pvpak floppy contains preview software for CD-i images that will preview CD-i IFF files from an NFS-mounted host file system directory.

The ubridge floppies are the goldmine (and also the 38U0 format ones) as they contain a full set of native Microware C compiler/assembler/linker/debugger software for OS-9 complete with CD-i header files and libraries and C runtime startup sources. Both the srcdbg and sysdbg debuggers are included as well as the rdump utility for dumping ROFF (Relocatable Object File Format) files.

Unfortunately, most of the above software except for the pubsoft contents is copyrighted property of Microware (now Radisys) or OptImage (a former Philips/Microware joint venture) which means that I cannot distribute it, even though they could be very useful to CD-i homebrew developers. For that the hopefully soon-to-be available GCC cross-port will have to be enough...

While investigating all of the above I also stumbled upon a 450 MB OS-9 hard disk image for MediaWorkshop. The os9.exe program recognizes it just enough to say that it does not support it so I have no real idea about its contents except the obvious.

To remedy that problem I’m in the process of adding SCSI disk support to CD-i emulator so that I can use the SCSI support in the CD-i 605 ROMs to mount the disk image and look at it. This should also allow the CD-i 180 to boot from a SCSI disk if I ever find drivers for it (a possible path to that has just appeared, we’ll see...).

CD-i 180 experimentation

2011-09-30T23:49:00.002+02:00

Early this week, CDinteractive.co.uk forum user Erroneous came by and we spent an interesting evening taking apart our CDI 18x units and figuring out serial ports.

Whereas my set consists of a CDI 180/37 and a CDI 181/37 unit, his set is the full 180/181/182 ensemble with the added bonus of supporting 220V power. I was not previously aware that such units even existed, but it turns out he has a 180/20 + 181/20 + 182/00 combination.

I’ve taken some photographs from his set, both intact and in various dismantled states, and these can be found here on the CD-i Emulator website. Nothing particularly surprising except for the small size ROM in the 182 unit, it’s only a pair of 27C512 chips which hold 32 KB each for a total of only 64 KB!

Erroneous sold me his spare CD-i 180 remote unit and serial port adapter so I now have a mostly functioning CD-i 180 set. Unfortunately, it turns out that my 180 CD drive unit has problems so I cannot play actual discs, but the set works fine using the E1 Emulator.

It turned out that his set, however, has some defect in the 181 MMC unit which prevents it from reading discs, either from the 180 CD drive or from the E1 Emulator. Using my 181 and his 180 and 182 units we managed to get a fully working set, albeit running on mixed 120V / 220V wall power!

Because at first we couldn’t get a working command prompt on the serial port of his 182 unit, he undertook to solder a spare DB9 connector to the 181/182 interconnection bus, based on the pinout of the serial adapter which attaches to that same bus (which matches the pinout I had previously figured out by tracing the circuit board). This gave output but not a working command prompt either.

It finally turned out to be a feature of the OS9 System Disk that we were using; it boots properly when you select “Floppy Application” from the “System” menu, but it’s final step starts a command prompt for the /term device and it turns out there is no such device in the 180 player. It has three (!) serial port devices but they are named /t0, /t01 and /t2 (see below) whereas the 60x players for which this disk was apparently intended do have a /term device. On the 180, avoiding the startup script by choosing “System” / “CD-RTOS” works fine however.

When we figured this out, we could get a command line prompt on either serial port, the ROMs are smart enough to select the device where a terminal is actually connected.

We confirmed that the serial I/O chip in the 182 unit is indeed an 68681 chip as I previously suspected, which supports two serial port devices of which only one has a connector on the outside of the unit. The connected device is supported with the /t0 device name, the unconnected one uses /t01. In addition to the 68070 built-in serial port this means that the 181+182 combination actually has three serial ports, but the usual hardware setup makes only one of them accessible at a time (connecting the units uses up the interconnection bus which means that the serial adapter cannot be connected at the same time).

At this point, it was getting late and Erroneous departed for home, graciously allowing me to temporarily borrow his 18x set for some more experimentation and dumping.

When the serial port allowed me to take look inside the running 180 player, it turned out that the four ROMs that I previously dumped were not in fact co-located in the address space. The “system” ROM pair lives at $180000 as expected but on Maxi-MMC it is only 256 KB; the other 256 KB ROM pair lives at $700000 (I’ve called it the “asset” ROM because it contains only a font and pictures). Leaving out the asset ROM inside CD-i Emulator gives a working player but without any background images or buttons, just the text over a black background. You can still start a CD-i application of play a CD-Audio disc, though...

Another small factoid is that the 182 ROM contains a single picture ps_child.dyuv that at first appears to be a revised version of the identically-named one in the 181 ROM, but both pictures are bitwise identical except for the module edition number and CRC. Weird...

Dumping the ROMs of Erroneous’s 181 set turned up nothing new; they are bitwise identical to the ones from my own unit (not really surprising as both units have big “1.1” stickers on the back which signifies the “final” ROM update that all Philips 18x players received shortly before the market introduction of CD-i).

Having the 182 unit ROMs I have now extended CD-i Emulator to also support the two additional serial ports, even though the second of these is not usable on the actual player! The floppy controller and the parallel and SCSI port remain for the future.

Later this week I also took apart my new CD-i 180 remote unit, which can be used over infrared but also supports a cable connection (I’ll need to make my own cable). Pictures of this are on my site here. I suspected that the interconnection would use the I²C protocol and this indeed turned out to be the case. The unit contains another 84C21 mask-programmable microprocessor labeled “REMOCON Ver. 2.0” and its I²C SDA and SCLK pins are more or less directly connected to the cable connector, which also has RESET, GND and +5V power connections. This should allow me to connect any home-brewn pointing device over I²C.

From a bit of running system and driver inspection I also found out some more details about the bus locations of the floppy and SCSI controller chips in the 182 unit. There are two surprisingly empty ROM sockets on the SCSI extension board that are probably intended for SCSI driver and application software; except for booting support the other ROMs contain none of this.

With the information learned so far I have expanded the cditypes.rul file with CD-i 180 ROM recognition and put it in the CD-i Types section of the site.

Having two working floppy drives also allowed me to review my CD-i floppy collection and most of those appear to be perfectly readable; they may yet turn out to contain something interesting.

CD-i 180 disc playing

2011-09-24T23:36:00.002+02:00

After a few hours of tweaking I’ve gotten CD-i 180 disc playing working in CD-i Emulator. Most of the time was spent on basic disassembly of the driver, which is very similar to the CD-i 605 version (but missing some features such as multi-session support and seek delay). Not very surprising as both players use the same CDIC chip, although they are supposedly different versions.

The problems were really quite trivial but there are of course complications...

The first problem turned out to be a memory map issue. On all CD-i players that I’ve encountered so far, a memory map chunk size of 128 KB works fine because memory from different devices (whether general purpose RAM or device-specific memory such as NVRAM and CDIC local memory) never lives inside the same 128 KB chunk of address space. On the CD-i 180 this is not true, as can be seen from the following fragment of maximmc.brd:

$00300000   cdic.dev     level=4
$00310000   nvr.dev

My code silently overwrote the CDIC mapping with the NVR mapping which meant that any access to CDIC local memory terminated in a bus error. Needless to say, this was not conducive to disc playback. Simply lowering the memory map chunk size to 64 KB fixed this problem.

Next it turned out that during initialization the cdap18x driver wants to do some kind of data transfer to address $320001, possibly DSP parameters or something, and there was no device there yet. This was easily fixed also.

At this point the disc play got started but the interrupt routine ignored all interrupts. After inspecting the driver it turned out that the CD-i 180 expects an additional status bit to be set in the DBUF register that was never used on other CD-i players with a CDIC. Still easy to fix.

Now the interrupt routine started trying to process the sector but it got bus errors in two places. It attempted to read the FILE selection register, which other CD-i players never did, and it took the buffer index bits (these specify the DMA buffer the hardware used from the sector) from a previously unimplemented register that I’ve dubbed DSEL for now. Easy fixes galore...

After this one sector got properly transferred to memory but an unending stream of interrupts hung up the player. It turns out that the CD-i 180 ROMs don’t read the CDIC XBUF register that CD-i Emulator used to clear the interrupt. Generating and clearing the interrupt from the DBUF register fixed that also, and now the disc actually started playing!

Of course I did some tests next and there are currently three known problems:

- Reads of the disc table-of-contents (TOC) take ages, probably because the ROMs are checking for or waiting on some flag or data byte that CD-i Emulator currently doesn’t provide. This should not be very hard to fix.

- Audio playback from memory doesn’t work, the symptoms are very similar to those of the audio sync issues that occurred during the MESS collaboration in 2009. This may be a hard problem to crack.

- And finally the most hairy issue: the CDIC emulation modifications break other players. The breakage is often subtle but very definitely there so I will have to do some careful juggling to get it working on all players, if necessary by explicit version checking but I hope to avoid that. The hardware should not be that dissimilar, really...

On the serial port and pointing device front, I’ve found someone who has a surplus serial port interface and an infrared remote. He also has a complete CD-i 180 set (including the CDI 182 unit) and I’m looking forward to extracting the ROMs from that one also.

He also took apart his mouse and it seems to be a fairly standard Amiga design which means that it should not be very hard to manufacture a replacement.

I’ve been looking into making a small PIC-based interface for PS2 and/or CD-i peripherals and it may still come to that but it seems to be not as necessary as I thought.

CD-i 180 internals

2011-09-24T02:02:00.002+02:00

In the previous post I promised some ROM and chip finds. Well, here goes. To understand some of the details, you'll need some microprocessor and/or digital electronics knowledge, but even without that the gist of the text should be understandable.

The CDI 181 MMC unit contains the so-called Maxi-MMC board that is not used in any other CD-i player. Its closest cousin is the Mini-MMC board that is used in the CD-i 605 and CD-i 220 F1 players (a derivative of it is used in the CD-i 350/360 players).

The Mini-MMC board uses two 68HC05 slave processors for CD and pointing device control (they are usually called SERVO and SLAVE). The Maxi-MMC board does not have these chips, but it does have two PCF80C21 slave processors labeled RSX and TRANSDUCER that perform similar functions.

From their locations on the board I surmise that the RSX performs CD control functions; I know for sure that the TRANSDUCER performs only pointing device control. The latter is connected to the main 68070 processor via an I²C bus (I've actually traced the connections); I'm not completely sure yet about the RSX.

In order to emulate the pointing devices in CD-i Emulator, I had to reverse engineer the I²C protocol spoken by the TRANSDUCER chip; this was mostly a question of disassembling the "ceniic" and "periic" drivers in the ROM. The first of these is the low-level driver that serves as the common control point for the I²C bus; the second is the high-level driver that is instantiated separately for each type of pointing device. The ROMs support three such devices: /cdikeys, /ptr and /ptr2, corresponding to the player control keys and first and second pointing devices (the first pointing device is probably shared between the infrared remote sensor and the left pointing device port). Both pointing devices support absolute (e.g. touchpad) as well as relative (e.g. mouse) positioning.

Note that there is no built-in support for a CD-i keyboard or modem (you could use a serial port for this purpose).

However, knowing the I²C protocol does not tell me the exact protocol of the pointing devices, which therefore brings me no closer to constructing a "pointing device" that works with the two front panel MiniDIN-9 connectors. Note that these connectors are physically different from the MiniDIN 8 pointing device connectors used on most other CD-i players. According to the Philips flyers, these connectors have 6 (presumably digital) input signals and a "strobe" (STB) output signal. From the signal names I can make some educated guesses about the probable functions of the signals, but some quick tests with the BTN1 and BTN2 inputs did not pan out and it could be too complicated to figure out without measurement of a connected and working pointing device.

There is, however, also an infrared remote sensor that is supposed to expect the RC5 infrared signal protocol. This protocol supports only 2048 separate functions (32 groups of 64 each) so it should not be impossible to figure out, given a suitably programmable RC5 remote control or in the best case a PC RC5 adapter. I've been thinking about building one of the latter.

There is also a third possibility of getting a working pointing device. Although the case label of the front MiniDIN 8 connecter is "CONTROL", the Philips flyers label it "IIC" which is another way of writing "I²C", although they don't give a pinout of the port. It seems plausible that the connector is connected to the I²C bus of the 68070, although I haven't been able to verify that yet (the multimeter finds no direct connections except GND, so some buffering must be involved). If there is indeed a connection, I would be able to externally connect to that bus and send and receive the I²C bus commands that I've already reverse engineered.

If even this doesn't work, I can always connect directly to the internal I²C bus, but that involves running three wires from inside the player to outside and I'm not very keen on that (yet, anyway).

Now, about the (so far) missing serial port. There is a driver for the 68070 on-chip UART in the ROMs (the u68070 driver which is accessible via the /t2 device), and the boot code actually writes a boot message to it (CD-i Emulator output):

  PHILIPS CD-I 181 - ROM version 23rd January, 1992.
Using CD_RTOS kernel edition $53  revison $00

At first I thought that the UART would be connected to the "CONTROL" port on the front, but that does not appear to be the case. Tonight I verified (by tracing PCB connections with my multimeter) that the 68070 serial pins are connected to the PCB connector on the right side (they go through a pair of SN75188/SN75189 chips and some protection resistors; these chips are well-known RS232 line drivers/receivers). I even know the actual PCB pins, so if I can find a suitable 100-pins 0.01" spaced double edge print connector I can actually wire up the serial port.

Now for the bad news, however: the ROMs do not contain a serial port download routine. They contain a host of other goodies (more below) but not this particular beast. There is also no pointing device support for this port, contrary to all other players, so connecting up the serial port would not immediately gain me anything, I still need a working pointing device to actually start a CD-i disc…

There are no drivers for other serial ports in the ROMs, but the boot code does contain some support for a UART chip at address $340001 (probably a 68681 DUART included in the CDI 182 unit which I don't have). The support, however, is limited to the output of boot messages although the ROMs will actually prefer this port over the 68070 on-chip device if they find it.

As is to be expected from a development and test player, there is an elaborate set of boot options, but they can only be used if the ROMs contain the signature "IMS-TC" at byte offset $400 (the ROMs in my player contains FF bytes at these locations). And even then the options prompt will not appear unless you press the space bar on your serial terminal during player reset.

However, adding a -bootprompt option that handles both the signature and the space bar press to CD-i Emulator was not hard, and if you use that option with the 180 ROMs the following appears when resetting the player:

  PHILIPS CD-I 181 - ROM version 23rd January, 1992.

A-Z = change option : <BSP> = clear options : <RETURN> = Boot Now

Boot options:- BQRS

As specified, you can change the options by typing letters and pressing Enter will start the boot process with the specified options.

From disassembling the boot code of the ROMs I've so far found the following options:

D = Download/Debug
F = Boot from Floppy
L = Apply options and present another options prompt (Loop)
M = Set NTSC Monitor mode
P = Set PAL mode
S = Set NTSC/PAL mode from switch
T = Set NTSC mode
W = Boot from SCSI disk (Winchester)

It could be that there's also a C option, and I've as yet not found any implementations of the Q and R options that the ROMs include in the default, but they could be hidden in OS-9 drivers instead of the boot code.

Once set, the options are saved in NVRAM at address $313FE0 as default for prompts during subsequent reboots, they are not used for reboots where the option prompt is not invoked.

Options D, F and W look interesting, but further investigation leads to the conclusion that they are mostly useless without additional hardware.

Pressing lower-case D followed by Enter / Enter results in the following:

Boot options:- BQRSd
Boot options:- BDQRS
Enter size of download area in hex - just RETURN for none
called debugger

Rel: 00000000
Dn: 00000000 0000E430  0007000A 00000000   00000000 00000001  FFFFE000 00000000
An: 00180B84 00180570  00313FE0 00410000   00002500 00000500  00001500 000014B0
SR: 2704  (--S--7-----Z--)                     SSP: 000014B0      USP: 00000000
PC: 00180D2E   - 08020016            btst #$0016,d2
debug:

One might think that entering a download size would perform some kind of download (hopefully via the serial port) but that is not the case. The "download" code just looks at location $2500 in RAM that's apparently supposed to be already filled (presumably via an In-Circuit Emulator or something like it).

However, invoking the debugger is interesting in itself. It looks like the Microware low-level RomBug debugger that is described in the Microware documentation, although I haven't found it in any other CD-i players. One could "download" data with the change command:

debug: c0
00000000  00 : 1
00000001  00 : 2
00000002  15 : 3
00000003  00 :

Not very userfriendly but it could be done. The immediate catch is that it doesn't work with unmodified ROMs because of the "IMS-TC" signature check!

Trying the F option results in the following:

Boot options:- BQRSf
Boot options:- BFQRS
Booting from Floppy (WD 179x controller) - Please wait

This, however, needs the hardware in the CDI 182 set (it lives at $330001). I could emulate that in CD-i Emulator of course, but there's no real point at this time. It is interesting to note that the floppy controller in the CD-i 605 (which I haven't emulated either at this point) is a DP8473 which is register compatible with the uPD765A used in the original IBM PC but requires a totally different driver (it also lives at a different memory address, namely $282001).

Finally, trying the W options gives this:

Boot options:- BQRSw
Boot options:- BQRSW
Booting from RODIME RO 650 disk drive (NCR 5380 SCSI) - Please wait
Exception Error, vector offset $0008 addr $00181908
Fatal System Error; rebooting system

The hardware is apparently supposed to live at $410000 and presumably emulatable; it's identical or at least similar to the DP5380 chip that is found on the CD-i 605 extension board where it lives at $AA0000).

Some other things that I've found out:

The CDI 181 unit has 8 KB of NVRAM but it does not use the M48T08 chip that's in all other Philips players, it's just a piece of RAM that lives at $310000 (even addresses only) and is supported by the "nvdrv" driver via the /nvr device.

In the CD-i 180 player the timekeeping functions are instead performed by a RICOH RP5C15 chip, the driver is appropriately called "rp5c15".

And there is a separate changeable battery inside the case; no "dead NVRAM" problems with this player! I don't know when the battery in my player was last changed but at the moment it's still functioning and had not lost the date/time when I first powered it on just over a week ago.

The IC CARD slot at the front of the player is handled like just another piece of NVRAM; it uses the same "nvdrv" driver but a different device: /icard. According to the device descriptor it can hold 32 KB of data, I would love to have one of those!

CD-i 180 adventures

2011-09-19T04:18:00.005+02:00

Over the last week I have been playing with the CD-i 180 player set. There’s lots to tell about, so this will be a series of blog posts, this being the first installment.

The CD-i 180 is the original CD-i player, manufactured jointly by Philips and Sony/Matsushita, and for a score of years it was the development and “reference” player. The newer CD-i 605 player provided a more modern development option but it did not become the “reference” player for quite some years after its introduction.

The CD-i 180 set is quite bulky, as could be expected for first-generation hardware. I have added a picture of my set to the Hardware section of the CD-i Emulator website; more fotos can be found here on the DutchAudioClassics.nl website (it’s the same player, as evidenced by the serial numbers).

The full set consists of the CDI 180 CD-i Player module, the CDI 181 Multimedia Controller or MMC module and the CDI 182 Expansion module. The modules are normally stacked on top of each other and have mechanical interlocks so they can be moved as a unit. Unfortunately, I do not have the CDI 182 Expansion module nor any user manuals; Philips brochures for the set can be found here on the ICDIA website.

Why am I interested in this dinosaur? It’s the first mass-produced CD-i player (granted, for relatively small masses), although there were presumably some earlier prototype players. As such, it contains the “original” hardware of the CD-i platform, which is interesting from both a historical and an emulation point of view.

For emulation purposes I have been trying to get hold of CD-i 180 ROMs for some years, there are several people that still have fully operational sets, but it hasn’t panned out yet. So when I saw a basic set for sale on the CD-Interactive forum I couldn’t resist the temptation. After some discussion and a little bartering with the seller I finally ordered the set about 10 days ago. Unfortunately, this set does not include a CDI 182 module or pointing device.

I had some reservations about this being a fully working set, but I figured that at least the ROM chips would probably be okay, if nothing else that would allow me to add support for this player type to CD-i Emulator.

In old hardware the mechanical parts are usually the first to fail, this being the CDI 180 CD-i Player module (which is really just a CD drive with a 44.1 kHz digital output “DO” signal). A workaround for this would be using an E1 or E2 Emulator unit; these are basically CD drive simulators that on one side read a CD-i disc image from a connected SCSI hard disk and on the other side output the 44.1 kHz digital output “DO” signal. Both the CDI 180 and E1/E2 units are controlled via a 1200 baud RS232 serial input “RS” signal.

From my CD-i developer days I have two sets of both Emulator types so I started taking these out of storage. For practical reasons I decided to use an E1 unit because it has an internal SCSI hard disk and I did not have a spare one lying around. I also dug out an old Windows 98 PC, required because the Philips/OptImage emulation software doesn’t work under Windows XP and newer, and one of my 605 players (I also have two of those). Connecting everything took me a while but I had carefully stored all the required cables as well and after installing the software I had a working configuration after an hour or so. The entire configuration made quite a bit of mechanical and fan noise; I had forgotten this about older hardware!

I had selected the 605 unit with the Gate Array AH02 board because I was having emulation problems with that board, and I proceeded to do some MPEG tests on it. It turns out the hardware allows for some things that my emulator currently does not, which means that I need to do some rethinking. Anyway, on with the 180 story.

In preparation for the arrival of the 180 set I next prepared an disc image of the “OS-9 Disc” that I created in November 1993 while working as a CD-i developer. This disc contains all the OS-9 command-line programs from Professional OS-9, some OS-9 and CD-i utilities supplied by Philips and Microware and some homegrown ones as well. With this disc you can get a fully functional command-line prompt on any CD-i player with a serial port, which is very useful while researching a CD-i player’s internals.

The Philips/Optimage emulation software requires the disc image files to include the 2-second gap before logical block zero of the CD-i track, which is not usually included in the .bin or .iso files produced by CD image tools. So I modified the CD-i File program to convert my existing os9disc.bin file by prepending the 2-second gap, in the process also adding support for scrambling and unscrambling the sector data.

Scrambling is the process of XORing all data bytes in a CD-ROM or CD-i sector with a “scramble pattern” that is designed to avoid many contiguous identical data bytes which can supposedly confuse the tracking mechanism of CD drives (or so I’ve heard). It turned out that scrambling of the image data was not required but it did allow me to verify that the CD-I File converted image of a test disc is in fact identical to the one that the Philips/Optimage mastering tools produce, except for the ECC/EDC bytes of the gap sectors which CD-I File doesn’t know how to generate (yet). Fortunately this turned out not to be a problem, I could emulate the converted image just fine.

Last Thursday the 180 set arrived and in the evening I eagerly unpacked it. Everything appeared to be in tip-top shape, although the set had evidently seen use.

First disappointment: there is no serial port on the right side of 181 module. I remembered that this was actually an option on the module and I had not even bothered to ask the seller about it! This would make ROM extraction harder, but I was not completely without hope: the front has a Mini-DIN 8 connector marked “CONTROL” and I fully expected this to be a “standard” CD-i serial port because I seemed to remember that you could connect standard CD-i pointing devices to this port, especially a mouse. The built-in UART functions of the 68070 processor chip would have to be connected up somewhere, after all.

Second disappointment: the modules require 120V power, not the 220V we have here in Holland. I did not have a voltage converter handy so after some phone discussion with a hardware-knowledgeable friend we determined that powering up was not yet a safe option. He gave me some possible options depending on the internal configuration so I proceeded to open up the CDI 181 module, of course also motivated by curiosity.

The first thing I noticed was that there were some screws missing; obviously the module had been opened before and the person doing it had been somewhat careless. The internals also seemed somewhat familiar, especially the looks of the stickers on the ROM chips and the placement of some small yellow stickers on various other chips.

Proceeding to the primary reason for opening up the module, I next checked the power supply configuration. Alas, nothing reconfigurable for 220V, it is a fully discrete unit with the transformer actually soldered to circuit board on both input and output side. There are also surprisingly many connections to the actual MMC processor board and on close inspection weird voltages like –9V and +9V are printed near the power supply outputs, apart from the expected +5V and +/–12V, so connecting a different power supply would be a major undertaking also.

After some pondering of the internals I closed up the module again and proceeded to closely inspect the back side for serial numbers, notices, etc. They seemed somewhat familiar but that isn’t weird as numbers often do. Out of pure curiosity I surfed to the DutchAudioClassics.nl website to compare serial numbers, wanting to know the place of my set in the production runs.

Surprise: the serial numbers are identical! It appears that this exact set was previously owned by the owner of that website or perhaps he got the photographs from someone else. This also explained why the internals had seemed familiar: I had actually seen them before!

I verified with the seller of the set that he doesn’t know anything about the photographs; apparently my set has had at least four owners, assuming that the website owner wasn’t the original one.

On Friday I obtained a 120V converter (they were unexpectedly cheap) and that evening I proceeded to power up the 180 set. I got a nice main menu picture immediately so I proceeded to attempt to start a CD-i disc. It did not start automatically when I inserted it, which on second thought makes perfect sense because the 181 MMC module has no way to know that you’ve just inserted a disc: this information is not communicated over 180/181 interconnections. So I would need to click on the “CD-I” button to start a disc.

To click on a screen button you need a supported pointing device, so I proceeded to connect the trusty white professional CD-i mouse that belongs with my 605 players. It doesn’t work!

There are some mechanical issues which make it doubtful that the MiniDIN connector plugs connect properly, so I tried an expansion cable that fit better. Still no dice.

The next step was trying some other CD-i pointing devices, but none of them worked. No pointing devices came with the set, and the seller had advised me thus (they were presumable lost or sold separately by some previous owner). The only remaining option seemed to be the wireless remote control sensor which supposedly uses RC5.

I tried every remote in my home, including the CD-i ones, but none of them give any reaction. After some research into the RC5 protocol this is not surprising, the 180 set probably has a distinct system address code. Not having a programmable remote handy nor a PC capable of generating infrared signals (none of my PCs have IrDA) I am again stuck!

I spent some time surfing the Internet looking for RC5 remotes and PC interfaces that can generate RC5 signals. Programmable remotes requiring a learning stage are obviously not an option so it will have to be a fully PC-programmable remote which are somewhat expensive and I’m not convinced they would work. The PC interface seems the best option for now; I found some do-it-yourself circuits and kits but it is all quite involved. I’ve also given some thought to PIC kits which could in principle also support a standard CD-i or PC mouse or even a joystick, but I haven’t pursued these options much further yet.

Next I went looking for ways to at least get the contents of the ROM chips as I had determined that these were socketed inside the MMC module and could easily be removed. There are four 27C100 chips inside the module, each of which contains 128Kb of data for a total of 512Kb which is the same as for the CD-i 605 player (ignoring expansion and full-motion video ROMs). The regular way to do this involves using a ROM reading device, but I haven’t gotten one handy that supports this chip type and neither does the hardware friend I mentioned earlier.

I do have access to an old 8 bit Z80 hobbyist-built system capable of reading and writing up to 27512 chips which are 64Kb, it is possible to extend this to at least read the 27C100 chip type. This would require adapting the socket (the 27512 is 28 pins whereas the 27C100 has 32 pins) and adding one extra address bit, if nothing else with just a spare wire. But the Z80 system is not at my house and some hardware modifications to it would be required, for which I would have to inspect the system first and dig up the circuit diagrams; all quite disappointing.

While researching the chip pinouts I suddenly had an idea: what if I used the CD-i 605 Expansion board which also has ROM sockets? This seemed an option but with two kids running around I did not want to open up the set. That evening however I took the board out of the 605 (this is easily done as both player and board were designed for it) and found that this Expansion board contains two 27C020 chips, each containing 256Kb of data. These are also 32 pins but the pinouts are a little different, so a socket adapter would also be needed. I checked the 605 technical manual and it did not mention anything about configurable ROM chip types (it did mention configurable RAM chip types, though) so an adapter seemed the way to go. I collected some spare 40 pin sockets from storage (boy have I got much of that) and proceeded to open up the 180 set and take out the ROM chips.

When determining the mechanical fit of the two sockets for the adapter I noticed three jumpers adjacent to the ROM sockets of the expansion board and I wondered… Tracing of the board connections indicated that these jumpers were indeed connected to exactly the ROM socket pins differing between 27C100 and 27C020, and other connections indicated it at least plausible for these jumpers to be exactly made for the purpose.

So I changed the jumpers and inserted one 180 ROM. This would avoid OS-9 inadvertently using data from the ROM because only half of each 16-bit word would be present, thus ensuring that no module headers would be detected, and in the event of disaster I would lose only a single ROM chip (not that I expected that to be very likely, but you never know).

Powering up the player worked exactly as expected, no suspicious smoke or heat generation, so the next step was software. It turns out that CD-i Link already supports downloading of ROM data from specific memory addresses and I had already determined those addresses from the 605 technical manual. So I connected the CD-i 605 null-modem cable with my USB-to-Serial adapter between CD-i player and my laptop and fired off the command line:


 cdilink –p 3 –a 50000 –s 256K –u u21.rom

(U21 being the socket number of the specific ROM I chose first).

After a minute I aborted the upload and checked the result, and lo and behold the u21.rom file looked like an even-byte-only ROM dump:

00000000  4a00 000b 0000 0000 0004 8000 0000 0000 J...............
00000010  0000 0000 0000 003a 0000 705f 6d6c 2e6f .......:..p_ml.o
00000020  7406 0c20 0000 0000 0101 0101 0101 0101 t.. ............

This was hopeful, so I restarted the upload again and waited some six minutes for it to complete. Just for sure I redid the upload from address 58000 and got an identical file, thus ruling out any flakey bits or timing problems (I had already checked that the access times on the 27C100 and 27C020 chips were identical, to say 150ns).

In an attempt to speed up the procedure, I next attempted to try two ROMs at once, using ones that I thought not to be a matched even/odd set. The 605 would not boot! It later turned out that the socket numbering did not correspond to the even/odd pairing as I expected so this was probably caused by the two ROMs being exactly a matched set and OS-9 getting confused as the result. But using a single ROM it worked fine.

I proceeded to repeat the following procedure for the next three ROMs: turn off the 605, remove the expansion board, unsocket the previous ROM chip, socket the next ROM chip, reinsert the expansion board, turn on the 605 and run CD-i Link twice. It took a while, all in all just under an hour.

While these uploads were running I wrote two small programs rsplit and rjoin to manipulate the ROM files into a correct 512Kb 180 ROM image. Around 00:30 I had a final cdi180b.rom file that looked good and I ran it through cditype –mod to verify that it indeed looked like a CD-I player ROM:

  Addr     Size      Owner    Perm Type Revs  Ed #  Crc   Module name
-------- -------- ----------- ---- ---- ---- ----- ------ ------------
0000509a      192     0.0     0003 Data 8001     1 fba055 copyright
0000515a    26650     0.0     0555 Sys  a000    83 090798 kernel
0000b974      344     0.0     0555 Sys  8002    22 b20da9 init
0000bacc     2848     0.0     0555 Fman a00b    35 28611f ucm
0000c5ec     5592     0.0     0555 Fman a000    17 63023d nrf
0000dbc4     2270     0.0     0555 Fman a000    35 d6a976 pipeman
0000e4a2      774     0.0     0555 Driv a001     6 81a3e9 nvdrv
0000e7a8      356     0.0     0555 Sys  a01e    15 e69105 rp5c15
0000e90c      136     0.0     0555 Desc 8000     1 f25f23 tim070
0000e994      420     0.0     0555 Driv a00c     6 7b3913 tim070driv
0000eb38      172     0.0     0555 Driv a000     1 407f81 null
0000ebe4      102     0.0     0555 Desc 8000     2 cf450e pipe
0000ec4a       94     0.0     0555 Desc 8000     1 f54010 nvr
0000eca8       96     0.0     0555 Desc 8000     1 17ec68 icard
0000ed08     1934     0.0     0555 Fman a000    31 b41f17 scf
0000f496      120     0.0     0555 Desc 8000    61 dd8776 t2
0000f50e     1578     0.0     0555 Driv a020    16 d0a854 u68070
0000fb38      176     0.1     0777 5    8001     1 a519f6 csd_mmc
0000fbe8     5026     0.0     0555 Sys  a000   292 e33cc5 csdinit
00010f8a      136     0.0     0555 Desc 8000     6 041e2b iic
00011012      152     0.0     0555 Driv a02c    22 e29688 ceniic
000110aa      166     0.0     0555 Desc 8000     8 c5b823 ptr
00011150      196     0.0     0555 Desc 8000     8 a0e276 cdikeys
00011214      168     0.0     0555 Desc 8000     8 439a33 ptr2
000112bc     3134     0.0     0555 Driv a016    11 faf88d periic
00011efa     4510     0.0     0555 Fman a003    96 a4d145 cdfm
00013098    15222     0.0     0555 Driv a038    28 122c79 cdap18x
00016c0e      134     0.0     0555 Desc 8000     2 35f12f cd
00016c94      134     0.0     0555 Desc 8000     2 d2ce2f ap
00016d1a      130     0.0     0555 Desc 8000     1 1586c2 vid
00016d9c    18082    10.48    0555 Trap c00a     6 5f673d cio
0001b43e     7798     1.0     0555 Trap c001    13 46c5dc math
0001d2b4     2992     0.0     0555 Data 8020     1 191a59 FONT8X8
0001de64      134     0.0     0555 Desc 8000     2 c5ed0e dd
0001deea    66564     0.0     0555 Driv a012    48 660a91 video
0002e2ee    62622     0.1     0555 Prog 8008    20 ec5459 ps
0003d78c     4272     0.0     0003 Data 8001     1 9f3982 ps_medium.font
0003e83c      800     0.0     0003 Data 8002     1 c1ac25 ps_icons.clut
00040000     2976     0.0     0003 Data 8002     1 0a3b97 ps_small.font
00040ba0     7456     0.0     0003 Data 8002     1 764338 ps_icons.clu8
000428c0   107600     0.0     0003 Data 8002     1 7b9b4e ps_panel.dyuv
0005cd10    35360     0.0     0003 Data 8001     1 2a8fcd ps_girl.dyuv
00065730    35360     0.0     0003 Data 8002     1 e1bb6a ps_mesa.dyuv
0006e150    35360     0.0     0003 Data 8002     1 8e394b ps_map.dyuv
00076b70    35360     0.0     0003 Data 8002     1 c60e5e ps_kids.dyuv
 
    File      Size    Type       Description
------------ ------ ------------ ------------
cdi180b.rom    512K cdi000x.rom  Unknown CD-i system ROM
cdi180b.rom    512K cdi000x.mdl  Unknown CD-i player
cdi180b.rom    512K unknown.brd  Unknown board

Of course cditype didn’t correctly detect the ROM, player and board type, but the list of modules looks exactly like a CD-i player system ROM. It is in fact very similar to the CD-i 605 system ROM, the major differences are the presence of the icard and *iic drivers, the absence of a slave module and the different player shell (ps module with separate ps_* data modules instead of a single play module).

It being quite late already, I resocketed all the ROMs in the proper places and closed up both players, after testing that they were both fully functional (insofar as I could test the 180 set), fully intending to clean up and go to bed. As an afterthought, I took a picture of the running 180 set and posted it on the CD-Interactive forums as the definitive answer to the 50/60 Hz power question I’d asked there earlier.

The CD-i Emulator urge started itching however, so I decided to give emulation of my new ROM file a quick go, fully intending to stop at any major problems. I didn’t encounter any of those, however, until I had a running CD-i 180 player three hours later. I reported the fact on the CDinteractive forum, noting that there was no pointing device or disc access yet, and went to a well-deserved sleep. Both of these issues are major ones and those I postponed for the next day.

To get the new player type up and running inside CD-i Emulater, I started by using the CD-i 605 F1 system specification files cdi605a.mdl and minimmc.brd as templates to create the new CD-i 180 F2 system files cdi180b.mdl and maximmc.brd. Next I fired up the emulator and was rewarded with bus errors. Not unexpected and a good indicator of where the problems are. Using the debugger and disassembler I quickly determined that the problems were, as expected, the presence of the VSR instead of VSD and the replacement of the SLAVE by something else. Straightening these out took a bit of time but it was not hard work and very similar to work I had done before on other player types.

This time at least the processor and most of the hardware was known and already emulated; for the Portable CD-i board (used by the CD-i 370, DVE200 and GDI700 players) both of these were not the case as they use the 68341 so-called integrated CD-i engine which in my opinion is sorely misnamed as there is nothing CD-i about the chip, it is just the Motorola version of an 68K processor with many on-chip peripherals in remarkably similar to the Philips 68070 in basic functionality.

Saturday was spent doing household chores with ROM research in between, looking for the way to get the pointing device working. It turned out to be quite involved but at the end of the day I had it sort of flakily working in a kludgy way; I’ll report the details in a next blog post.

Sunday I spent some time fixing the flakiness and thinking a lot about fixing the kludginess; this remains to be done. I also spent time making screenshots and writing this blog post.

So to finish up, there is now a series of 180 screenshots here on the CD-i Emulator website as reported in the What's New section. A very nice player shell, actually, especially for a first generation machine.

I will report some ROM and chip finds including new hopes for replacing the missing pointing device in a next blog post.

MPEG decoding, state save/restore, NRF emulation, ...

2011-09-04T01:28:00.006+02:00

It's been a while since I wrote anything here, but that doesn't mean that work on CD-i Emulator has stopped. On the contrary, a lot has happened in the last month and describing all of it will take a very long blog post. So here goes…

Last January an annoying date-checking bug was found which forced me to release beta2 somewhat earlier than anticipated. After that I did no further work on CD-i Emulator. There were various reasons for this, but the most import one was a very busy period at my day job.

After a well-earned vacation I resumed CD-i related work in early August. First I spent a few days on Walter Hunt's OS-9 port of gcc, the GNU C/C++ Compiler that I found in October of last year. Getting it working on a modern Cygwin installation was interesting and something very different from my usual line of work. The result could be useful for homebrew activities: it's a much more usable C compiler then the Microware OS-9 one and supports C++ as a bonus. I intend to use this for ROM-less emulation validation some day; see also below. The sources need to be released but I haven’t gotten to that stage yet.

After that I had another go at the Digital Video cartridge emulation. At the point where I left off last year the major stumbling block was the presumed picture / frame buffering logic of the MPEG video driver. When the appropriate interrupt status bits are set the driver starts copying a bulk of status information to an array of device registers and it will sometimes also read from those registers. This is all controlled by several status and timing registers that are also referenced elsewhere and I previously could not get a handle on it.

My first attempt this time was spending another few days staring at it and tracing it, but this did not gain me much new understanding. Finally I decided to just leave it for now and see how far I could get without understanding this part of the driver. I decided to once again attempt to get "CD-i Full Motion Video Technical Aspects" working.

This CD-i was produced by Philips to give future Full Motion Video (as the new MPEG playback functions were called at the time) developers a demonstration of the technical capabilities of the new hardware, at a time when this hardware was still in the early beta phase. The CD-i actually contains the compiler libraries necessary for making FMV calls from CD-i applications, as these had not previously been widely distributed.

It is not a very slick disc visually, being intended for developers, but it demonstrates a number of FMV techniques such as regular playback, playback control including pause, slow motion and single step, freeze frame and forward/backward scan, special effects like scrolling the FMV window, a seamless jump and a sample of overlay effects with the CD-i base case video planes.

I had previously tried to run this disc on CD-i Emulator, but it always crashed for an unknown reason that I attributed to MPEG device emulation problems. This time I traced back the crash and it turned out to have nothing at all to do with FMV playback but was instead caused by an incorrect emulation of the 68000 instruction "move ea,ccr" which is supposed to set the condition code register (ccr) to the value specified by the effective address (ea). In the processor manual this is classified as a word instruction and I had emulated it as such, which turned out to be wrong as it caused a word write to the full status register which should have been a byte write to the lower eight bits of it which hold the condition codes.

The problem manifested itself when the application calls the math trap handler for some mundane number calculations, which were naturally supposed to set the condition codes. The value written to the status register inadvertently changed the processor from user to system mode (and also scrambled the active interrupt masking level) which caused an instant stack switch that caused a bus error when the trap handler attempts to return to the application program (the cpu took the return address from the wrong stack and got garbage instead).

Most CD-i applications probably don't use the math trap handler so the problem went undetected for a long time. Now that it's fixed some other titles have probably started working but I haven't tested that.

After this, the FMV Technical Aspects application would get to its main menu screen, allowing me to start FMV playback operations. Regular playback worked fine until the end of the video clip, where there turned out to be status bit generation issues that prevented the application from properly detecting the end of video clip condition (the decoder is supposed to send a "buffer underflow" signal, among others, after the end of the MPEG data and my emulation didn't do that yet).

This was not very easy to fix because of the way that MPEG data buffering and decoding is handled inside CD-i Emulator, which I'll get into below. So it took me some time.

Regular play working fine, I started worrying about window control. This was the area where I feared the picture buffering stuff, but it turned out that this was easily bypassed. The horizontal / vertical scrolling functions were ideal to test this but it took me some time to get it working. There were bugs in several areas, including my integration of the MPEG video decoding code, which I took from the well-known mpeg2dec package. This code is written to decode a single video sequence and consequently did not handle image size changes without some re-initialization calls at the appropriate times. Failing that, it mostly just crashed (at the Windows application level) due to out-of-bounds video buffer accesses.

Another issue was the timing of device register updates for image size changes; I turned out to have the basic mechanism wrong and consequently the driver would keep modifying the window parameters to incorrect values.

Having all of the above fixed, I returned my attention to playback control. So far I can get the video playback properly paused, but I haven't been able to get it properly resumed. For some reason the application resumes the MPEG playback but it doesn't resume the disc playback. Since the driver waits for new data to arrive from disc before actually resuming MPEG playback nothing happens (this is documented as such). The application is presumably expecting some signal from the driver to get into the proper state for resuming disc playback, but I haven't found it yet.

At this point, it seemed promising to look at other CD-i titles using playback control and the Philips Video CD application is an obvious candidate. Again, regular playback appears to work fine, but playback control (including pause/resume) does not. It turns out that this application uses a different driver call (it uses MV_ChSpeed instead of MV_Pause, probably in preparation for possible slow motion or single step), which never completes successfully, probably again because of device status signaling. Similar issues appear to block playback control in a few other titles I tried.

I've given some thought to tracing driver calls and signals on an actual player to see what CD-i Emulator is doing wrong, and it appears to be relatively simple, there's just a bandwidth issue because all of the trace output will have to go out the serial port which can go no higher then 19200 baud. Some kind of data compression is obviously needed and I've determined a relatively simple scheme that should be enough (the CD-i player side will all need to be coded in 68000 machine language so simplicity is important!), but I haven't actually written any code for it yet.

I know there are issues with the proper timing of some video status signals. Things like start-of-sequence, end-of-sequence and start-of-picture-group should be delayed until display of the corresponding picture, at present they are delivered at decoding time, which can be a few pictures early. But that does not really affect the titles I've tried so far, because they do not attempt picture-synced operations. An application like The Lost Ride might be sensitive to thinks like this, though, and it needs to be fixed at some time. Similar issues are probably present with time code delivery. In addition, the last-picture-displayed and buffer-underflow signals are not always properly sent; I'm fixing these as I go along.

In the process, I decided that the magenta border was getting annoying and tried to fix it. That turned out to be harder then I thought. The MPEG chip has a special border color register that is written by the MV_BColor driver call and it seemed enough to just pass the color value to the MPEG window overlay routines. Well, not so. Again the issue turned out to be timing of decoder status signals, but of a different kind. The driver doesn't write the border color registers until it has seen some progress in certain timing registers related to the picture buffering thing, presumably to avoid visual flashes or something on the actual hardware. Fortunately, it turned out to be easy to simulate that progress, taking care not to trigger the complicated picture buffer code that I so far managed to keep dormant.

At some point, possibly related to slow motion or freeze frame, I might need to actually tackle that code but I hope to by that time have gained more understanding of the supposed workings of the MPEG chip.

Looking at the above, you might think that all of the difficulties are with the MPEG video decoding and that is indeed mostly true. I did have to fix something in the MPEG audio decoding, related to the pause/resume problems, and that was the updating of the audio decoder clock. When audio and video playback are synchronized the MPEG video driver uses the MPEG audio clock as it's timing reference, which means that it has to be stopped and restarted when video playback control operations occur. Since I had never before seriously tested this, the audio clock wasn't stopped at all and the video driver obligingly continued decoding and displaying pictures until it ran out of buffered data.

There is currently just one known problem with the MPEG audio decoding: the audio isn't properly attenuated as specified by the driver. This causes little audio distortions at some stream transitions and when buffers run out. There is also a problem with base case audio synchronization but that is hard to trigger and possibly even not audible in many titles so I'll worry about that much later.

Above I promised to get into the MPEG data buffering and decoding issue. The basic problem is one of conceptual mismatch: the CD-i decoding hardware gets data "pushed" into it (by DMA or direct device I/O) at the behest of the driver, whereas the MPEG decoding code (based on the publicly available mpeg2dec and musicout programs from the MPEG Software Simulation Group) expects to "pull" the data it needs during decoding. Things get messy when the decoding runs out of data, as the code does not expect to ever do so (it was originally written to decode from a disc file which of course never runs out of data until the end of the sequence). Some obvious solutions include putting the decoding in a separate thread (which given multi-core processors might be a good idea anyway from a performance perspective) and modifying it to become restartable at some previous sync point (most easily this would be the start of an audio frame or a picture or picture slice). Both options are somewhat involved although they have obvious benefits, and it may turn out that I will need to do one of them anyway at some point. For now I've avoided the problems by carefully timing calls into the MPEG decoding code so that enough data to decode a single audio frame or video picture should always be available; the MPEG data stream at the system level contains enough timestamp and buffering information to make this possible (in particular, it specifies the exact decoding time of every audio frame or video picture in relation to the timing of the data stream, thus making it possible to make those calls into the decoding code at a time that a valid MPEG data stream will have already filled the buffers far enough).

The approach depends on the timing of the MPEG data entering the decoder, which means that it does not handle buffer underflow conditions unless you add some kind of automatic decoding that continues even if no more MPEG data appears, and this is basically what I’ve done. In the end it was just relatively straightforward extension of the automatic decoding already there to handle the fact that MPEG audio streams do not have to explicitly timestamp every single audio frame (the CD-i Green Book does not even allow this unless you waste massive amounts of space in each MPEG audio data sector) and would have been needed anyway to correctly decode the last pictures of a sequence, but that had never been tested before.

For performance and possible patent reasons I have taken care to edit the MPEG decoding code (placing appropriate #ifdef lines at the right places) so that only MPEG 1 video and audio layer I/II decoding code is compiled into the CD-i Emulator executable. This is all that is needed for CD-i anyway and MPEG 2 video and audio layer III greatly complicate the decoding and thus significantly enlarge the compiled code.

Being somewhat stymied at the FMV front, I next decided to spend some time on another lingering issue. During testing, I often have to do the same exact sequence of mouse actions to get a CD-i application to a problem point and this is starting to be annoying. Input recording and playback are a partial solution to this but then you still have to wait while the application goes through it, which is also annoying and can sometimes take quite some time anyway. The obvious solution is a full emulation state save/restore feature, which I've given some thought and started implementing. It's nowhere near finished, though.

During the MESS collaboration I spent some time investigating the MESS save/restore mechanism. If at all possible I would love to be compatible for CD-i emulation states, but it turns out to be quite hard to do. The basic internal mechanism is quite similar in spirit to what I developed for CD-i Emulator, but it's the way the data is actually saved that makes compatibility very hard. Both approaches basically boil down to saving and restoring all the relevant emulation state variables, which includes easy things like the contents of cpu, memory and device registers but also internal device state variables. The latter are of course not identical between different emulators but they could probably be converted if some effort was thrown at it and for a typical device they aren't very complex anyway. The MESS implementation uses an initialization-time registration of all state variables; at save/restore time it just walks the registrations and saves or restores the binary contents of those variables. CD-i Emulator has a somewhat more flexible approach; at save/restore time it calls a device-specific serialize function to save or restore the contents of the state variables. The actual registration / serialization codes are structurally similar in the two emulators (a simple list of macro/function calls on the state variables) but the code runs at different times.

The real problem is that MESS includes very little meta information in the save files: only a single checksum of all the names and types of registered state variables in registration order. This is enough to validate the save data at restore time if the state variables of the saving emulator exactly match those of the restoring emulator, because there is no information to implement skipping or conversions. This holds between different versions or in some case even configurations of MESS emulators, but it holds even more so between MESS and CD-i Emulator! The meta information could of course be obtained from the MESS source code (relatively simply macro modifications could cause it to be written out) but that would require exact tracking of MESS versions because every version could have its own checksum corresponding to different meta information (in this case CD-i Emulator would need meta information sets for every MESS checksum value it wants to support).

I want CD-i Emulator to be more flexible, especially during development, so I decided to make full meta information an option in the save file. The saved state of every device is always versioned, which allows the save/restore code to implement explicit conversion where needed, but during development this isn't good enough. With full meta information turned on, the name and type of every state variable precedes the save data for that variable in the save file. This allows more-or-less automatic skipping of unknown state variables and when properly implemented the restore code can also handle variable reordering. At release time, I will fix the version numbers and save full metadata information sets for those version numbers so that the same automatic skipping and handling of reordering can be done even if the metadata isn't in the save file (it probably won't be because of file size considerations, although that may turn out to be a non-issue because save files need to include the full RAM contents anyway which is 1 MB of data in the simplest case without any compression, which is of course an option).

In addition to all of the above, I made some progress on the ROM-less emulation front. First I spent some time reading up on the internals of OS-9 file managers, because writing a replacement for the NRF file manager (NRF = Nonvolatile RAM File manager) seemed the logical next step. Actually writing it turned out not to be that hard, but there were of course bugs in the basic ROM emulation code. Most of them had to do with handlers not calling into the original ROM, which totally screwed up the tracing code. Some new functionality was also needed to properly read/write OS-9 data structures inside the emulated machine from the ROM emulation code; I wanted to implement this in such a way that compilation to "native" 68000 code remains a future option for ROM emulation modules. And of course the massive tracing described in the previous blog post had to be curtailed because it was impossible to see the relevant information in the morass of tracing output.

The new emulated NRF stores its files in the PC file system and it currently works fine when you start it with no stored files (i.e., the player will boot). In that case it will write out a proper "csd" (Configuration Status Descriptor) file. However, if this file already exists, the player crashes, although I have so far not found any fault in the NRF code. The origin of the problem probably lies elsewhere; I suspect it has to do with the hidden "player_shell_settings.prf" file. This file is read and written by the ROM bootstrap even before OS-9 is running; it does this by directly accessing the NVRAM memory (the file never changes size and is always the first one in NVRAM). Since the bootstrap accesses of this file do not go through the NRF file manager or even the NVRAM driver they are not redirected by the OS-9 emulation. However, later accesses by the player shell *are* redirected and the player shell does not seem able to handle the file not existing in the PC file system in the case where a csd file already exists. Solutions include extending the emulated NRF to always access this particular file from the NVRAM instead of the PC file system or somehow synchronizing the two locations for the file. The latter is probably the easiest route given the fixed location and size of the file, but the former is also useful as it would provide a full reimplementation of the original NRF that could in principle be compiled to native 68000 code to replace the "original" NRF in ROM (this is where gcc comes in as alluded to earlier, since all emulation code is written in C++).

In either case, I do not want the file manager to directly access emulated NVRAM although it could do so easily, as there is already an internal CNvramPort interface that provides just such access independent of the actual emulated NVRAM chip. The NRF file manager should instead call the NVRAM driver, which means that I need to implement cross-module calling first. It's not really hard in principle, the design has been done but there are a lot of little details to get right (the most obvious implementation uses at least 66 bytes of emulated stack space on each such call which I find excessive and might not even work; smarter implementations require some finicky register mask management or a "magic cookie" stacking approach, the latter having the best performance in the emulation case but being impossible in the native 68000 compilation case). When cross-module calling is working, I can also have the file manager allocate emulated memory and separate out the filename parsing functions by using the OS-9 system calls that provide these functions (the current emulated NRF does not allocate emulated memory which is arguably an emulation error and has the filename parsing coded out explicitly).

When everything works correctly with the emulated NRF, I have to find some way of integrating it in the user experience. You could always start over without any NVRAM files, but I'd like to have some way of migrating files between the two possible locations without having to run CD-i Emulator with weird options. Extending the CD-i File Extractor (cdifile) by incorporating (part of) the emulated NRF seems the obvious choice, which would also provide me with some impetus to finally integrate it with the CD-i File Viewer (wcdiview) program that's supposed to be a GUI version of cdifile but so far is just a very thin skeleton barely able to graphically display a single CD-i IFF image file passed on the command line (it doesn't even have a File Open menu) and will often crash. A proper implementation would look like Windows Explorer with a tree view on the left (CD-i file system, real-time channels and records, IFF chunk structure, etc) and a variable content display on the right (raw data view, decoded sector view, code disassembly view, graphical image view, audio playback, slideshow playback, decoded MPEG view, MPEG playback, etc).

That touches on another area in which I did some work last month: the saving of CD-i IFF image files for each emulated video frame. The motivation for this was to bring full-resolution real-time frame saving into the realm of the possible, as it would write only about 2 x (1024 + 280 x (384 + 32)) = 247 KB of raw CD-i video and DCP data per frame instead of 560 x 768 x 3 = 1260 KB of raw RGB. At least on my PC this has turned out not to be the case, however. The data is written out fine, which is not as easy as it sounds since video line data size can vary with each line because of pixel repeat and run-length encoding, but it's still too slow. That being so, I am not really very motivated to extend the CD-i IFF decoding implementation to actually decode this information. Some kind of compression could be an option, but that takes processor time and makes things even harder and possibly slower. Perhaps using another thread for this would be a solution, on a multi-core machine this should not greatly impact the basic emulation performance nor the debugging complexity as the compression code would be independent of the emulation itself.

So there is still a lot of work to be done, but it's all quite interesting and will provide for some entertaining evenings and weekends in the coming weeks or possibly months.

ROM-less emulation progress

2010-11-08T01:40:00.003+01:00

Over the last two weeks I have implemented most of the high-level emulation framework that I alluded to in my last post here as well as a large number of tracing wrappers for the original ROM calls. In the next stage I will start replacing some of those wrappers with re-implementations, starting with some easy ones.

It turns out I was somewhat optimistic; so far I have wrapped over 450 distinct ROM entry points (the actual current number of wrappers is 513 but there are some error catchers and possible duplicates). Creating the wrappers and writing and debugging the framework took more effort then I expected, but it was worth it: every call to a ROM entry point described or implied by the Green Book or OS-9 documentation is now wrapped with a high-level emulation function that so far does nothing except calling the original ROM routine and tracing its input/output register values.

Surely there aren't that many application-callable API functions, I can hear you think? Well actually there are, for sufficiently loose definitions of "application-callable". You see, the Green Book specifies CD-RTOS as being OS-9 and every "trick" normally allowed under OS-9 is theoretically legal in a CD-i title. That includes bypassing the OS-supplied file managers and directly calling device drivers; there are many CD-i titles that do some of this (the driver interfaces are specified by the Green Book). In particular, all titles using the Balboa library do this.

I wanted an emulation framework that could handle this so my framework is built around the idea of replacing the OS-9 module internals but retaining their interfaces, including all the documented (and possibly some undocumented) data structures. One of the nice features of this approach is that native ROM code can be replaced by high-level emulation on a routine-by-routine basis.

How does it really work? As a start, I've enhanced the 68000 emulation to possibly invoke emulation modules whenever an emulated instruction generates one of the following processor exceptions: trap, illegal instruction, line-A, line-F.

The emulation modules can operate in two modes: either copy an existing ROM module and wrap its entry points, or generate an entirely new memory module. In both cases, the emulation module will emit line-A instructions at the appropriate points. The emitted modules will go into a memory area appropriately called "emurom" that the OS-9 kernel scans for modules. Giving the emitted modules identical names but higher revision numbers then the ROM modules will cause the OS-9 kernel to use the emitted modules.

This approach works for every module except the kernel itself, because it is entered by the boot code before the memory scan for modules is even performed. The kernel emulation module will actually patch the ROM kernel entry point so that it jumps to the emitted kernel module.

The emitted line-A instructions are recognized by the emulator disassembler; they are called "modcall" instructions (module call). Each such instruction corresponds to a single emulation function; entry points into the function (described below) are indicated by the word immediately following it in memory. For example, the ROM routine that handles the F$CRC system call now disassembles like this:

modcall kernel:CRC:0
jsr XXX.l
modcall kernel:CRC:$
rts

Here the XXX is the absolute address of the original ROM routine for this system call; the two modcall instructions trace the input and output registers of this handler. If the system call were purely emulated (no fallback to the original ROM routine) it would look like this:

modcall kernel:CRC:0
modcall kernel:CRC:$
rts

Both modcall instructions remain, although technically the latter is now unnecessary, but the jsr instruction has disappeared. Technically, the rts instruction could also be eliminated but it looks more comprehensible this way.

One could view the approach as adding a very powerful "OS-9 coprocessor" to the system.

If an emulation function has to make inter-module calls, complications arise. High-level emulation context cannot cross module boundaries, because the called module may be native (and in many cases even intra-module calls can raise this issue). For this reason, emulation functions need additional entry points where the emulation can resume after making such a call. The machine language would like this, e.g. for the F$Open system call:

modcall kernel:Open:0
modcall kernel:Open:25
modcall kernel:Open:83
modcall kernel:Open:145
modcall kernel:Open:$
rts

The numbers following the colon are relative line numbers in the emulation function. When the emulation function needs to make a native call, it pushes the address of one such modcall instruction on the native stack, sets the PC register to the address it wants to call and resumes instruction emulation. When the native routine returns, it will return to the modcall instruction which will re-enter the emulation function at the appropriate point.

One would expect that emulation functions making native calls need to be coded very strangely: a big switch statement on the entry code (relative line number), followed by the appropriate code. However, a little feature of the C and C++ languages allows the switch statement to be mostly hidden. The languages allow the case labels of a switch statement to be nested arbitrarily deep into the statements inside the switch.

The entire contents of emulation functions are encapsulated inside a switch statement on the entry number (hidden by macros):

switch (entrynumber)
{
case 0:
...
}

On the initial call, zero is passed for entrynumber so the function body starts executing normally. Where a native call needs to be made, the processor registers are set up (more on this below) and a macro is invoked:

MOD_CALL(address);

This macro expands to something like this:

MOD_PARAMS.SetJumpAddress(address);
MOD_PARAMS.SetReturnLine(__LINE__);
return eMOD_CALL;
case __LINE__:

Because this is a macro expansion, both invokations of the __LINE__ macro will expand to the line number of the MOD_CALL macro invokation.

What this does is to save the target address and return line inside MOD_PARAMS and then return from the emulation function with value eMOD_CALL. This value causes the wrapper code to push the address of the appropriate modcall instruction and jump to the specified address. When that modcall instruction executes after the native call returns, it passes the return line to the emulation function as the entry number which will dutifully switch on it and control will resume directly after the MOD_CALL macro.

In reality, the code uses not __LINE__ but __LINE__ - MOD_BASELINE which will use relative line numbers instead of absolute ones; MOD_BASELINE is a constant defined as the value of __LINE__ at the start of the emulation function.

The procedure described above has one serious drawback: emulation functions cannot have "active" local variables at the point where native calls are made (the compiler will generate errors complaining that variable initialisations are being skipped). However, the emulated processor registers are available as temporaries (properly saved and restored on entry and exit of the emulation function if necessary) which should be good enough. Macros are defined to make accessing these registers easy.

When native calls need to be made, the registers must be set up properly. This would lead to constant "register juggling" before and after each call, which is error-prone and tedious. To avoid it, it is possible to use two new sets of registers: the parameter set and the results set. Before a call, the parameter registers must be set up properly; the call will then use these register values as inputs and the outputs will be stored in the results registers (register juggling will be done by the wrapper code). The parameter registers are initially set to the values of the emulated processor registers and also set from the results registers after each call.

The following OS-9 modules are currently wrapped:

kernel nrf nvdrv cdfm cddrv ucm vddrv ptdrv kbdrv pipe scf scdrv

The *drv modules are device drivers; their names must be set to match the ones used in the current system ROM in order to properly override those. The *.brd files in the sys directory have been extended to include this information like this:

** Driver names for ROM emulation.
set cddrv.name=cdapdriv
set vddrv.name=video
set ptdrv.name=pointer
set kbdrv.name=kb1driv

The kernel emulation module avoids knowledge of system call handler addresses inside the kernel by trapping the first "system call" so that it can hook all the handler addresses in the system and user mode dispatch tables to their proper emulation stubs. This first system call is normally the I$Open call for the console device.

File manager and driver emulation routines hook all the entry points by simply emitting a new entry point table and putting the offset to it in the module header. The offsets in the new table point to the entry point stubs (the addresses of the original ROM routines are obtained from the original entry point table).

The above works fine for most modules, but there was a problem with the video driver because it is larger then 64KB (the offsets in the entry point are 16-bit values relative to the start of the module). Luckily there is a text area near the beginning of the original module (it is actually just after the original entry point table) that can be used for a "jump table" so all entry point offsets fit into 16 bits. After this it should have worked, but it didn't because it turns out that UCM has a bug that requires the entry point table to *also* be in the first 64KB of the module (it ignores the upper 16-bits of the 32-bit offset to this table in the module header). This was fixed by simply reusing the original entry point table in this case.

One further complication arose because UCM requires the initialisation routines of drivers to also store the absolute addresses of their entry points in UCM variables. These addresses were "hooked" by adding code to the initialisation emulation routine that changes these addresses to point to the appropriate modcall instructions.

All file managers and drivers contain further dispatching for the SetStat and GetStat routines, based on the contents of one or two registers. Different values in these registers will invoke entirely separate functions with different register conventions; they really must be redirected to different emulation functions. This is achieved by lifting the dispatching to the emulation wrapper code (it is all table-driven).

Most of the above has been implemented, and CD-i emulator now traces all calls to ROM routines (when emurom is being used). A simple call to get pointing device coordinates would previously trace as follows (when trap tracing was turned on with the "et trp" command):

@00DF87E4(cdi_app) TRAP[5812] #0 I$GetStt <= d0.w=7 d1.w=SS_PT d2.w=PT_Coord
@00DF87E8(cdi_app) TRAP[5812] #0 I$GetStt => d0.w=$8000 d1.l=$1EF00FD

Here the input value d0.w=7 is the path number of the pointing device; the resulting mouse coordinates are in d1.l and correspond to (253,495),

When modcall tracing is turned on, this "simple" call will trace as follows:

@00DF87E4(cdi_app) TRAP[5812] #0 I$GetStt <= d0.w=7 d1.w=SS_PT d2.w=PT_Coord
@00F86EE0(kernel) MODCALL[16383] kernel:GetStt:0 <= d0.w=7 d1.w=$59 [Sys]
@00F86D10(kernel) MODCALL[16384] kernel:CCtl:0 <= d0.l=2 [NoTrap]
@00F86D1A(kernel) MODCALL[16384] kernel:CCtl:$ =>
@00F8A460(ucm) MODCALL[16385] ucm:GetPointer:0 <= u_d0.w=7 u_d2.w=0
@00FA10A4(pointer) MODCALL[16386] pointer:PtCoord:0 <= d0.w=7
@00FA10AE(pointer) MODCALL[16386] pointer:PtCoord:$ => d0.w=$8000 d1.l=$1EF00FD
@00F8A46A(ucm) MODCALL[16385] ucm:GetPointer:$ =>
@00F86D10(kernel) MODCALL[16387] kernel:CCtl:0 <= d0.l=5 [NoTrap]
@00F86D1A(kernel) MODCALL[16387] kernel:CCtl:$ =>
@00F86EEA(kernel) MODCALL[16383] kernel:GetStt:$ =>
@00DF87E8(cdi_app) TRAP[5812] #0 I$GetStt => d0.w=$8000 d1.l=$1EF00FD

You can see that the kernel dispatches this system call to kernel:GetStt, the handler for the I$GetStt system call. It starts by doing some cache control and then calls the GetStat entry point of the ucm modules, which dispatches it to its GetPointer routine. This routine in turn calls the GetStat routine of the pointer driver, which dispatches it to its PtCoord routine. This final routine performs the actual work and returns the results, which are then ultimately returned by the system call, after another bit of cache control.

The calls to ucm:GetStat and pointer:GetStat are no longer visible in the above trace as the emulation wrapper code directly dispatches them to ucm:GetPointer and pointer:PtCoord, respectively; it doesn't even trace the dispatching because this would result in another four lines of tracing output.

As a sidenote, all of the meticulous cache and address space control done by the kernel is really wasted, as CD-i systems do not need these. But the calls are still being made, which makes the kernel needlessly slow; one major reason why calling device drivers directly is often done. Newer versions of OS-9 eliminate these calls by using different kernel flavors for different processors and hardware configurations.

The massive amount of tracing needs to be curtailed somewhat before further work can productively be done; this is what I will start with next.

I have already generated fully documented stub functions for the OS-9 kernel from the OS-9 technical documentation; I will also need to generate for all file manager and driver calls, based on the digital Green Book.

It is perhaps noteworthy that some kernel calls are not described in any of the OS-9 version 2.4 documentation that I was able to find, but they *are* described in the online OS-9/68000 version 3.0 documentation.

Some calls made by the native ROMs remain undocumented but those mostly seem to be CD-i system-control (for example, one of them sets the front display text). Of the OS-9 kernel calls, only the following ones are currently undocumented:

F$AllRAM
F$FModul
F$POSK

Their existence was inferred by the appropriate constants existing in the compiler library files, but I have not seen any calls to them (yet).

CD-i Emulator Cookbook

2010-10-27T01:45:00.002+02:00

Just a quick note that work on CD-i Emulator hasn't stopped.

I have some wild ideas about ROM-less emulation; this would basically mean re-implementing the CD-RTOS operating system. Somewhat daunting; it contains over 350 separate explicit APIs and callable entry points and many system data structures would need to be closely emulated. But it can be done, CD-ice proved it (although it took a number of shortcuts that I want to avoid).

I'm not going to tackle that by myself; my current thinking is to make a start by implementing a high-level emulation framework, tracing stubs for all the calls (luckily these can mostly be generated automatically from the digital Green Book and OS-9 manuals) and some scaffolding and samples.

One of the pieces of scaffolding would be a really simple CD-i player shell; one that just shows a big "Play CD-i" button and then starts the CD-i title :-)

For samples I'm thinking about a few easy system calls like F$CRC, F$SetCRC, F$SetSys, F$CmpNam, F$PrsNam, F$ID, F$SUser, F$Icpt, F$SigMask, F$STrap, F$Trans, F$Move, F$SSvc (I may not get through the entire list) and a new NVRAM File Manager (NRF).

It would be nice to do a minimal UCM with Video and Pointer driver so that the simple CD-i player shell would run, but that might be too much. We'll see.

However, it's the new NRF that would be the most immediately interesting for CD-i Emulator users. It would intercept NVRAM access at the file level and redirect it to the PC file system (probably to files in the the nvr directory). This would allow easy sharing of CD-i NVRAM files (e.g. game-saves) over player types or between CD-i emulator users.

To allow all of the above and clean up some dirty tricks that were needed for input playback and handling Quizard, I've done some internal restructuring of CD-i Emulator. In particular, I introduced a new "handler" class beneath the existing "device" and "memory" classes (which are now no longer derived from each other but from a common "component" base class). This restructuring isn't finished yet, but it will allow the input and Quizard stuff to become handlers instead of devices (the latter is improper because they shouldn't be visible on the CD-i system bus).

The new "module" class (a subclass of handler) will be used to add high-level emulation of OS-9 and CD-RTOS rom modules. I want to preserve the interfaces between the modules and the public data structures as much as possible, because it will allow a gradual transition from "real" to "emulated" modules.

To prepare for all of the above I had to do some fairly heavy design, which caused me to properly write down some of the design information and tradeoffs for the first time. This will be invaluable information for co-developers (if they ever materialize), hence the title "CD-i Emulator Cookbook". Well, at present it's more like a leaflet but I hope to expand it over time and also add some history.

Pieces of the cookbook will be added to the CD-i Emulator website if I feel they're ready.

I've also been giving some thought on a collaboration model for the ROM-less emulation. If there is interest I could do a partial source release that would allow other developers to work on the ROM-less emulation. This release would *not* contain any non-video chipset emulation but it would contain "generic" CD-i audio and video decoding. You would still need to use (part of) a system ROM (in particular the OS-9 kernel and file managers) until enough of the emulation is finished.

I'm still considering all this, but I wanted to get the word out to see if there is interest and to show that I haven't abandoned the project.

Potential co-developers should start boning up on OS-9 and CD-RTOS. All of the technical OS-9 documentation is online at ICDIA and links to the digital Green Book can also be found.

Release blues and Quizard 1.7

2010-10-12T03:35:00.002+02:00

Saturday I released CD-i Emulator 0.5.3-beta1. That was a big milestone for me.

The Internet didn't seem to feel that way. There was some interest but it was nothing compared to five years ago when the first version came out. Oh well...

Sunday a "disc activity" indicator was suggested by a MESS tester. It seemed cool so I added it by coloring the background of the "disc activity" textbox:

blue = reading TOC data
red = reading CD-audio data (mode 0)
yellow = reading CD-ROM data (mode 1)
green = reading CD-i data (mode 2)

I also wrote a proof-of-concept compatibility report table page for the web site so that I could view the reports entered, so far by a single user only (except myself). It needs more work, I won't tell you the URL yet :-)

A few days ago the MESS CD-i programmer requested help in figuring out why the Quizard 1.7 game didn't work and I provided some serial port help. In the process we deduced that the 1993-vintage 68070 datasheets on the Internet all have some of the UART command bits wrong. But I found a few datasheets dated April 1991 which have it right (and so does my 68070 User Manual, luckily, which is dated November 1991). With this fixed in MESS, both emulators produced the same effect: some loading activity and then remaining at a black screen.

Now, Quizard is one of those rare CD-i titles that require external hardware, in this case the so-called "CD-i Jamma" board that interfaces a CD-i player to the industry-standard (at one time) arcade enclosure interface. Bas wrote about this board in his Interactive Dreams blog here and here.

Part of the external interface is a "security" option that causes the game not to run unless the proper hardware is present, and this is what was causing Quizard to remain at a black screen.

Although it is certainly possible in principle to emulate the external board, including its security features, I was in the midst of releasing CD-i Emulator so I let it go for some time.

But today Just Deserts/Harmony/MooglyGuy (the main MESS CD-i developer) posted details about how to "patch" Quizard to avoid the security check and he also posted sufficient details to allow emulation of the arcade case buttons. So I dove in...

It took me an hour or two to create a special "quizard" device for CD-i Emulator that patches Quizard as needed and connects to the serial port to give the CD-i title the communications that it wants. For now I simply intercept PC keyboard digits 0 to 9 and pass them to Quizard as "button" information.

It works alright: Quizard plays its leader and starts "attract" mode and I can even play a game!

The following movie shows only the leader, however:

Note also the green flashing disc activity indicator!

CD-i Emulator 0.5.3-beta1 released!

2010-10-10T00:09:00.004+02:00

I have just released the new version of CD-i Emulator. It has received very limited pre-beta testing, but that's what public betas are for.

I've changed relatively little since the prebeta1 version.

The most import change is the addition of the Help | Report menu option that contains a link to the new Report section of the website and will automatically fill in most of the information fields on that form when clicked.

Unfortunately, to make this work properly I had to touch most files in the sys directory to add correct player model, extension and digital video cartridge identification.

I've also added checksum reporting of DV cartridges, let's find out how complete my current collection is. These checksums will not (yet) trigger an "Unknown ROM" dialog if unknown, but they are posted with compatibility reports.

The -writepng option also had to be fixed to support macros in its filename, otherwise each video frame would overwrite the previous one! The macros that I've decided to support for now are $seq$, which produces a 6-digit sequence number and $time$ which produces the 6-digit frame time (mmssff).

The Release Notes have been modified to include these changes.

Use of the input recording/playback functions revealed a number of bugs; I've fixed those and also added more player information recording so that recorded input files contain enough information for compatibility reports (this isn't used yet).

Website updates, bugfixes, compatibility reports

2010-10-04T00:23:00.005+02:00

Today I updated the CD-i Emulator website for the upcoming beta release and fixed some remaining bugs, mainly with input recording/playback.

I updated the Release Notes section, put up the hopefully final version of the Full Release Notes, added more screenshots and YouTube embeds and did some miscellaneous text edits. I also updated the CD-i Types section and added a download for it.

It was my intention to release tonight, but that didn't happen because the closed pre-beta testers aren't ready yet. I'll give it a few more days.

My next project will be updating the Title Support section of the website; I want to replace the static table with a much more dynamic one that can be expanded and sorted in various ways, based on a database of actual compatibility reports for specific CD-i player ROMs and CD-i titles that could be entered online.

I might even accept MESS CD-i compatibility reports to provide a basis for comparison. Users would also be able to "me too" on reports.

I would also like to add a "Report Compatibility" option to CD-i Emulator; that would allow automatic filling in of most of the report fields so that entering a compatibility report becomes pretty much a question of clicking only a "compatibility class" button and possibly entering some notes (fields like Reporter could be remembered on the reporter's PC using cookies). It should be easy to add the CD-i Emulator part of this; I might even do it for the current beta if the testers take long enough :-)

For crashes or decoding issues, it would also be useful to allow uploading of input recordings. Actually, uploading a "good recording" would be nice even for successful reports. I could even add an "autorecord" setting to CD-i Emulator to make recording easy; the performance hit should be minimal.

This would also allow "batch" uploading of compatibility reports at some later time; input recordings contain all of the necessary information except for the compatibility class and notes and these could easily be added with input annotations, which I was planning to add anyway.

Unfortunately, uploading a file cannot be automated easily from within CD-i Emulator: the user will have to manually browse for the file. Another way to handle this would be to automatically generate an e-mail compatibility report...

Lucky Luke on YouTube

2010-10-02T12:00:00.005+02:00

I have just posted a "proof of concept" demonstration of CD-i Emulator MPEG decoding on YouTube:

The Philips Media bumper animation is MPEG (audio and video) and so is the audio for the piano sequence. The in-game crash is due to an MPEG sound effect.

This is with a Gate Array MPEG AH01 cartridge (22ER9141 F1 AH01); as the cartridge types get newer the decoding starts working lesser and lesser. Later GMPEG cartridges will crash earlier in the gameplay, VMPEG and IMPEG will not play MPEG at all. This is simply because I haven't gotten around to fixing those things yet; there are still major bugs in the buffering of the MPEG data and while that remains unfixed there is no point.

The magenta border indicates "beta" quality; it is not that hard to display the proper color (black in this case), but this will remind everyone that it is a work in progress.

Many other DVC titles will play their Philips bumper but crash soon thereafter; some others won't even get that far. In some cases this is not even due to MPEG decoding issues but to various "other" reasons for crashing (The 7th Guest is a notorious example).

I have also put up a new draft of the upcoming version 0.5.3-beta1 Release Notes on the website.

Release Notes

2010-10-02T00:19:00.000+02:00

I have just put up a first draft of the Release Notes for the upcoming version 0.5.3-beta1 on the CD-i Emulator website.

More info coming...

CD-i Emulator pre-beta released

2010-10-01T00:39:00.002+02:00

I have just released the pre-beta version of CD-i Emulator to my beta test group.

This version is intended for quick tests; if nothing drastically happens, the final beta1 release will happen in a few days.

In the meantime, I will be writing release notes and updating the CD-i Emulator web site. The final release notes will appear in the Release Notes section of the web site.

I managed to finish the input recording/playback code and also fix a few minor bugs.

Bug reports, comments or suggestions about this version should be posted on the closed CD-i Emulator beta Testing Forum.

Finalizing input record/playback

2010-09-30T00:34:00.005+02:00

Today I resumed working on the input record/playback code to get it towards its final specification for this beta.

I want the record/playback code to be generic, by which I mean:

1. Recorded input files must be dumpable (e.g. by cdifile) without having to know intimate details of the recorded devices and messages. This can be achieved by labeling all input channels with name and type and recording the formats of all input message types, which is now mostly done.

2. Recorded input files must contain all the information required for faithful playback, which includes things like CD-i player model, DVC cartridge type, extension roms, PAL/NTSC and special startup options and emulated disc insertions.

The ideal is that "wcdiemu -playback" will just work without the need to fiddle with emulator options to reproduce the recording environment. This is now also mostly done except for the disc insertions (they are recorded but cannot yet be played back).

3. It must in principle be possible to play back on a different CD-i player model or with a different DVC cartridge. This means that the input channel matching must be somewhat "intelligent". The groundwork for this is in place but the actual channel matching is not yet there.

All of the above are needed to get the feature usable for extensive compatibility testing, which is a goal of this beta: it should be possible to exchange input recordings that reproduce crashes or rendering bugs, even when they require some time to reproduce. Emulator state snapshots would make this even better, but that is too much work for now.

Recorded input files are in IFF format with the following generic structure:
- each file contains a single FORM chunk of type INPT (recorded input)
- the first chunk inside the FORM is an IVER chunk (version information)
- the next chunk inside the FORM is an ICHN chunk (channel information)
- the final chunk inside the FORM is an IMSG chunk (input messages)

Input messages are recorded in binary format to keep their size small; the first time that a message type is used, the message format string is prepended.

The cdifile dumping code will not be released with this beta, there's too much unfinished code in there (i.e. the -dir[ectory] option to display the directory of a CD-i disc image file).

I will attempt to finish this tomorrow, but it's just barely achievable...

Until then, you'll have to do with this picture:

Housecleaning

2010-09-28T00:13:00.003+02:00

Today I did some housecleaning for the upcoming beta, which I'm still hoping to release before the end of this month.

I decided to keep the current somewhat buggy MPEG decoding stuff in the beta, which meant that I had to clean up the MPEG ROM detection logic so that it actually works (until now I always used the -dvc option to explicitly force the DVC model).

The MPEG ROM detector will now only use the supplied "dummy" xmpeg.rom file if it cannot find another recognized ?mpeg*.rom file. If it does find such a file, the DVC type will now be displayed in the CD-i Emulator title bar (GMPEG, IMPEG, VMPEG). For GMPEG the AH0x number is also displayed, as the AH00/AH01 cartridges are very different from the AH02/AH03 ones.

Some little-known DVC tidbits:
- The Philips CD-i 370 uses VMPEG hardware but at a different memory address
- The Philips CD-i 615 uses IMPEG hardware but the ROM also contains non-FMV software
- The DVS VE-200 and LG GDI-700 use IMPEG hardware but at a different memory address

More details on the state of the MPEG emulation will be available in the beta release notes.

Earlier I had started using a manifest file to get a newer common controls DLL, thus modernizing the visual "look" of CD-i Emulator somewhat. However, this caused Windows Vista and Windows 7 to stop using "virtual store" redirection of the cdiemu.ini file which broke all settings stuff (including the recently used list).

Today I figured out that leaving out the UAC "requested level" entry from the manifest restores the "virtual store" redirection. This is good enough for this beta.

I also killed the "file registration" code because it would hijack any existing non-beta registration (this has nothing to do with licensing but with the way Windows finds executables to open certain files).

Finally, I added video decode skip validation. This gets a bit technical...

In CD-i players, video decoding steals bus cycles and thus slows down the machine. The amount of bus cycles stolen depends on the video mode, pixel repeat settings and, for run-length modes, on the actual data.

If the emulation starts to lag behind, CD-i Emulator starts skipping the decoding of video frames. Previously, I used a "worse-case" estimate for the number of stolen bus cycles in this case. However, this introduces problems with input recording and playback: if frame skipping differs (which is almost guaranteed to be the case), the playback will start to "drift" from the recording. Over many frames this will result in an incorrect playback.

I coded a "SkipLine" routine that does no actual video decoding like "DecodeLine" but it does decode just enough to accurately determine the number of stolen bus cycles. This routine is now used when the video frame is skipped.

The validation consists of running both routines when the video frame is not being skipped and ensuring that both "steal" the same number of bus cycles.

Quick status update

2010-08-01T00:11:00.006+02:00

It's been just over a half year since my last posting here. A quick status update...

In March I upgraded my trusty old desktop PC to a new laptop with a dual-core Pentium T4300 at 2.1 GHz. In the process of migrating my data I broke my old P-ATA hard disk :-(

The breakage was very minor (a broken connector lead) and seemed to be fixable with some soldering, but it took me nearly two months to get around to obtaining the proper cable and connector. Then I quickly made a full copy to a new S-ATA hard disk and breathed a sigh of relief.

I did have a backup but it was months old and did not include my most recent CD-i Emulator work, so I really wanted the up-to-date data.

However, as Charles commented on my previous entry, Visual Studio 6 has problems working under Windows 7. That demoralized me such that I left it hanging for a few months more. My older CD-i Emulator development builds didn't even work under Win7 :-(

Yesterday I finally got around to doing a full restore from the backup disk to my laptop and I also installed Visual Studio 2010 just to take a look at it.

Having done that, I realized that I could try building CD-i Emulator again!

I just finished doing that, and after finding the single source line that generated the 300+ errors I got on the first build, it appears that I now again have a working development build of the most recent CD-i Emulator, optimistically labeled v0.5.3-beta1. It seems to run like a charm, doing 50/50 frames on non-FMV titles and even 34/50 on my FMV test cases. Wow!

Using Visual Studio 2010 means that the minimum system requirement is now Windows XP, as earlier Windows versions are no longer supported as C++ build targets. But I don't think that should really be a problem.

At the end of next month it will be five years since the original CD-i Emulator v0.5.2 public release; I'm hoping to release 0.5.3-beta1 before that.

Please bear with me...

Input record/playback, I2M emulation and a new start

2010-01-20T22:34:00.002+01:00

It has been more then a month since my last blog entry. There are several reasons for this, among others the very busy month of december and a mild CD-i Emulator burn-out. This blog entry is part of an attempt to restart the process and hopefully arrive at a releasable beta version in the not too distant future.

Early december I did a little work on rewriting the input record/playback code to its final specifications, using the full IFF reading/writing code that I reported on in the previous entries. It was hairy work (refactoring often is) and I more or less threw in the towel at that point.

This was followed in mid december by an attempt to regain focus. Triggered by some forum discussions, I did some work on emulating the I2M Media Playback CD-i board for the PC (in a sense another completely different player generation).

This board uses a Motorola 68341 so-called "CD-i Engine" processor chip, which is a CPU32 processor core with some on-chip peripherals (a DMA controller, two different serial interfaces, a timer, etc), a VDSC video chip and a completely undocumented "host interface" to the PC bus. So far the board does not appear to have a separate CD/Audio interface, but it does have a VMPEG chip.

I had already implemented CPU32 emulation and partial emulation of its on-chip peripherals, but this needed to be extended a bit more. The main "problem" here appeared to be that these peripherals appear to ignore address bit 23.

I also had to reverse engineer the host interface, which amounted to some disassembly and tracing. In the process I also figured out that I had built a bad ROM image (the host software generates these on the fly from "ROM fragment" files and I had mis-interpreted the script file that tells it how to do this). I used the CD-i Playback 2.2.1 files for this.

The I2M board now successfully boots OS9, including displaying some tracing messages (it turns out that it can do this either via the host interface or via the serial interface, which was reason for lot's of head scratching until the "aha!" moment). The video display gets initialized (some kind of "blue" screen) and the then board "hangs" inside a watchdog process, presumably waiting on some signal from the host telling it what to do (it's not a crash but appears to be waiting for some host interrupt). Figuring this out requires disassembly of the watchdog process and that's where I stopped, having worked on it for about three evenings.

Somewhere in between (can't remember the exact timing) I also did a test build with Visual Studio 2008, which resulted in a few portability fixes but nothing dramatic. I had hoped this would make some speed difference but I didn't notice any. I've returned to my old trusty Visual Studio 6; it's smaller and faster and has everything I need.

That's more or less the current status; I hope to resume work on the input record/playback code soon which is the last thing holding up the current beta release.

IFF reading and AVI writing bugfix

2009-11-24T00:02:00.002+01:00

I've taken a few days off from CD-i work, mainly to catch some sleep. So there is only a small amount of progress to report today.

Yesterday I wrote and debugged proper IFF reading code. Well, it turns out the IFF reader itself was pretty much "obviously" correct but there were major bugs in the interaction of buffering and the SetPosition / SetLimit functions of my CArchive class.

Once finished, I added rudimentary IFF dump code (just chunk header info) to cdifile and proceeded to use that to test the IFF code against some "real" files (mainly WAVE and AVI).

Using my "bad" Burn:Cycle AVI from before I found the cause of the badness: the dwLength field of the audio 'strh' chunk was ridiculously large. Turns out I misread Media Player: the AVI claimed an audio length of 33 minutes although in reality it was only about 6. It had to be some bug in the aviio.c file that I've taken from MESS/MAME (this is allowed by the license for that file, which is now BSD).

I did come code reading and tried some "obvious" fixes, but not until I added tracing code today did I find the culprit: the total number of samples was being incremented way too much on each audio buffer flush. The fix was an obvious oneliner that I've also submitted to MAME and MESS.

You can download the fix here.

Burn:Cycle on YouTube

2009-11-19T00:52:00.002+01:00

It turns out that Ultra MP4 Video Converter could handle my "bad" AVI file, although the trial version only does the first five minutes. That's all right as the video contained no interesting stuff after that :-)

You can see it here.

AVI writing fixed and some fast machine issues

2009-11-19T00:38:00.003+01:00

In a new attempt to get a usable Burn:Cycle movie out of CD-i Emulator, I decided to add multiple PNG writing as a poor substitute for AVI writing. After all, if you have the images and audio, you can always postprocess them into a movie!

I hooked up the PNG writing code from MESS/MAME, which seemed simple enough. There was the small matter of adding 24 bits per pixel bitmap support (their code handles 16 and 32 but not 24 which is however the internal format of CD-i Emulator; I had already partially added such support for the AVI writing code).

However, the resulting PNG files would not display in Explorer or any other image tool, giving messages like 'Bad file format' and 'Cannot read'. In exploring the cause for this, I was looking at memory buffers being filled and suddenly noticed that they where being filled in little-endian byte order, which is definitely wrong as PNG is a big-endian byte order file format.

The source of the problem turned out the be a configuration error with the MESS/MAME code: it needs a macro LSB_FIRST so it knows that it's running on a little-endian machine, and I had forgotten to give it one. When I fixed this, the PNG files came out right. And even better, the AVI files suddenly also seemed correct (not surprisingly, as they use the same byte order flipping macros as the PNG writing code).

So now I have both PNG writing and AVI writing hooked up in CD-i Emulator. I proceeded to do some timing tests and in the process added a few new writing options. The -writesilent option writes AVI files without audio; the -writerate option sets the framerate for the AVI files.

Full framerate AVI files still play "jumpy" on my machine but that's caused by their uncompressedness (is that even a word?); if you convert them to MP4 for uploading they play perfectly.

I uploaded a first test to YouTube; you can see it here.

If you set the AVI framerate low enough, you can do real-time movie captures. On my slow home machine I can do real-time capturing with -writerate 2 or less (which gives very jumpy AVI files, but the audio is correct).

To do non-realtime movie rendering you need to use the input recording and playback options. I did that for one of my Burn:Cycle recordings (not a very good one, but it's hard to play on my machine as the framerate it really too low and as a result the mouse gets jerky, which makes you overshoot your targets, which results in easy death at the hands of the security people).

The rendering proceeded at about 15% real-time speed while I was typing the above and is now finished; it gave me an AVI file of 27 GB for a nearly 7 minute session. There's something wrong with that file; Media Player claims it's only 33 seconds but proceeds to play all 7 minutes of it. Huh?

I also have problems converting it to MP4, probably caused by the same problems. Currently testing conversion programs to find one that can handle it...

During all of the fast machine testing yesterday, I also noticed that after a few minutes the audio begins to lag behind when it shouldn't. This is probably caused by the audio not being decoded at exactly 44.1 kHz relative to the PC timers (or something like that). When audio is playing, it really needs to be taken as the master clock for emulation speed control but that is somewhat hard with the current speed control code. Not for this beta. To ameliorate the problem, I've added a "Flush Audio" option (keyboard shortcut Ctrl-F) that allows you to manually flush the lagging audio so that it regains emulation sync.

Yesterday I also started adding proper IFF reading code, as opposed to the basic version handling only a single large data chunk that is currently used by the input playback code. This is needed so I that can properly finish the recording and playback functionality.

The beta release seems to be coming nearer...

Fast machines, file writing and some bits

2009-11-17T01:02:00.002+01:00

Today I used my work PC to examine CD-i Emulator on a fast machine.

It turns out that cpu throttling was already there; titles run at the right speed. However, the frame skip detection logic is too sensitive: it will skip frames even on fast hardware, just because of a momentary slowness (I think). I added an option -noskip to turn off frame skipping altogether; beter autodetection is too much for this beta.

Likewise, for people who always want to run full-speed, I've added an option -nowait to turn off cpu throttling. This can make your CD-i title unplayably fast and it will also cause the audio to lag behind if the title plays continuous audio (most games do).

On the AVI writing front I was less lucky. I cannot get decent write performance and this is not caused by the AVI writing code; just writing the raw data to a file will already cause very "skippy" emulation. Perhaps using a separate thread would solve this (especially on a multicore machine).

And what's worse, the resulting AVI files seem to have timecode problems because they are also skippy, which should not be the case. This is really weird as I've previously been able to write AVI files correctly. It needs more work, obviously.

On the other hand, the AVI files contain the correct audio data: if I extract the audio stream with avidemux, it's identical to a separately written WAV file.

In the process of testing all of this (with Burn:Cycle, what else?) I also fixed the audio crackling problem; it turns out that Burn:Cycle opens all four audio attenuators (L-R, L-L, R-L, R-R) and for this possibility I hadn't accounted in the mix scaling.

I also added the two keyboard emulations that I mentioned yesterday: Backspace for button 2 and Esc for button 3 (buttons 1 and 2 together). This works well for The Apprentice and Dimo's Quest and these are my reference titles anyway :-) I will ultimately add a UI for changing the keyboard map, of course.

Soundmap cleanup, file writing, beta preparation

2009-11-16T00:23:00.003+01:00

Today I started with the planned cleanup of the revised soundmap playback code. This took some time but I had it finished my mid-afternoon (please note that I have small children around, which can sometimes make the progress quite slow).

By way of final test I did a 10 minute Burn:Cycle game session; there's still an audio decoding issue there because some parts of the music crackle heavily.

Then I continued working on the WAV / AVI writing front. I have this essentially working now; both types of files are correctly written (the AVI files now include both audio and video). There are new options -writewav and -writeavi to invoke these functions from the command line.

I used the WAV writing feature to check out the Burn:Cycle audio decoding issue. The audio samples go way out of range at the point of the crackles (they actually clamp against the minimum/maximum values). There is probably something wrong in the scaling or something.

One generic issue with WAV file writing is that silent periods (where the CD-i application doesn't play any audio) do not show up in the recorded audio. For the moment, I've decided that this is a feature :-)

There is also some kind of audio timing issue in AVI file writing; the recorded files sometimes sound "skippy". However, this may also be related to the speed of my PC; these are uncompressed AVI files which take 61.5 MB per second for the CD-i video data alone (50 x 3 x 768 x 560 bytes per second). Compared to this, audio is only a measly 172 KB per second :-)

I've found that real-time AVI writing is nearly impossible on my hardware; the framerate drops below 2/50 sometimes (I've lowered the previous 10/50 limit) which makes the titles unplayable. Tomorrow I'll try my work PC which is much faster. That should also give me an opportunity to add frame rate throttling; on fast PC's the emulator currently runs too fast.

However, non-realtime AVI writing also gives the "skippy" sound, so it may be unrelated. Sometimes the audio also gets way out of sync...

One worrying generic issue is that CD-i Emulator seems to have become slower as of late, for no good reason that I can think of. It may be that some debugging code somewhere is slowing things down, but I haven't found it yet. Or it could be that the titles I'm currently running are simple more demanding...

During testing, I also fixed a recent bug where mouse input and keyboard input interfere with each other. I also took out the use of the Shift key for button 2, as it is prone to generate false button presses when using Alt-Tab / Shift-Alt-Tab to switch between windows. You can still use the numeric "+" key for button 2, though, but that is too far away for generic use: I need an alternative closer to the space bar. Perhaps Backspace for button 2, and Esc for button 3 (buttons 1 and 2 together)?

Finally, I've started putting together the first v0.5.3 beta distribution. It will be mostly identical to a v0.5.2 one, with some updates in the sys directory, updated cdiroms.ini and cditypes.rul files and of course an updated executable which will be named wcdiemu-v053b1.exe (for beta 1) to avoid accidentally overwriting an existing v0.5.2 executable. I've also taken a first crack at a release notes document (very descriptively named BETA1).

Soundmap interrupt timing (II)

2009-11-15T03:15:00.002+01:00

Yesterday I did some thinking about the Lemmings sound issue and concluded that the soundmap interrupt timing of the hardware was very different from the one in CD-i Emulator (and MESS CD-i). Today I set out to correct this.

After the timing was right (which turned out to be two separate issues) there was still a sound issue, because every other Lemmings sound buffer was silent. This turned out to be caused by a misassumption on my part: a hardware register that I had assumed was the actual sound playback address turned out to be just the starting address for a new play operation (I concluded this because the register keeps getting reset to the same value, even though sound is active and the DMA buffer addresses keep properly alternating).

A whole lot of CD-i titles suddenly work much better or even apparently perfect:
- Lemmings now has background music and appears to play perfectly.
- Burn:Cycle appears to play perfectly.
- The professional title that was my 05140000 crash testcase plays perfectly.
- Sargon plays much better (it has frameskip issues).
- Escape from CyberCity menu screens now work (the actual game doesn't start).
And several other soundmixing titles still appear to work okay!

The code is still a bit messy and needs to be cleaned up, but I'm spent for today so it will have to wait a bit.

After the cleanup I will continue working on the WAV and AVI writing functions; these and input record/playback are the last issues holding up the beta release.

Soundmap interrupt timing (I)

2009-11-13T02:09:00.004+01:00

On the MESS board, attention was called to Lemmings not properly playing background music, with a plea for help.

So I started out trying to check this out in CD-i Emulator. There were some obstacles along the way, however.

The first obstacle was that the Lemmings menu screen would not display. This turned out to be very easy to fix, CD-i Emulator simply wasn't handling the CLUT4 display mode. I would have sworn this was implemented long ago, but at least it is now. I've also added RL3 display for the future.

Next, the main game would "hang" after issuing a SEEK command. Weird, as I've seen the SEEK command working in other titles. It turned out that the BIOS expected a certain bit to abort the incoming sectors and CD-i Emulator didn't honor that; it does now.

After that I got the exact same failure mode that MESS CD-i has currently: the "Let's go!" sound effect plays, followed by the sound effect that corresponds to the door opening, then background music starts to play, but dies.

Looking at a DMA trace I concurred with the MESS people that the cause was almost certainly the timing of "soundmap done" interrupts, but experiments in varying it have been unsuccesful so far.

At least Lemmings is now playable in CD-i Emulator, though without background music or sound effects :-(

Yesterday I also did some more work on AVI and WAV writing but it still isn't finished yet (too much infrastructure needed writing). I hope to finish it tomorrow.

IFF writing and DYUV color keying

2009-11-11T00:42:00.003+01:00

Today I implemented proper IFF writing with two new classes: CIffIoBase and CIffWriter. I will need this to write WAV files for the new -writewav option, as WAV files use the Microsoft RIFF format which is a variation of Electronic Arts IFF (Interchange File Format).

The new -writewav option will be a companion to -writepng and -writemng that I can mostly implement using MESS source code (just like -writeavi); the license for the MESS WAV writing code does not allow me to use it and reuse would not be feasible on technical grounds anyway because this code is not as modularly written as the MESS PNG/MNG and AVI writing code.

Proper IFF writing will also be useful for savestates, but that is not for the current beta.

Microsoft says that RIFF stands for Resource Interchange File Format but my source code calls it "Reversed IFF" which is also a correct description :-)

I tested the new IFF writer code by rewriting the CInputStream::Record functions to use it. These functions already wrote IFF format, but now they properly fill in the chunk sizes (previously I left them at 0xFFFFFFFF which limited the file to a single data chunk).

Debugging it took more time then I expected because my binary CArchive class turned out to have bugs in its Position() / SetPosition() functions for file i/o. So I didn't get to the WAV file writing but that is not hard.

Also faithfully emulated a hardware "quirk" that allows Jeopardy to properly display its menu screens, it makes assumptions about the color key bit when using DYUV mode; this was uncovered by the MESS guys. I've implemented this by adding a transparancy test against black which matches the Jeopardy DCP and seems more "correct" then the simple MESS fix of always forcing color key to true. To know this for sure a test program would have to be written but I didn't feel like it.

Beta licensing

2009-11-08T23:27:00.001+01:00

Today I finished up the licensing implementation for the upcoming v0.5.3 beta releases.

Each beta release will contain a single CD-i Emulator executable; it will act as a limited edition unless you have already activated an unlimited edition v0.5.2 on your machine or (at least for the initial releases) you're using a Mono-I player system ROM (BIOS).

The former exception has two purposes: it delivers the promised free compatibility updates to my existing customers, and it should encourage new customers as well (several people have promised to buy CD-i Emulator as soon as keyboard support was working).

The latter exception is mainly intended to allow for full compatibility tests against the MESS CD-i driver; I haven't decided yet for how long this will be maintained.

It will not be possible to directly activate beta releases, for that you will need an unlimited v0.5.2 edition (which will remain available). However, when the beta cycle is over, I will do a full unlimited release that can be activated directly (for paying customers only, of course).

After that I fixed a few bugs, including an outright crash that turned out to be caused by a small array misdeclaration in the signal tracking code (used only when you turn on Traps tracing using the debugger "et trp" command, but I often do this when tracking some bug). I had been looking for this for ages, but today it finally "clicked" that the crash only happened (sometimes) while tracing.

Also slightly extended the input record/playback functionality; the recording now includes the CD-i player model and title and the disc image being emulated. Next step is to automatically select these on playback and to account for a few options such as -playcdi that significantly impact the input.

Need to add input dump support to cdifile sometime...

Created a new title screen; this is a complex exercise as the screen is actually using a CLUT7 overlay over a DYUV background. When I get proper CD-i IFF support finished (also needed for save states) I will allow the title screen to be loaded from an external file, possibly customized per player model and/or CD-i title. Not for the first beta.

Proper CD-i IFF support would also allow a frame to be saved in about 250K (actual CD-i data including DCP) instead of the 1.2 MB for a full RGB dump. This would potentially make movie file generation faster, and it would allow offline inspection of video data and control information (useful to catch bugs).

CD-i keyboards

2009-11-08T00:58:00.003+01:00

Today I spent some time adding limited CD-i keyboard support to CD-i Emulator. It wasn't very hard to do because all the pieces were already there.

I had to disassemble the keyboard drivers for the SLAVE and IKAT processors to figure out the SLAVE/IKAT commands but after that it was smooth sailing. Luckily these are very small drivers.

This has limited utility because there are only a very few titles out there that support a CD-i keyboard. I know of the following ones:
- Philips/Standaard Interactieve Encyclopedie (to enter search terms)
- Philips Medische Encyclopediedisc (to enter search terms)
- CD-Online discs (to enter URLs)
All of these require a DV cartridge and are thus not (yet) supported by CD-i Emulator.

However, there is (very rudimentary) keyboard support in the Philips player shells, just enough to verify that your CD-i keyboard is actually working.

The "old player shell" (found on CD-i 60x, CD-i 200, CD-i 210 F2 and below, CD-i 220 F3 and below and CD-i 350) supports the "p" key (lowercase only!) as equivalent to clicking the "PLAY CD-I" button.

CD-i 605 with old player shell

The "new player shell" (found on CD-i 210 F3 and up, CD-i 220 F4 and up, CD-i 4xx, CD-i 615, CD-i 660 and CD-i 740) supports any key to abort the introductory animation.

CD-i 210 with new player shell

The above now also works in CD-i Emulator.

Supposedly one or both player shells also support the keyboard for naming favorite track selections, but I haven't verified this.

Most people will think of the "CD-i KeyControl" (22ER9042) when they hear the term "CD-i keyboard", but there also exists a true "CD-i Keyboard" (22ER9041). The former was introduced in 1996 with the CD-Online discs but the latter is much older, its specification was published in May 1993.

CD-i KeyControl, photo courtesy of http://www.freetimeweb.nl

CD-i Keyboard, photo taken by myself (click to enlarge)

CD-i Keyboard backside, photo taken by myself (click to enlarge)

The "CD-i Keyboard" is a full PC-AT style keyboard (without a "Windows" key, and using the term "SuperShift" for what is usually called "Alt") that connects to a CD-i player just like any other peripheral. It was intended for the professional market (specifically, the so-called "CD-IX" configuration) and is colored accordingly (whitish). This is the keyboard that the original CD-i keyboard protocol (later called "K-mode" because of it's ASCII "K" device identifier byte) was written for. It transmits key events in two-byte packets at 1200 baud.

The CD-i KeyControl also supports a so-called "T-mode" which uses a different protocol. The keyboard actually masquerades as a graphical tablet with a device identifier byte of ASCII "T". This protocol transmits key events in four-byte packets at 1200 baud.

Why was the "T-mode" introduced when there was already a perfectly usable keyboard device protocol? The reasons for this are arcane, to say the least, and provide a very nice "horror" story about product design and compatibility; it also illustrates the lengths that Philips went to in order to maintain the "All CD-i titles can be used with all CD-i players" principle.

First we need a little background. Most CD-i players have two input ports, usually one on the back and one on the front. Newer players support the use of a splitter cable to connect two peripherals to the front port, in reality these players have two "front" ports that are smartly wired together in a single connector (dual front port support is a feature of the IKAT coprocessor in the newer CD-i players, specifically Mono-III and later).

However, front and back ports are not functionally identical. Back ports are fully bidirectional RS232 ports supporting both serial input and serial output; they are invariably connected directly to the on-chip UART port of the 68070 processor. Front ports are unidirectional; they can only receive serial input and are connected via the SLAVE or IKAT coprocessor (an 68HC05).

The SLAVE processor fully supports the "K" protocol on its single input port, but the IKAT processor only supports it on its secondary port (the one that only becomes accessible when a splitter cable is used). There is no good technical reason for this limitation as far as I know: both ports are supported by software in the IKAT and it should have been easy to support the "K" protocol on both.

Hence the following requirement from the CD-i KeyControl manual:

Splitter cable is required for models CDI 210/40, CDI210/60 and CDI 470/00 and up.

(you can see the full manual here).

There is however a complication with the CD-i 450 and FW 380i players (and their variations the CD-i 550 and Magnavox CDI 5000 and CDI 9000): they do not have a back port! Instead one of the two "ports" that make up the front port is connected to the 68070 and thus bidirectional, the other is unidirectional. For some reason the secondary port was chosen to be the bidirectional one, possibly to lessen the load in the main 68070 processor in the usual case or for fail-safety reasons (if a user blows up a port it would tend to be the primary one, which would leave the 68070 port accessible for diagnostics).

The CD-i modem has to be connected to a bidirectional input port, otherwise the CD-i player would not be able to send out data. This means the back port on those players that have it, or the secondary front port on those that do not.

From the above you can deduce that this leaves players without a back port without the possibility to connect a K-mode keyboard while also using a modem! That means that the CD-i 450 and FW 380i players (and their variations) could not be used in the modem + keyboard configuration, which was unacceptable for both marketing and philosophical reasons.

So a workaround was devised. By having the CD-i KeyControl masquerade as a graphical tablet, it could be supported on the primary input port of the IKAT. The coprocessor dutifully reports the "tablet" data to the main processor, which then (using a custom driver that was included on the application disc) transforms it back into keyboard data that the CD-i application can process in the normal way.

When both a modem and a CD-i KeyControl are connected to a CD-i 450 or the FW 380i player, there is no possibility for connecting a wired pointing device (both input ports are in use). These players have no remote (infrared) capability which would make it impossible to "point and click". To remedy this, another workaround was devised. In the T-mode, the function keys F3 to F6 act as cursor keys while the F1 and F2 keys act as button and button 2, respectively

Keyboard stuff

2009-11-05T00:11:00.003+01:00

Today I finally added keyboard support to CD-i Emulator, in preparation for the upcoming v0.5.3 release.

You can now use the arrow keys and the numeric keypad for movement (the latter allows diagonal movement as well). The default movement rate is 4 (high-resolution) pixels per frame, but you can modify this with the new "keydelta" option.

You can use Space bar and both Enter keys for button 1 and the Shift and numeric keypad "+" keys for button 2; the numeric keypad "-" key acts as button 3 (both buttons simultaneously). I'm not sure yet if the numeric keypad button assignments aren't too easy to hit accidentally.

Internally this is all implemented with a full key mapping but there is currently no UI for changing the mapping.

I've tested a few games and it seems to work okay; compat testing will learn if the default keydelta is right. It might also be necessary to delay the movement actions until the keys have been pressed for a few frames; right now you need to do a VERY quick keypress to get a single delta movement. Or possibly a multistage delta array so that movement accelerates if you press the keys longer.

I've also added automatic Windows cursor hiding, the thing becomes annoying if you're not using it. You can regain the Windows cursor by pressing the Ctrl key, which now also temporarily disables the "cursor grabbing" done by CD-i Emulator (previously this could only be done by pressing Alt or F10 which also paused the emulation).

Roadmap

2009-11-03T00:48:00.003+01:00

No CD-i activity today, except reading Interactive Dreams and letting Bas know about this blog. As I expected, he announced it immediately :-)

I've also just announced this blog on the MESS CD-i WIP topic, the CD-i Emulator Support forum and added links to the CD-i Emulator site.

So for the new visitors (all four of you :-) a little roadmap of my current CD-i activities. You could say it's a To Do list except that I'm not that structured in my hobby activities.

1. CD-i Emulator programs

- I've been working on support for the Digital Video cartridge. It can currently play the Philips "bumper" movie of a few discs and play some MPEG audio; subsequent MPEG plays have a tendency to crash. This is currently for Gate Array MPEG cartridges only, but the IMPEG/VMPEG cartridges look to be highly similar. It needs to be made more reliable.

- I've also been fixing quite a bit of video and CDIC (Mini-MMC/Mono-I) bugs in conjuction with or triggered by the MESS CD-i work. This work continues.

- Support for chd (compressed hunks of data) disc images has been added using source code from MESS/MAME. Proper handling of subcode data needs to be added.

- Support for AVI movie saving has been added using source code from MESS/MAME. This needs a bit of polishing and doesn't currently save audio.

- Primitive support for input recording and playback has been added; needs a bit of polishing.

- DirectX support promises some performance improvements; I have skeleton code but it needs to be looked into.

- I've drafted a new video combining pass that should be substantially faster and should easily handle non-RGB24 display formats as used by DirectX; it needs to be fleshed out.

- Some new player types are partially emulated, in particular those using the 68341 so-called "Integrated CD-i Engine" (which is a great misnomer). These players tend to use a fourth CD interface chip that I've dubbed RCHIP.

- Needs savestate support to be added (partially drafted).

- Needs emulation bugs fixed (mostly on Mono-II and higher).

- Command-line version needs to be revived for movie rendering from recorded input.

2. CD-i File programs

The first version of the CD-i File Extractor (command-line tool) has been released as a CD-i disc image conversion tool (converts image to raw/cue and creates chd).

- Needs extension with "dir" and "chd" options to extract particular files within the CD-i disc image.

- Needs an interactive mode to browse the CD-i disc image.

- Needs header and sector map dumps to analyze CD-i realtime files.

- Needs to be extended with channel/sector selection to support data extraction.

- Needs to be extended with audio and video decoding to support data extraction.

- Needs to be extended with CD-i IFF parsing to support data extraction.

- Needs to be repackaged as CD-i File Player (graphical UI tool) for actual playback and display of audio and video data; should also support graphical sector maps and disc image browsing.

3. CD-i Base library

Announcement has been accidentally released by a recent website update. It is supposed to be a support library for CD-i application development.

- Needs to be started with basic initialization framework (has been drafted).

- Needs to incorporate the initialization code that I supplied to Charles Doty for Frog Feast, supporting access of the CSD (Configuration Status Descriptor) file from NVRAM and video and pointer hardware access.

- Needs to have disc playback code added (specs have been drafted).

- Needs to have sound playback code added (not even drafted).

- Sound mixing, blitting, sprite compilation, MPEG playback, etc.

4. CD-i Bits blog

The blog gremlins need to be kept fed :-)

5. CD-i Full Functional Specification (Green Book) document

The Green Book (often abbreviated FFGB) is the "bible" of CD-i application development; it incorporates the OS9 documentation (available on the ICDIA site) by reference.

There are now public links to the Green Book on the Internet. These are plaintext versions which are quite faithful renditions of the originals except that most of the ASCII art is messed up and there is no proper indexing and only rudimental cross referencing.

- Write a program to restore the ASCII art and convert to hyperlinked HTML with index and cross-reference links so that the digital Green Book becomes more useful.

And probably lots of things I forgot to mention...

Bugs galore

2009-11-02T00:24:00.004+01:00

Worked on three things today, mainly so I wouldn't go crazy...

Looking for RGB555 discs I found an old professional title that provides a simple testcase for the "DMA to 05140000" problem manifesting in both MESS and CD-i Emulator. I've been backtracing this through the ROMs and trying to find out how to solve it, but no luck so far.

When I got nearly crazy from staring at the disassembly listings I'd switch to fixing another bug: CD-i Link no longer works with CD-i Emulator, and there is someone urgently needing this. I've made some progress here (it downloads the stub okay, but still cannot upload) but it's not finished.

To escape from all the frustration I also did some new development: I extended the disc path table and directory sector parsing in preparation for more CD-i File functionality (specifically, the "dir" and "chdir" options). But here I also hit weird bugs in memory allocation.

It just wasn't my day, I guess...

Some fixes

2009-11-01T00:57:00.002+01:00

Triggered by messages on the MESS CD-i WIP topic, I fixed a few CD-i Emulator bugs today.

I implemented the CDIC "seek" command which for some reason I never got around to; it allows International Tennis Open and Flashback to progress past their intros (they would hang otherwise). The SS_Seek API which is what triggers this command is just a (possible) performance optimization and very few CD-i titles use it (every read operation has an implied disc seek anyway).

Also fixed the CDIC sector dispatching logic so that it only handles audio sectors specially during a mode2 read (it could otherwise cause bugs with mode1 reads and seeks).

Fixed a region control bug (the region flags values were set from the lowest coordinate bit instead of the proper operation bit); this surfaced when trying to fix Zenith (it had no visual effect there but did show up in the video traces).

And finally todays biggie: fixed a transparancy control bug that caused planes to never go transparent, even when they should, as soon as any region flag got set. This was caused by a very subtle macro definition bug that caused the transparency control masks to get bad values for transparency cases i > 4, which are the cases where one or both region flags are set.

I'm frankly amazed the last one slipped through for so long; it even had a visible effect in the intro sequence of The Apprentice (when the little horse riding knight rides accross the screen, you could briefly see it in black in the left and right margins, whereas it should have been transparent, and now is). And I actually knew this was wrong but never got around to figuring it out...

But with the above changes, Zenith finally renders fully correct, Flashback and International Tennis Open boot okay (they have sound and gameplay issues, though), and The Apprentice also renders fully correct (as far as I know).

I have to figure out a way to get all these fixes out there...

Video mode refactoring

2009-10-30T00:21:00.002+01:00

In search of something small, I refactored the video mode calculation today, in preparation for a more complete VSC (SCC66470) emulation and CD-i File video decoding. This gets a bit technical...

It turns out that the VSC chip supports much more video modes than the "standard" CD-i ones supported by the VDSC, and in the interest of accuracy I will eventually want to emulate that. These modes include widths of 224, 256 and 320 pixels in addition to the standard ones of 360 (NTSC) and 384 (PAL) and an height of 210 lines in addition to the standard ones of 240 (NTSC) and 280 (PAL). Charles Doty accidentally hit one of these resolutions during his Frog Feast development and we were both extremely baffled until we figured out that a stray bit had been set in his "set video mode" instruction. The VSC chip is only used in Mini- and Maxi-MMC players which means these resolutions exist only on the CD-i 180, CD-i 220 F1 and CD-i 60x players.

I also wanted to generalize the mode stuff in preparation for the decoding of arbitrarily sized images as needed by future versions of CD-i File.

So instead of four bits (CF,ST,FD,SM) taken directly from the VDSC registers DCR1 and CSR1 the CCdiVideo::SetMode function now takes an SCdiVideoMode structure containing all the parameters previously derived from these four bits: field and line count, starting video line and total number of video lines, video pixels and decoded pixels. I've also added pixel resolution which previously was a separate parameter that chooses between decoding to normal resolution (384) or double resolution (768).

The calculations of the parameters from the hardware register bits have been moved into the CCdiVdsc and CCdiVsc classes (the VDSC and VSC emulations) so that the VSC emulation can in the future extend the calculation to include the non-standard video modes.

Blog start

2009-10-29T00:47:00.004+01:00

Well, I've finally succumbed to this blog thing...

I will try to make regular (but not daily) posts here about whatever I'm doing related to CD-i. That of course includes work on CD-i Emulator, but also on tools (CD-i File comes to mind), interesting information I found out, etc.

If wou want more CD-i info, I can recommend the CD-i Forum at http://www.cdinteractive.co.uk/forum/ and the CD-i WIP topic on the MESS forum at http://www.bannister.org/forums/ubbthreads.php?ubb=showflat&Number=54354.

Today I added CD-i File stuff to the Downloads and What's New sections of the CD-i Emulator site, didn't have much time for other things, as I also had to install my wife's new laptop a bit more. And I was home late also.

The last weeks have been fairly hectic, I blame MESS for it :-) The CD-i interest seems to have risen sharply, and I've actually fixed a few CD-i Emulator bugs and added a few features to CD-i Emulator as the result of it.

And of course I polished up CD-i File Extractor and released its first version. This is intended as the command-line version of the much-requested CD-i File Player. Whereas the Extractor will only be able to extract and convert audio and video (images), using the command line, the Player will be able to actually play the audio and show the video. But that's still a while to go...