CD-i Emulator 0.6.0 and beyond

Today I have released the seventh and probably last beta release of CD-i Emulator version 0.5.3, which is now in maintenance mode as all development has shifted to version 0.6.0.

Even though it is still called beta, this can be considered a "stable" release. It is only a very small change from the previous beta release of this version which has been out since April 2023 with no major issues reported.

The same cannot be said for the current state of version 0.6.0; many structural changes have been made and it is more than likely that this will result in regression issues relative to version 0.5.3-beta7.

It will probably take some time to get these issues ironed out and in the meantime today’s release can provide the fallback position.

I will try very hard to keep all the configuration files (especially cditypes.rul and the files in the sys directory) compatible with both versions.

I do not want to repeat the 0.5.3 "many betas" pattern with the new version; after the needed stabilization work a formal non-beta release should follow relatively quickly.

Highlights

As hinted above, version 0.6.0 will be a massive update, basically all the work done after the version 0.5.3-beta5 release in March 2019 (which was not intended as a public beta but leaked anyway; it has since been made available on my website).

Here are some highlights of the coming version 0.6.0:

- The three-minute time limit between player resets is gone. Therefore, there are no longer separate "limited" and "unlimited" editions. Consequently, it is no longer needed to activate the program and it will be available free of charge.

- Archived disc image files (zip, 7zip, gzip, bzip2, etc) can now be emulated directly; they will be unarchived to a temporary directory as needed. This features utilizes libarchive for reading the archive files.

- Infrastructure has been put in place for CD-i player and CD-i disc "art" images to be shown at startup, based on hashing the disc label sector. The initial release will ship with a basic collection of art files, in a future version these art images could be downloaded automatically. I am working with “The world of CD-i” to leverage their comprehensive collection of cover art images. This feature uses zlib, jpeglib, giflib and the MAME png code to support various image formats.

- The Philips CD-i 604, Philips FW380i, Grundig CD-i 110E and Bang & Olufsen Beocenter AV5 CD-i player systems are now supported (with some limitations). These are all variations of existing Philips players so the changes for this were relatively small.

- The Philips Model BO Videosynthesizer Prototype system (boproto) is now supported as an OS-9 system; ROM and disc image files for it are already publicly available from Jorne / MrMii6's Coffee Fueled Retro website at https://www.cfretro.net/philips-bo-prototype/. This needed support for some new chips and a SCSI device hack but was very fun to do.

- There is a new ROM version identification system based on ROM tags as described in my previous CD-i ROMs blog post. This replaces the error-prone minor version detection system based on module versions used before and also adds some validation to the detection process.

- There is a new CD-i Type program that can produce ROM information directly from the ROM files that previously could only be produced by CD-i Link. This program is also capable of collating a collection of such ROM information files into various forms, including an HTML table format that will be made accessible via a new “CD-i ROMs” page at the CD-i Emulator website https://www.cdiemu.org/.

Note: Only ROM information files will be made available, not the actual ROM files themselves, comparable to e.g. the redump website http://redump.org/ for disc images.

- The CD-i Type program can also produce a "datfile" for CD-i ROMs called cdiroms.dat which will also be made available via the website. With the availability of this file, it will become possible to use ROM managers such as ClrMamePro, RomCenter and RomVault to check CD-i ROM file collections.

- The above ROM managers will put ROM files inside folders or archives, which is now also supported by CD-i Emulator inside the rom directory.

- ADPCM audio decoding has been improved by keeping intermediate results with bigger precision and changing the rounding. Many technical details can be found on Discord around this message: https://discord.com/channels/675054061665386544/676505089103495187/1094786111408189502

- All sources are now maintained in a git repository, making for easier maintenance, and paving the way for a future source release on GitHub.

- All documentation files now have proper file extensions (e.g. RELNOTES.txt), making it easier for non-technical users to find and open them.

Further work

Some more work that might be done in later 0.6.x releases, not in any particular order or priority:

- Modification of the Help > Report menu option to automatically create a GitHub issue.

- Support for CUE files which would allow disc images to be split into separate files (typically one per track) as often done by disc dumping programs or archive sites. Each track file could have a different format, for example MODE2/2352 for the CD-i track and WAV or MP3 for audio tracks.

- Support for CHD version 5 files (the current version); currently CD-i Emulator only supports up to version 4 CHD files.

- Support for ROM sets (e.g. for specific player configurations or for testing) in subdirectories based on the existing folder and archive support.

- Graphical UI version of CD-i Link to make CD-i ROM downloading easier for users unfamiliar with the command line.

- Graphical UI support for more CD-i Emulator settings, esp. the CD-i player model and/or ROMs to use in case you have more than one, again mainly intended for users unfamiliar with the command line.

- Finishing support for CD-i 60x floppy drives; possibly also adding support for CD-i 180 and CD-i 615 floppy drives (they use different controllers).

- Adding disc and audio emulation for the Sony and Kyocera CD-i players, which means handling more variants of the RCHIP disc controller already supported for the Philips CD-i 370 and Goldstar GPI-1200M players.

- Adding proper support for the RTC functions of 32K NVRAM chips as used in more high-end players; currently the date and time is not correctly read from these chips.

- Support for loading and saving of NVRAM files directly from the hard disc (this requires partial high-level emulation of the /nvr driver).

- Completion of OSD support for the B&O Beocenter AV5 player. I have one of those now, so that should make it somewhat easier.

- More support for the various I2m boards, right now they do not even boot.

- Support for save/restore of player state snapshots.

- Creation of a pure command-line version of CD-i Emulator, useful for regression tests and rendering high-quality videos of sessions recorded with the existing input recording/playback functionality. You would not be able to directly control the CD-i player, only control emulation from the debug command prompt.

- Creation of an SDL version of CD-i Emulator, paving the way for future ports to systems other than Windows. This version would not have a GUI to control emulation, you would again need to use the debug interface for that. For more information about SDL, see https://www.libsdl.org/.

- Creation of basic ports to systems other than Windows using SDL. Envisioned target systems include Linux, MacOS, iOS and Android. Especially the latter two present challenges because they do not have an easy way to create a debug interface except via the network.

Basic research and sometimes partial implementation have been done for all of the above.

As you can infer from the above, the primary goal of the version 0.6.x releases is not to improve the core emulation performance or quality, though some enhancements and bugfixes have been and will continue to be made and support for some new supported exotic players might also be added.

Rather, the primary goal will be to improve the user interface and experience, both for emulation (CD-i Emulator) and ROM extraction (CD-i Link) and collation (CD-i Type).

Roadmap

The primary goal of the 0.5.x releases was always to get a basic CD-i emulator working, which I feel has been more then accomplished by now.

In my mind the primary goals for the coming minor releases are:

0.6.x – Improve user interface and experience

0.7.x – Fix Digital Video cartridge emulation issues

0.8.x – Fix as many other emulation issues as possible

0.9.x – Do whatever still needs to be done for a...

1.0.x – Reasonably complete CD-i Emulator!

When the 1.0.0 milestone is reached, I will do a source release on GitHub. It would be nice if that could be done some time in 2025 because that would be twenty years after the first public release in 2005, a nice round number.

Future work

Some things that I would still like to do that might go after that, but also (partially) before if I get the itch:

- Complete ROM-less CD-i emulation, requiring high-level emulation of most of the CD-RTOS operating system, in effect adding a high-level "CD-RTOS coprocessor" to the emulated system.

- Better support for title developers and reverse engineers. A more graphical UI debugger, video DCP and pixel inspection, data tracing, etc.

- Add chip access collation to support better documentation of SLAVE/IKAT command variation between versions.

- Validate processor emulation by producing instruction traces that can be downloaded into and validated on actual hardware, including timing information (cycle counts)

- Enhance CD-i Emulator for use as a hybrid OS-9 development system like “os9exec”, useful for running OS-9 binaries like the C compiler and other tools.

- Finish and release CD-i file conversion programs; right now they are a hodge-podge of just what I personally needed and not functionally or even technically complete in any way.

- Create a CD-i file viewer program. This should allow viewing of the contents of any CD-i related file, from disc images down to disassembly. In many ways this would be a "lite" version of an emulator because it would have all the format decoding (but no exact chip emulation).

- Enhance the existing CD-i File program to support extracting the contents of any CD-i related file, in essence the command-line version of the CD-i file viewer.

- Create a CD-i disc image building program, ideally based on title development support (e.g. partial disc images) integrated into CD-i Emulator.

- Create a CD-i disc image editing program based on both of the above.

And whatever else may come up!

Discussion

If you would like to reach out to discuss features or their priority, the #emulation channel of the Philips CD-i Discord server at https://discord.gg/TKPejTfw6D is the right place.

You can also use the CD-i Emulator issue tracker on GitHub at https://github.com/cdifan/cdiemu/issues to suggest features.

Announcing CD-i Emulator version 0.5.3-beta7 for Windows

A new beta release of CD-i Emulator is now available from the Downloads section of the CD-i Emulator website at https://www.cdiemu.org/downloads/.

Release notes can be found in the RELNOTES file included in the download; a summary as well as the full version is available from the Release Notes section at https://www.cdiemu.org/relnotes/.

This version of CD-i Emulator is the seventh and probably last beta release for version 0.5.3, which is now in maintenance mode as all development has shifted to version 0.6.0 for which a first public beta release is expected soon.

Relative to the previous version 0.5.3-beta6 release only a few bugfixes and small enhancements have been made; see the Release Notes for details.

Note to would-be crackers of this version (you know who you are):

I really would prefer it if you didn't. There is really no need!

As with all previous beta versions, using a Mono-I CD-i system ROM (also called BIOS) will give you unlimited emulation functionality; this is the same ROM file needed for the MESS/MAME CD-i driver.

Saving the contents of the CD-i NVRAM to a file is still supported.

To obtain unlimited emulation functionality with a different CD-i system ROM file, please buy the still available unlimited edition of version 0.5.2. Activating this on your PC will also unlock the unlimited emulation functionality of this beta version.

If you have bug reports or feature requests please use the GitHub issue tracker at https://github.com/cdifan/cdiemu/issues; as of right now no CD-i Emulator source code is on GitHub but eventually there will be.

For comments, questions and discussion you can use the #emulation subforum of the Philips CD-i Community Discord server reachable at https://discord.gg/TKPejTfw6D. I am known there as CD-i Fan (cdifan).

You can also reach me via e-mail using cdifan@gmail.com, but this will not involve the larger CD-i community and is hence discouraged.

If you like my work and/or would like to support future development, you can also send me a donation. You can send PayPal payments to payment@cdiemu.org or use one of the following websites:

https://www.buymeacoffee.com/cdifan

https://ko-fi.com/cdifan

As of right now you can also still buy the unlimited edition of version 0.5.2, but in the future this will no longer be possible. Go to the Payments section of the website and get an original serialized version while you still can!

Have fun emulating your favorite CD-i titles!

Announcing CD-i Emulator version 0.5.3-beta6 for Windows

A new beta release of CD-i Emulator is now available from the Downloads section of the CD-i Emulator website at https://www.cdiemu.org/downloads/.

Release notes can be found in the RELNOTES file; a summary is available from the Release Notes section at https://www.cdiemu.org/relnotes/.

This is the sixth beta release for version 0.5.3. It is hugely out of date with the current development version 0.6.0; the only reason for its release is to get a working version available again.

The only major change is that emulation speed throttling has been fixed for modern machines so games running at unplayable speeds should be a thing of the past.

Note to would-be crackers of this version (you know who you are):

I really would prefer it if you didn't. There is really no need!

As with all previous beta versions, using a Mono-I CD-i system ROM (also called BIOS) will give you unlimited emulation functionality; this is the same ROM file needed for the MESS/MAME CD-i driver.

This beta version also has the NVRAM save functionality enabled, and before the built-in beta date limit expires a new beta release for version 0.6.0 should be available.

To obtain unlimited emulation functionality with a different CD-i system ROM file, please buy the unlimited edition of version 0.5.2 which is still available at https://www.cdiemu.org/payments/.

Activating the unlimited edition on your machine will also unlock the unlimited emulation functionality of this beta version.

If you like my work and/or would like to support future development, you can also send me a donation. You can send PayPal payments to payment@cdiemu.org or use one of the following websites:

https://www.buymeacoffee.com/cdifan

https://ko-fi.com/cdifan

Have fun emulating your favorite CD-i titles!

CD-i ROMs

Most currently existing CD-i emulators use so-called low-level emulation for the system software. This means that to perform their emulation function they need copies of this software.

The CD-i system software is typically stored in read-only memory (ROM) chips inside the CD-i player; the only known exceptions to this are the PCI/ISA CD-i boards from I2m / Philips that have their system software loaded from the hard disk of the host system.

Philips CD-i 210 F3 ROM 1.1

Philips CD-i 210 F3 ROM 1.5

B&O Beovision AV5 ROM 1.9

ROM types and sizes

The typical CD-i player contains 512KB of system ROM which contains the CD-RTOS operating system based on Microware OS-9/68000 version 2.4. This system ROM contains the core system software including the OS-9 kernel, CD‑RTOS file managers, drivers for the specific CD-i player hardware, as well as system startup programs. The core system software typically takes less than half of the ROM, the rest is taken up by code and resources (e.g., images, fonts) for the player shell and usually some utility modules. On some of the later Philips players the system ROM is so full that niceties such as the CD+Graphics decoder “cdgr” or the Service Shell “sv” have been omitted.

Notable exceptions to this are the Sony IVO players which have 1MB of system ROM, although only a little over 835KB of that is used. The core system software takes less then 200KB of this, the remainder of the ROM is again filled with code and resources for the player shell.

Most CD-i Digital Video Cartridges (DVCs) contain 128KB of additional ROM with CD-RTOS file managers and drivers to support the MPEG playback functions. Little over half of this ROM is typically used, it varies somewhat with the hardware generation. Some cartridge versions also contain some utility modules as well.

Professional or authoring players typically have other additional ROMs, usually located on some kind of (removable) extension board. For example, the CD-i 182 unit contains an additional 64KB of ROM with drivers for the floppy drives and other hardware in this unit. Extension boards for this unit have sockets for additional ROM chips but I have never seen them filled.

In contrast to this, the CD-i 605 extension board contains 512KB of additional ROM to support the Ethernet and SCSI controllers and other hardware on the board, and it is mostly filled.

The CD-i 615 and CD-i 670 use another extension board which also contains 512KB of ROM with support modules for their additional hardware, which in this case includes IMPEG digital video.

Dumping ROMs

A low-level CD-i emulator needs access to the contents of these ROMs, which must be obtained from the hardware using a process called “dumping”.

All CD-i system ROMs are standard ROM chips that can be read out by a ROM dumping device. These devices can be custom made hardware, but an off-the-shelf EPROM programmer device can also be used. For this to happen the ROM chip typically needs to be removed from the CD-i player, which means opening the player and often requires desoldering the ROM chip.

However, the contents of these ROMs are also accessible from inside the CD-i player (that is after all their primary function), so they can also be dumped using software. This is typically done over the serial port as most consumer players have no other easily accessible output port (theoretically the ROM contents could also be dumped over the audio or video outputs but that has so far not been necessary).

Early 2005 I wrote a program called “CD-i Link” that uses the built-in download function of most CD-i players to download a small “CD-i Stub” program into the CD-i player so that memory can be scanned for ROMs and their contents can be dumped via the serial port. If the player has no such download function, a “CD-i stub” disc can be used to start the stub program.

The system ROMs of all CD-i players encountered so far can be dumped this way, with one exception: the Sony IVO-V10/V11 players. These players do not have a regular serial port accessible from the outside. Fortunately, they do have a service port that supports serial communication although it needs to be accessed directly from the system board which in this case does mean opening the player and doing a little soldering.

Sony IVO-V11 player with service switch and serial cable

In addition to the CD-i system ROMs, which are accessible from the main 68000 CPU, CD-i players and digital video cartridges often contain microcontroller and/or signal processing chips that have their own ROM. Dumping this can be complex, in some cases the chips need to be “decapped” or a special test or manufacturing mode needs to be used in order to read out the ROM contents.

Fortunately, CD-i Emulator does not use low-level emulation for these chips and hence does not need dumps of their ROMs. Other emulators may need these dumps now or in the future, but it is not relevant to the rest of this article and I will ignore such ROMs below.

ROM versions

There are several brands of CD-i players, and typically multiple player models and versions within each brand. Even nominally identical players will sometimes contain different ROM versions.

The intended scenario for using CD‑i Emulator is that a user dumps the ROMs from his own CD‑i player using CD‑i Link and then proceeds to use these ROM files with the emulator. For this reason, it is important that CD‑i Emulator supports as many ROM versions are practical.

In practice, CD‑i ROM files can also be found on the Internet and many CD‑i Emulator users obtain them this way. These files are however still copyrighted, and their distribution is not legally allowed.

ROM detection

So how does any emulator deal with this bewildering variety of ROM versions? It will need to detect what hardware is expected by a particular ROM, assuming the emulator supports that hardware.

CD-i Emulator uses rule-based detection to determine the CD-i player type and the expected hardware based on the contents of the ROMs. The detection is driven by a rules file with conditions on the OS-9 modules contained in the ROMs. Doing it this way means that the detection is generally tolerant of ROM variations, which was important early on because it was unclear what kinds of ROMs were out there.

For an explanation and the current contents of the rules file used by CD-i Link and CD-i Emulator see the CD-i Types section of my website at www.cdiemu.org.

For a different approach, the MAME emulator uses detection based on hashes (checksums) of the ROM files, which means that it only works with specific ROMs (actually, MAME does not so much “detect” ROM versions as well as “verifies” them, its source code contains lists of ROM hashes that are accepted by specific emulation drivers).

Refining ROM detection

Some CD-i users have been collecting CD-i ROMs for many years now. It has become apparent that many more versions of CD-i system ROMs exist than initially expected. Therefore a better way of cataloguing ROMs instead of just by main player type is needed. Also, as more ROMs became available holes were uncovered in the detection rules used by CD-i Emulator. These holes usually resulted in mis-detecting the player version, although the emulation would often still work as the hardware variations were minor or non-existent.

In January of this year, I made an attempt to fix some mis-detection issues based on information supplied by rosewood of retrostuff.org and I also tried refining the detection rules to differentiate between ROM versions. The latter quickly became unwieldy, but it turned out that there is a much better way.

ROM tags

The low-level tests of many players display ROM ID and ROM release information as well as ROM checksums, and newer player shells display some of this information in the copyright screen as well. Could this information be used for ROM detection?

Pointed in the right direction by rosewood, I found out that this “ROM tag” information is stored in a consistent, documented, way across all Philips and Philips-based CD-i ROMs which makes it easy to extract that information for detection purposes.

From the CD-i 220 phase 1 and CD-i 350/360 service manuals (these are both Mini-MMC players):

 

Information for the Mono-I players can be found in the CD-i 210 phase 1 and CD-i 220 phase 2 service manuals:

      Note: The information about the last 1024 bytes
      is incorrect for Mono players!

This information was dropped from later service manuals, but it turns out to be valid (with suitable modifications) for all Philips and Philips-derived CD-i system ROMs, included the additional ROMs used in extension boards and digital video cartridges.

The ROM ID and ROM release bytes are stored in BCD (binary-coded-decimal) with an implied decimal point. Some low-level tests render these bytes directly (e.g., 12), others include the decimal point (e.g., 1.2).

So far, the only CD-i system ROMs found to not contain ROM tags are those from the Philips CD-i 180, Sony IVO and Kyocera Pro players.

Copyright screen

CD-i players with the 2^nd Philips player shell display the ROM ID and release numbers in the copyright screen in the format

1AA5BB-2CC4DD-3EE

where AA is the system ROM ID and BB is the system ROM release. The other numbers are not important for ROM version detection but are interesting nonetheless, CC is the SERVO chip version (Mini-MMC, Mono-I, and Mono-II players) or the extension ROM release (CD-i 615 / 670 players) or 00 (other players), DD is the SLAVE/IKAT version, and EE is the player shell “ps” module edition.

Philips CD-i 210 F3 copyright screen for ROM 1.5

Some non-Philips CD-i player shells display some of this information in a different format, e.g., the Digital Video System DVE-100 player shell “vs” uses the format

1AA2BB-3CC4DD-5EE

where AA is again the system ROM ID and BB is the system ROM release. The CC and DD numbers are hardware related, and EE is the player shell “vs” module edition.

ROM tag complications

As described by the service manuals, the ROM tag information is always stored in the last four visible bytes of each CD-i ROM chip. But there are some complications not described in those manuals.

The Maxi-MMC CD-i players (CD-i 601) and the analog digital video boards (22ER9141) use two 8-bit wide ROM chips to provide the main CPU with 16-bit wide ROM data. In the case of such a ROM chip pair, each of these ROM chips contains its own ROM tag information. All such pairs encountered so far have the same ROM release byte with the ROM ID bytes differing by one (e.g., 10:11 or 20:21). In these cases, the CPU will see two interleaved ROM tags in the last eight visible bytes of the ROM space.

Underdumping

On the Mini-MMC and Maxi-MMC boards, the last 1024 bytes of the system ROM are not accessible by the CPU; the video registers of the VSC video chips are located there as described in the service manual. On these boards, the ROM tag is in the last four bytes just before the 511KB mark, and the rest of each ROM is considered empty (full of FF bytes) for checksum calculation purposes.

As only 511KB is available from within the CD-i player, the current CD-i Link program will “underdump” these ROMs (i.e., it will omit the last 1024 bytes).

ROM ID and release values

From the information gathered so far, it appears that Philips has assigned the ROM ID bytes in such a way that they almost exactly correspond to the player model / phase combinations. Within a single combination, there can be multiple ROM releases, but these are almost always of the form 1n (e.g. 1.n).

For digital video cartridges, the ROM ID bytes are always 10:11 (22ER9141) or 10 (22ER9956), and the ROM release bytes indicate the cartridge type as described in the Interactive Engineer 96/05 article Which DV cartridge is in a 605? that can be found here: http://www.icdia.co.uk/iengineer/.

Detection using ROM tags

Over the last two months I have extended the rule-based CD-i detection logic used by both CD‑i Link and CD‑i Emulator to use the ROM tag information for player type and ROM version detection. There appears to be some logic to the ROM ID values but it is not exact, so the ID-based detection is still rule-based although the ROM version detection is totally automated.

To reflect the ID and release information, a new naming convention for CD-i ROMs has been defined, and together with rosewood I have been gathering information for a “ROM catalogue” (see the section Datfiles below).

As an example of the new detection rules, the following is now included in the rules file:

cdi220a??.rom: Philips CD-i 220 F1 system ROM ?.?
      {511K} #$26 ::cdi220a.rom

cdi220a.rom: Philips CD-i 220 F1 system ROM
      csd_220 minimmc.brd

The first rule says that any ROM with a ROM tag at 511KB that has a ROM ID byte with value hexadecimal 26 is a CD-i 220 F1 system ROM, with the specific version in the ROM release byte. The ::cdi220a.rom part means that the ROM must conform to the generic cdi220a.rom detection rule that is specified just below it, which uses the existing module-based detection syntax.

The conformation requirement ensures that both detection types remain in sync.

ROM file names

I have extended the previously somewhat ad-hoc ROM file names to include the ROM tag information.

Special logic included in the detection engine will replace the question marks in the ROM name cdi220a??.rom and its description above with the ROM release byte, so that a ROM with ROM ID 26 and ROM release 11 will be recognized as cdi220a11.rom with description “Philips CD-i 220 F1 system ROM 1.1”.

In addition to these “short names” a “full name” has also been defined that includes the ROM ID and ROM checksum in the filename. The above-mentioned CD-i 220 F1 ROM happens to have checksum E620, which means that its full name is cdi220a11-26-E620.rom.

For cases where two 8-bit ROM chips are used together, both interleaved ROM tags are added to the full name. E.g., the 22ER9141 F2 VMPEG digital video cartridge has ROM IDs 10 and 11; version 4.1 of this interleaved ROM has the full name vmpega41-10-11-FFD9-4BA9.rom.

Full ROM information also includes the CRC32, MD5 and SHA-1 hashes of the ROM contents (for a 511KB ROM file, the additional 1024 empty FF bytes are included in these hashes even though the actual ROM file does not have those bytes). These three values will now also be displayed by CD-i Link.

CD-i Type

A new CD-i Type command-line program has been created to manage collected ROM information. This program can be used to list ROM information just like CD-i Link does, but it can do so without the hassle of dumping the ROM.

Using CD-i Type, the example ROMs mentioned above would be listed as:

    File             Addr    Size    Type             Description

------------------ -------- ------ ------------------ ------------

cdi220a.rom        00000000   511K cdi220a11.rom      Philips CD-i 220 F1 system ROM 1.1

cdi220a.rom        00000000   511K cdi220a.mdl        Philips CD-i 220 F1 player

cdi220a.rom        00000000   511K minimmc.brd        Mini-MMC board

cdi220a.rom        00000000   511K cdi220a11.tag      {511K} ID: 26 Rel: 11 Sum: E620

cdi220a.rom        00000000   512K cdi220a11.crc      CRC: FC623645

cdi220a.rom        00000000   512K cdi220a11.md5      MD5: D669D26A9E7228DF18E84E380F6CA6AF

cdi220a.rom        00000000   512K cdi220a11.sha1     SHA1: 7DC5D62CE1686E45F2B68AA3943FE50BC47C71A7

------------------ -------- ------ ------------------ ------------

vmpega.rom         00000000   128K vmpega41.rom       Philips VMPEG digital video cartridge ROM 4.1

vmpega.rom         00000000   128K vmpeg.dvc          Philips VMPEG digital video cartridge

vmpega.rom         00000000   128K vmpega41.tag       {128K} ID: 10:11 Rel: 41 Sum: FFD9:4BA9

vmpega.rom         00000000   128K vmpega41.crc       CRC: 3685382E

vmpega.rom         00000000   128K vmpega41.md5       MD5: 9694C466F9B65C1990A81B7A6280546B

vmpega.rom         00000000   128K vmpega41.sha1      SHA1: 1D9C040B4FE974F5EB600C6F80797F40B45EFCF1

CD-i Type is also capable of taking this information from informational text files produced by the new CD-i Link as well as itself, so that it is not necessary to ship around ROM files to catalogue them.

The individual 8-bit ROM files from a ROM pair are not recognizable by OS-9 module rules because they contain only half of each module. Instead, CD-i Type uses the rules file to recognize them by their ROM tags and/or checksums.

Overdumping

Although CD-i digital video cartridges contain 128KB of ROM, the sockets containing the ROM chips appear to have an unused address line. This is probably the reason that the ROM contents appear duplicated in the CD-i player address space, causing “overdumping” by the current version of CD-i Link. The resulting ROM files are 256KB even though the actual ROM contents are only half that.

The new CD-i Link will avoid overdumping by detecting the duplication; as a bonus this will make the dump operation faster.

Fixing ROM dumps

There are some issues with existing CD-i Link ROM dumps (overdumping, underdumping) and there can also be issues when ROMs are dumped with a ROM dump device (pair splitting, byte swapping). The CD‑i Type program can “fix up” these issues when instructed to do so with the -fixuproms option (it will also rename ROM files in this case).

To correct overdumped digital video cartridge ROM files, the CD-i Type program discards the duplicate data.

To correct underdumped 511KB ROM files, the CD-i Type program adds the missing 1024 empty FF bytes.

To compensate for pair splitting, the CD-i Type program interleaves the separate ROM files into the proper interleaved file usable by CD-i emulators. For completeness, the reverse operation of splitting interleaved ROM files is also available with the -splitroms option.

Finally, to correct for byte swapping, the CD-i Type program swaps the bytes in ROM files if it detects them being swapped as indicated by a swapped ROM tag.

Untagged ROMs

Untagged ROM versions can still be identified by checksums although there are no ROM IDs or clearly discernible ROM release numbers. The actual ROM chips in fact often include that checksum on the ROM labels. For this reason, the full name for these ROMS includes the checksum, for example pro1000s-986D.rom for the Kyocera Pro-1000S CD-i player.

Kyocera Pro-1000S ROM

Datfiles 

The final function of the CD-i Type program is cataloguing a set of ROM (information) files in the form of an XML “datfile” bundling all the information in a single file.

With the availability of such a “datfile” for CD-i ROMs it will become possible to use ROM managers such as ClrMamePro, RomCenter and RomVault to check CD-i ROM collections.

I worked with rosewood in figuring out the best way to format the CD-i ROM information into datfile entries for general use without losing information; the new “short name” and “long name” formats were tuned for this.

Typical resulting datfile entries for the above example ROMs would be:

       <game name="cdi220a11" board="Mini-MMC">
              <description>CD-i 220 F1 system ROM 1.1</description>
              <manufacturer>Philips</manufacturer>
              <rom name="cdi220a11-26-E620.rom" size="524288"

                   crc="FC623645"

                   md5="D669D26A9E7228DF18E84E380F6CA6AF"
                   sha1="7DC5D62CE1686E45F2B68AA3943FE50BC47C71A7"/>
       </game>

and

       <game name="vmpega41" board="VMPEG">
              <description>VMPEG digital video cartridge ROM 4.1</description>
              <manufacturer>Philips</manufacturer>
              <rom name="vmpega41-10-11-FFD9-4BA9.rom" size="131072"

                   crc="3685382E"
                   md5="9694C466F9B65C1990A81B7A6280546B"
                   sha1="1D9C040B4FE974F5EB600C6F80797F40B45EFCF1"/>
       </game>

With this input, the above ROM managers will put each ROM file inside a folder or archive using the short name while the ROM file inside will use the full name.

For example, the following ZIP files would be produced using maximum compression:

Archive:  cdi220a11.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
  524288  Defl:X   230291  56% 2021/03/14 00:04 fc623645  cdi220a11-26-E620.rom
 
Archive:  vmpega41.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
  131072  Defl:X    42344  68% 2021/03/14 00:04 3685382e  vmpega41-10-11-FFD9-4BA9.rom

Notice that the CRC-32 of each ROM file inside the ZIP archive matches the CRC-32 values listed above; this is by design as the exact same algorithm is used.

Given the above information, the ClrMamePro ROM manager is in some cases also capable of “fixing up” the issues with ROM dumps mentioned above. In an ideal world, it would not be necessary to use the command-line CD-i Type program for this but that is not currently the case.

To avoid confusion, both CD-i Type and CD-i Emulator will be extended to support this new folder/archive way of storing ROM files inside the rom folder.

Release timing

The new and updated CD-i programs will be released together with the new CD-i ROMs datfile cdiroms.dat; at that time, the CD-i Emulator Home website at http://www.cdiemu.org will also be extended with a CD-i ROMs page that contains the collected information in browsable form.

The actual ROM files themselves are copyrighted and will not be available.

Why is CD-i emulation hard, or isn't it?

Recently, it has been argued on Reddit, Twitter and elsewhere that CD-i emulation in MAME is difficult and why this is unlikely to improve in the foreseeable future.

The main reasons given (paraphrased) are that

a)       there are different hardware implementations of the CD-i platform, and

b)      applications do not directly interact with that hardware but are mediated by the BIOS, therefore

c)       hardware access patterns look identical across the entire library of applications.

The conclusion from this is that

d)      an emulator developer cannot “more clearly infer certain subtleties of the hardware functionality”.

Note: This article uses the term “BIOS” for what I have called “CD-i system ROM” in the past. The latter is more descriptive, but the former appears to be what many other emulators use.

Some parts of the CD-i hardware are custom chips for which not much or only very skimpy documentation is available, some reverse engineering is therefore required. But is it really hard?

One of the ways to reverse engineer hardware is to look at the software that uses it. There are various ways of doing that, the classic MAME way is to look at the access patterns and the code immediately surrounding that. If you have a sufficient variety of accessing code, you will ultimately develop a good (emulated) approximation of the original hardware, by a sometimes lengthy process of trial and error. If this is your goal, the fact that all accesses go through the BIOS can indeed be a problem.

But is having such a good approximation actually necessary to have a working emulator? If every application can directly access the hardware, as typical for most consoles, certainly. But this is not the case for CD-i!

The fact that all CD-i specific hardware access is through the BIOS is actually an opportunity when viewed in the right light. It is not necessary to develop a generic “good” approximation of the hardware, the only thing that is needed is an approximation “good enough” for the BIOS!

Now, this goes against the stated goals of the MAME project which is actually *not* to produce a working emulator but to document the hardware legacy. From mamedev.org, right at the top:

MAME’s purpose is to preserve decades of software history […] by documenting the hardware and how it functions. The source code to MAME serves as this documentation. The fact that the software is usable serves primarily to validate the accuracy of the documentation.

Seen in this light, so-called “high-level emulation” (emulating the visible effects of hardware instead of the actual inner workings) is akin to blasphemy: it goes against the entire philosophy of MAME.

My own goal in developing CD-i Emulator was never to “document the hardware”, though. It was to have a good enough emulation to actually play CD-i discs. For this, it is technically not even necessary to emulate *any* existing CD-i specific hardware, as CD-i is basically a software API standard. If you have a correct implementation of that standard you are done (there are of course some complications).

It is interesting to note that CD-Ice, the first ever (public) CD-i emulator, did exactly this. The author used his knowledge of the CD-i specification (Green Book) to implement the API from scratch using his source code for “Rise of the Robots” as a working sample. Although often treated as a single-game emulator, this is not in fact correct as his emulator ran many other games as well.

The Green Book states that CD-i players run the CD-RTOS operating system using an “68000-family processor”. This is therefore the minimum amount of *actual* hardware that needs emulation, and it’s exactly what CD-Ice did. All other parts were implemented based on the specification and not on actually existing hardware.

Going the full HLE route like CD-Ice did has its own problems of course, mostly because CD‑RTOS incorporates significant parts of the OS‑9/68000 operating system and a full emulator would therefore need to implement that as well. Some features such as multitasking and interrupts can be particularly hard to tackle with a full HLE strategy. Using full HLE also reintroduces the “many access patterns” issue but on a different level. Since you are emulating the (software) interface against which developers work directly, you can only get it “good enough” by throwing lots of applications against it.

For all these reasons, I chose a different approach for CD-i Emulator. Knowing that fully implementing CD-RTOS would be hard, I wanted to use the existing BIOS software. This meant that I would have to emulate some hardware, but only to the extent that the BIOS uses it. And I would not just stare at access patterns and the immediately surrounding (disassembled) BIOS code but take a more holistic approach as described below. Of course, it also meant that I would need to emulate all the hardware versions, which in retrospect has been more work than expected but not intrinsically hard.

All the CD-i specific hardware accesses done by the BIOS are located in OS-9 driver modules which together are less than 20% of a typical BIOS (counted by module size in bytes). And significantly more then half of that is the video driver, mostly taken up with drawing code that is totally uninteresting from the emulation point of view. All in all, only about 5% of the BIOS is concerned with accessing CD‑i hardware, which is about 25KB of 68000 machine language, i.e. less than 10 000 lines of machine code.

The interesting thing about these drivers is that their service interface is specified in the Green Book. Starting from this documented interface you can infer at a high level what the hardware is supposed to be doing as a consequence of all the device register accesses done by the BIOS. Actually figuring out the functions of the individual device bits does require disassembling the drivers and tracing their logic and dataflow but with a symbolic disassembler that is doable.

Most drivers are (much) less than 2KB of machine language, meaning that the path between the documented software interface and the to-be-emulated hardware is typically less than a few hundred or so lines of machine code. This is not hard to trace through.

The only driver size exceptions are the Video driver and the CD+Audio driver. The video driver totals about 65KB of machine code but as said above most of that drawing code not relevant for hardware emulation. Moreover, the video hardware is actually fully documented starting from the Mono-I hardware revision which uses the Motorola MCD212 VDSC chip. Earlier hardware revisions utilize two Philips SCC66470 VSC chips that are also documented and a back-end processor that is technically undocumented but whose functional aspects follow completely from the VSC documentation and the Green Book specifications. So the Video driver is not a major hurdle.

That leaves the CD+Audio driver, arguably the most complex driver of the entire CD-i system. The size of this driver module varies somewhat with the actual chipset used (about 18KB for CDIC, 26KB for DSP and 20KB for CIAP, in all cases excluding downloaded microcode). There is some uninteresting stuff such as EDC/ECC error correcting code in there but most of this driver is actually relevant to emulating the hardware. This driver is therefore the single biggest stumbling block to good CD-i emulation, and it shows in the kinds of bugs and issues that CD-i emulators have.

The fact that the CD+Audio hardware actually has three different versions (in Philips hardware alone, non-Philips players use yet another few versions) makes proper emulation of all player models more work but not much harder, it helps that the actual drivers are descended from each other. Once you have a good understanding of cdapdriv (CDIC), it is not that hard to understand dspdriv (DSP) or ciapdriv (CIAP): the major structure is the same, only the details of driving the hardware differ.

All of the above is predicated on understanding drivers, i.e. *software*, not *hardware*. It therefore goes somewhat against the grain for MAME which has traditionally followed a hardware-centric view. Also, a relatively deep understanding of the Green Book and parts of OS-9 is required.

Given those prerequisites I would not say that CD-i emulation is hard. It is a lot of work, sure; getting a good emulation requires deep study of a few thousand lines of 68000 assembly language. But very doable.

The approach used by CD-i Emulator as outlined above also explains why I am not interested in non‑HLE emulation of the various microcontrollers and DSP chips. It serves no purpose in playing CD-i discs: getting good HLE emulations of these chips so the BIOS can do its thing is enough. As a not insignificant side bonus to this, no chip decapping or other exotic hardware exercises are required for this approach.

CD-i update

Over the last few days I did some CD-i website updates.

On the CD-i Emulator website http://www.cdiemu.org I added lots of hardware pictures taken in August of last year. These include an extensive series about my (then) new 180/181/182 authoring player set and some overview pictures of the FW380i player and my E1/E2 emulator hardware. I have many more pictures of the internals of those emulators but these still need adding. I also added some more serial number information, especially from the DV cartridges. Some of the pictures of those appear to be mixed up a bit and will need sorting out.

I also started creating a Books section to list all my hardcopy documentation, as part of an effort to scan some of it to create digital versions where needed, but that is not yet online.

The end goal here is to get everything that could possibly be needed by a CD-i title developer online in a single place.  This effort started in July 2017 when I took over editorial maintenance of the ICDIA website at http://www.icdia.co.uk from Devin. We had decided that the CD-i Emulator site was not the right place to put general CD-i documentation and the ICDIA site seemed a much better fit.  And Devin was looking for a new ICDIA editor anyway, so...

As my first action on the ICDIA website I added the CD-i Full Functional Specification (Green Book). It had already been online for years but was not very easy to find.  Next followed some more Microware OS-9 documentation (in particular the Allen-Bradley set) that was also already online, and today many service manuals (repair instructions including board and circuit diagrams) were added.

A start has also been made with adding authoring documentation, but much more of that still needs to be added. My hard disk contains a number of raw scans from last year that still need to be processed and put online. In particular, this includes the CD-I System Libraries Technical Reference Manual that contains C bindings for all the CD-i specific CD-RTOS functions described in the Green book; this information is an absolute must to have for any serious developer.

Devin has been sitting on a complete set of CD-i Technical Notes for developers for a long time; this needs to be scanned and put online also (so far he has only done a very limited subset of it). These notes document many intricacies of the CD-i system and can be very insightful for title developers

It has now been a very long time since I put anything on this blog, but this does not mean that CD-i related software development has stopped. However, it has slowed down considerably. Below is a very short (and unordered) list of my CD-i development related activities in the 7 years since I last posted here; most of these I will probably expand on in the future.
 
- handling of CD-i Emulator beta time limit expiry
- emulation and deconstruction of The 7th Guest for the 20th anniversary edition
- leveraging CD-i Emulator code for a homebrew project
- investigation of possible Android and SDL ports of CD-i Emulator
- restructure of CD-i Emulator video decoding for flexibility and performance
- investigation of current status of Microware CD-i build tools
- investigation of existing OS-9 ports of the GNU C/C++ compiler
- investigation of restructuring CD-i Emulator MPEG decoding to fix buffering issues
- substantive efforts at ROMless emulation for CD-i Emulator (promising but not finished)
- reaching out in an attempt to collect more CD-i player ROMs

Regarding the last item, from the Philips consumer player line I’m still missing ROMs from the CDI 210/00, CDI 210/60, CDI 220/80, CDI 380/00 and CDI 740/20 players. If anybody has one of these players, please contact me.

I’m also interested in more exotic players like the Philips CDI 602, CDI 604, CDI 610, CDI 611 and of course a number of non-Philips players, in particular the Bang & Olufsen BeoCenter AV5. For some of these there are possible acquisition routes, but it requires bugging people about them and that can only be done some much.

Last year I also acquired a Kyocera Pro 1000S player that so far hasn’t given up its ROM contents (the player is incredibly densely packed internally, but there appears to be a serial extension port that could be usable), and I’m also still looking for the SONY IVO player ROMs. If nothing else works, it should be possible to play out the ROM contents of these players as audio.

There are also many future projects that I’m still interested in; a new one as of today is getting the built-in service tests of the Mini-MMC and Mono-I players working as documented in the service manuals. These tests need a special Service PCB connected to the serial port that has only three buttons (TEST, YES, NO) and an 8-position 7-segment LED display. Part of these tests is probably run by the SLAVE processor, but ROM contents for this processor have recently become available from the MESS dumping efforts so there appears to be a possible route. It would require implementing 6805 processor emulation but that shouldn’t be that hard, this is a relatively simple and well-documented 8-bit microprocessor.

Having made a promising start with ROMless emulation (in particular, mostly working C++ versions of the video and pointer device drivers and a good setup to start developing C++ versions of the kernel and remaining drivers and file managers), it appears worthwhile to pursue this path as well. It would  make the (apparently) holy grail of CD-i emulation a possibility (no need for copyrighted player ROM contents, just the disc images) and would open up a path to re-releasing CD-i titles on other platforms without ROM rights issues (several title rights-owners have contacted me about this in the last few years). If done right (which I’m trying), it would even allow a rebuild of the CD-RTOS software from the C++ versions by compiling them with an OS-9 C++ compiler (GNU C++ comes to mind). Of course, this would not be the “authentic” CD-i experience but it would allow playing of supported titles (eventually, this should be most of them).

It would also be interesting to make more progress towards an SDL port of CD-i Emulator. The main structural barrier to this has already been removed; the CD-i video emulation used to be hardcoded for producing 24-bit Windows-compatible RGB format. This has now been replaced by more generic pixel-format-parameterized pixel generation code that can efficiently produce any RGB format fitting in 32 bits, as required by SDL; in the process also achieving some decoding performance improvement (in particular, more optimized region processing). A faceless SDL emulation (no GUI) should now not be very hard anymore; at some point an SDL-based GUI or perhaps a native Android / iOS / MacOS GUI could then be added.

So there’s still lots of work to do…

JNMS and Maxi-MMC updates

This weekend I fixed some disc emulation issues for the JNMS and Maxi-MMC boards. I had previously erroneously identified these two boards but they are different.

The JNMS board is the one in the CDI 180 player (also called the JNMS player). It is not used in any other player and contains a CDIC (CD Interface Controller) chip but no SLAVE processor.

The Maxi-MMC board is the one in the CDI 601 and 602 players. From the emulation point of view it is virtually identical to the Mini-MMC board used by the CDI 605 player, but it has a different CDIC chip version. Both boards contain a SLAVE processor.

The link between the JNMS and Maxi-MMC boards is the CDIC chip: both turn out to have the same older CDIC chip version that differs in a few details from the version used on the Mini-MMC and Mono-I boards players (I described these differences in the earlier “CD-i 180 disc playing” post).

I noticed the JNMS / Maxi-MMC link from the CD-i player type table in the July 1996 issue of The Interactive Engineer (it’s on the ICDIA site); turns out I had misinterpreted the Board column on page 4 (there’s also an error there: the 601/602 certainly do not have the 180 board!).

After noticing this I did some testing and it turns out that the CDIC modifications needed for the 180 also work for the 601, including the TOC reading problem.

I have yet to find a way to get chip version information from the CDIC chip itself, so for the time being I’ve keyed the differences on the SLAVE software version. The 180 has no such chip, the 601 has version 1.x where the 605 has version 3.x. For now I’ve assumed that version 2.x also uses the older CDIC chip, but that may be wrong (the 602 or 604 might be interesting test cases).

Having done that, I did some more digging into the TOC read issue. It turns out that the 601 ROM performs CRC validation on the subcode Q data from the lead-in area (which is where the TOC is stored), and CD-i Emulator didn’t provide a valid CRC (no other ROMs I’ve seen so far validate this in software). The ROM even has compensation for shifts of between 1 and 7 bits in the incoming subcode Q data, probably because of some hardware timing issue.

I also noticed a bug in the ROM: it always uses the first sector buffer because it takes the current buffer bit from the wrong memory location. Not that this really matters because the TOC data is repeated multiple times; half of the repetitions will always land in the first buffer anyway. The bug is fixed in the 605 ROM.

Generating a valid CRC turned out to be straightforward (it’s just a simple CRC-CCITT calculation), but the ROM wouldn’t recognize it! After some head scratching I focused on the ROXL instruction (Rotate Left with Extend) used in the validation code. It is quite an esoteric instruction; could it be that there was an emulation bug here? It turns out that there was indeed; during rotation the contents of the X flag where put in the wrong bit. After fixing this the ROM properly recognized the data and the TOC reads became just as quick as other player models.

In search of version information for the CDIC chip I looked at the emulations and found one potential point of interest: the release number displayed by the service shell. This is a special GUI shell that performs some service functions; you can get to it by inserting a specially wired test plug into input port 1.

After some digging I found that the service shell obtains this number from the SLAVE processor, so it probably does not directly correspond to a CDIC version. The number does appear to differ from other version numbers, though, at least on my two 605 players.

The service shell obtains this number using two special I$SetStt calls into the CDIC driver; extending CD-i Link to remotely perform these same calls was easy. The new -cds[tatus] option can now be used to make the special calls. Here's some representative output of the -cds A3 option:

CD status A3000000 -> A3320000

Extending CD-i Link with remote OS9 calls is actually a fairly easy way to perform some information and tracing actions; I will probably use it for sorting out other dangling issues in the near future. When possible, this technique avoids the problems of writing a fullblown memory-resident trace module.

A new public beta release of CD-i Emulator that has full JNMS and Maxi-MMC support (among other things) is scheduled before the end of this year; there are still a few other issues that need sorting out first. This release should also have better support for the PCDI board used by several portable players, including the CD-i 370.

The major player holes still remaining are the Sony IVO-10/11 players, the Kyocera player, the Bang&Olufsen TV/player combi and of course the I2M board. There is some perspective for all of these but they are not high priority; except for the latter I expect all of them to be minor hardware variations of existing boards.

The I2M board has the interesting feature that it has multiple "ROMs" downloaded from the PC software (which is available for download from ICDIA); it also has a very different way of reading from CD as this is handled by the PC. As a consequence of this, audio is probably also handled differently. I have this board booting to a blue screen where it hangs on host communication.