MSX Dev 2020 part 4: 8kB, PSG, 9918

Here is a slowed-down gif of my "smooth scroll" routine:

Difficult to tell from the gif, but it runs at 60fps and only shifts 1 pattern column at a time.

Behind the scenes is much cooler. In instantanous fashion, the data for the next room (700 some bytes) is loaded from a swapped memory page into RAM and cycled into VRAM as shown above.

9918 limitations:
4 sprites per line
1 color per sprite
29 T-states between VRAM writes (!!!)

Monsters are 2x2 patterns, and while there's space for 100 in RAM, there will likely never be that many on-screen at once, regardless, this and projectile flicker allows for 2 players to have 2 colors - and monsters are actually more colorful than sprites would be. The downside is they move in 8 pixel chunks.

The VRAM writes are a big problem. This VDP tutorial is very good, but can be misleading. In my experience, the 9918 VDP ALWAYS has a 29 t-state wait. On the MSX2, you still need a small wait in between reads and writes. You can use ldir "unlmited" during vblank, but be careful. Related, if you compare carefully timed VDP writes DURING vblank on openMSX and real hardware, you will see discrepencies. This is because emulation down to the microsecond is physically impossible. At any rate, when coding for the MSX1, the suggested method of:

outloop:
    outi
    jp nz, outloop

Works very well (due to the exactness of the timing = 29 T-states).

When working with the 9938, the thing to keep in mind is that there is a MINIMUM time of 5-8 T-states between reads and writes. If you are polling VRAM:

    ld hl, (vram_addr_to_read)

    ld a, l 

    out (VDP_STATUS), a 

    ld a, h 

    out (VDP_STATUS), a 

    nop 

    nop                 ; do nothing for 8 cycles!

    in a, (VDP_DATA)

Otherwise, you will get erroneous graphics.

Using a garden variety of shit (RLE encoding, 3.7kb music player, cartridge paging) I managed to squeeze the requirements for the game down to 8kB of RAM. It looks pretty good so far, I think, and runs on the most bare minimum of hardware.

Horrah!

MSX Dev 2020 blog 3: Hardware woes

The hardest part about ensuring that your software will work across all MSX variants is proper slot management.

When an MSX computer is booted up, before it loads into BASIC (or DOS) and scans floppy drives for executables, it searches the first two bytes of page 1 and 2 of all cartridge slots for the word "AB" (0x42, 0x43). In other words, your cartridge must contain 4342h at either $4000 or $8000.

When "AB" is found, that page of that cartridge is switched to being active. An unexpanded, base MSX1 looks like this pre-boot:

Page 0: Main ROM
Page 1: Main ROM
Page 2: [nothing]
Page 3: 16K RAM

Thus it makes sense for smaller games to simply boot from $8000 and stay in page 2. If a game contains "AB" at $4000, however, the configuration ends up looking like:

Page 0: Main ROM
Page 1: Slot w/ cartridge
Page 2: [nothing]
Page 3: 16K RAM

A 32kB ROM's upper 16kB is not necessarily enabled on page 2 - this I have found to be dependent on what system you're using and how you are launching the ROM. Thus, at minimum to ensure 100% compatibility, a 32kB ROM with its header at $4000 and existing in pages 1 and 2 needs the following:

4000h - "AB" + rom header that jumps to:
+ "FindMySlot" routine
+ enables active slot on page 2
+ jumps to page 2 (e.g. actual game init code is at 8000h)

From then on, perform loading from page 1 or whatever is needed, then enable Main-ROM on page 1 again, for the final configuration of:

Page 0: Main ROM [undisturbed]
Page 1: Main ROM [4000-8000h of cartridge data behind]
Page 2: 8000-bfff of cartridge data
Page 3: 16K RAM [undisturbed]

It's no wonder that floppy disks are a preferred method. They are infinitely cheaper than larger cartridge ROMs, inevitably hold more data, are far easier to reproduce and even fix, and the BIOS disk routines are not that difficult to use!

I encountered a similar problem when attempting to use the MoonBlaster sound driver. It requires certain pages to be set as Main ROM or RAM, but doesn't always check, and doesn't change things back when its done. It works under certain conditions, but usually on an MFR it just crashes or resets my system. Games like from Kai Magazine have this issue on my hardware. I got a headache trying to grok the source code for the MB replayer to fix it, so I just disassembled MuSICA for BASIC by hand and adapted it myself for pure assembly.

Ugh!

MSX Dev 2020 Blog 2: The Tooling

I use Visual Studio Code and tniasm (freeware). The rest is just and tools and scripts made by myself in Python, and my build script in Bash.

I test using openMSX cfg'd/linked with build artifacts usually, otherwise I burn the ROM to my MFR SCC+ SD using SofaRun. I test using a stock Sony HB-F1XDmk2 with no expansions plugged in (except the MFR).

The interesting part:

The ROM is setup with a standard byte header. It occupies page 1 and 2. Code exists in page 2; page 1 is usually hidden behind Main-ROM, and is accessed when loading data. 4000h-7FFFh will only contained RLE'd game data - graphics, room data, music, etc.

I made a quick run-length encoding script in Python that compresses the game data. The decompression routine is extremely fast making it well worth it (and possibly can be optimized more).

The music player is a variant of the DOS-based MuSICA replayer that I made  a few months ago for an MSX2 project that uses OPLL. I cut out what I could from my disassembly that was obviously not for PSG-use and shaved a kB or so. I shoved it as high as I could in RAM and this bad boy fits neatly in E000h, still alotting a ton of space for game-use RAM. I composed the music on my F1XD using the bug-fixed and SCC+-compatible MuSICA editor I released. Music is 2-channel, allotting for software SFX on a third channel.

This image of the memory dump from ~EB00h to ~F100h shows almost a kilobyte of buffer before HIMEM.

MuSICA is a little on the slow side though, so it might need tweaking...

The graphics are all designed by me using my MSX2 graphic toolset. I export as .z80 when I'm ready and my build script does the rest. There are a few bugs with the toolset I need to tweak as of writing, like exporting from the screen tool only works with .z80 for now.

The enemies are background tiles a'la Gauntlet I on NES (9918 VDP has too low of a raster limit to make them real sprites). This actually lets them be very colorful on the MSX1 - I used my tool in sprite mode in one window, then copied them to the tool in pattern mode so I could do 16x16 at once. I looked at Final Fantasy Legend/SaGa sprites as inspiration.

That's all for now!

MSX Dev 2020 Blog 1

One reason for this post is the cool build pipeline I have going. I'm on Ubuntu, and I wrote a nice little one-click compile-and-run bash script that will do the following:

-Compile assets in .z80 format to .bin
-Compress .bin to .rle
-Move .rle to their asset folders and delete the .bin artifacts
-Compiles the ROM (using tniasm)
-Analyzes the label definitions to calculate remaining byte space AND automatically adds breakpoints to an openMSX config file
-Launches OpenMSX w/ the proper configs

I've pasted it at the bottom of this post.

BEHOLD, IRIDESCENT:

IRIDESCENT is a two-player, pseudo-random cyberpunk hack-and-slash. Imagine Gauntlet or Smash TV crossed with Shadowrun.

Progress shown above:
-Room data is decompressed on load, enemies are populated in RAM simultaneously.
-Support for 25 enemies at once
-Enemies animate and collide with the player and projectiles, player collides w/ 2px accuracy on walls/enemies
-Video buffering - 1/3 of the screen tiles from RAM buffer are drawn to the VDP per frame, easily allotting for 60 fps sprite action with no play impact
-PSG-only MuSICA playback

The white is video blank code. By only doing 1/3 of the screen tiles at once, this easily keeps update time within blank interval.

Green is per-frame CPU code. After optimizations today, it's been cut down to allot for PLENTY of more calculations!

compile-and-run.sh:

#!/bin/bash

check_success(){
    if [ $? -eq 0 ]; then 
        :
    else 
        cat errors.log
        echo 
        echo Compile script failed. Terminating.
        exit 1
    fi 
}

echo Compiling assets...
za=$(wine tniasm ./screens/screen1.z80 ./screen1.bin)
check_success

...

echo Compressing assets...
za=$(python3 ./rlenc.py screen1.bin)
check_success

...

# move compressed data to folders and remove artifacts
mv screen1.rle ./screens/
rm screen1.bin

...

echo Done. Compiling ROM...

calc_block_size () {
    # $1 = label to find, end of used-up memory
    # $2 = start of block (byte no.)
    # $3 = full size of block in b
    te="$(grep -i $1 tniasm.sym)"
    a="$te"
    b=${a#*:}
    c=${b%;*}
    te2="$(echo "$c" | tr -dc '0-9','A-F')"
    d="$te2"
    t3="$(echo "obase=10; ibase=16; $d" | bc)"
    e="$t3"
    f="$(($e-$2))"
    g="$(($3-$f))"

}

# compile rom, catch errors
w=$(wine ./tniasm.exe gamerom.z80 irides.rom 2> errors.log)
check_success

echo Success. 

find_debug_points(){
    db=($(grep -i breakpoint tniasm.sym))
    for i in ${db[@]}; do 
        vv=${i#*:}
        vb=${vv%;*}
        vc=$(echo "$vb" | tr -dc '0-9','A-F')
        vd="$vc"
        if [ ${#vd} == 5 ]; then
            echo debug set_bp 0x$vd >> m1.txt
            echo Breakpoint found @ \$$vd
        fi
    done 
}

echo plug joyporta keyjoystick1 > m1.txt

find_debug_points

calc_block_size PlayerCompressed_end 16384 16384
echo "$g bytes remaining in data block (\$04000-\$$d)"
calc_block_size EndMainCode 32768 9655
echo "$g bytes remaining in code block A (\$08000-\$$d)"
calc_block_size EndBlockTwo 42423 6729
echo "$g bytes remaining in code block B (\$0a5b7-\$$d)"
calc_block_size EndRAM 49152 3583
echo "$g bytes remaining for RAM (\$0c000-\$$d)"

echo Launching OpenMSX...

sudo /opt/openMSX/bin/openmsx -machine Sony_HB-10 -cart ./irides.rom -script m1.txt

The output looks like this: