Clownacy
Member
- Messages
- 36
Here is Sonic 2 with Knuckles and all of Sonic 1's levels. It's very unfinished, but playable from beginning to end.
Development Background
I'd like to take the 'release early, release often' approach with this hack to mitigate the problem of scope-creep and other setbacks delaying the completion (and therefore release) of the hack, so I'm releasing the current prototype for feedback and so that people who don't mind the hack's unfinished state get a chance to have fun with it.
Download
https://sonicresearch.org/clownacy/Sonic 1 & 2 & Knuckles.zip
Porting Trivia
Development Background
Recently, I released a hack of Knuckles in Sonic 2 that restores Sonic and Tails to the game. That hack served more than one purpose: not only was it intended to be released as a hack of its own, but it was also created to serve as the basis for another hack - this one.
My goal with this hack is to create my own definitive versions of Sonic 1 and 2, which are my favourite classic Sonic games. However, a hack of this scope will take a very long time to complete, so I have opted for a development strategy that emphasises 'vertical slices': the idea is to divide the process of creating the hack into completing a series of smaller projects, with each one having a defined beginning and end that is separate from the other projects. This means that I can take things one step at a time and have something complete and presentable at each milestone. Decompiling Knuckles in Sonic 2 was the first milestone, restoring Sonic and Tails to it was the second, and this is the third.
This third milestone is porting all of Sonic 1's levels to Sonic 2. By doing this, I don't need to make two separate 'definitive edition' hacks of Sonic 1 and Sonic 2, as they are both now part of the same game. It also means that Sonic 1 automatically gains Sonic 2 features such as Tails and the Spin Dash without me needing to backport them to Sonic 1's engine.
With that said, the hack is a bit rough around the edges at the moment. This is the result of porting Sonic 1's levels as-is, and some things do not make the transition very well. This is mostly a problem with the levels' colour palettes, but the occasional engine alteration between Sonic 1 and Sonic 2 can also cause issues, like with Scrap Brain Zone's running discs. Also, Knuckles' lower jump height renders certain parts of levels impossible to navigate.
My goal with this hack is to create my own definitive versions of Sonic 1 and 2, which are my favourite classic Sonic games. However, a hack of this scope will take a very long time to complete, so I have opted for a development strategy that emphasises 'vertical slices': the idea is to divide the process of creating the hack into completing a series of smaller projects, with each one having a defined beginning and end that is separate from the other projects. This means that I can take things one step at a time and have something complete and presentable at each milestone. Decompiling Knuckles in Sonic 2 was the first milestone, restoring Sonic and Tails to it was the second, and this is the third.
This third milestone is porting all of Sonic 1's levels to Sonic 2. By doing this, I don't need to make two separate 'definitive edition' hacks of Sonic 1 and Sonic 2, as they are both now part of the same game. It also means that Sonic 1 automatically gains Sonic 2 features such as Tails and the Spin Dash without me needing to backport them to Sonic 1's engine.
With that said, the hack is a bit rough around the edges at the moment. This is the result of porting Sonic 1's levels as-is, and some things do not make the transition very well. This is mostly a problem with the levels' colour palettes, but the occasional engine alteration between Sonic 1 and Sonic 2 can also cause issues, like with Scrap Brain Zone's running discs. Also, Knuckles' lower jump height renders certain parts of levels impossible to navigate.
I'd like to take the 'release early, release often' approach with this hack to mitigate the problem of scope-creep and other setbacks delaying the completion (and therefore release) of the hack, so I'm releasing the current prototype for feedback and so that people who don't mind the hack's unfinished state get a chance to have fun with it.
Download
https://sonicresearch.org/clownacy/Sonic 1 & 2 & Knuckles.zip
Porting Trivia
Porting levels from Sonic 1 to Sonic 2 has been pretty interesting: many of the data formats changed between the two games, requiring that Sonic 1's data be converted. For this, I wrote a small tool in C to automate the process. I figured that tools like MainMemory's LevelConverter would eventually prove too limiting for my needs, which eventually turned out to be true, so I'm glad that I went with writing my own.
Chunks and Tiles
One such format change was level 'chunk' data being resized from 256x256 to 128x128. Splitting the 256x256 chunks into 128x128 chunks is simple enough, but doing so can result in more chunks than the engine supports. Culling unused and duplicate chunks helps with this, but Spring Yard Zone and Labyrinth Zone still use too many chunks even afterwards. To resolve this, I made my tool split chunks between individual levels when necessary, which is a trick that Sonic Megamix also used.
The format of tiles did not change between games, however Sonic 2 has a much tighter VRAM budget due to having both Sonic and Tails together at the same time. Star Light Zone and Scrap Brain Zone use too many tiles for this, so I made my tool split the tile data for those zones as well.
Compatibility Shim
Another interesting thing that I did was implement a compatibility layer for ported Sonic 1 code. This is made necessary by the disassemblies of Sonic 1 and Sonic 2 being wildly different, each using their own naming schemes for variables and functions, and having their own directory structures. Rather than spend a bunch of time converting all of the ported Sonic 1 code to suit Sonic 2's disassembly, I instead implemented a compatibility shim that allows the Sonic 1 code to operate as if it were still in the Sonic 1 disassembly, allowing the ported code to be used almost completely unmodified. This is accomplished by aliasing symbols from the Sonic 1 disassembly to their equivalents in the Sonic 2 disassembly. Here's a snippet of that:
Curiously, Sonic 1 uses a slightly different animation script format to Sonic 2, so, in order to be able to use Sonic 1's Badniks and the like unmodified, I had to port Sonic 1's 'AnimateSprite' function and make the ported objects use it instead of Sonic 2's version.
Object ID Limit
One big challenge with making this hack was overcoming the engine's limit on object IDs. Each different type of object in the game has a unique ID, and this ID is stored in a byte, creating a limit of 256 different IDs. By porting many of Sonic 1's objects to Sonic 2, this limit is reached. I could have worked around this by having two sets of 256 IDs that are selected based on which zone the player is currently in, but I found that to be too much of a nasty hack for my tastes, so I opted to do things the 'proper' way by extending the IDs to 16-bit.
This required extending the object state struct ("Sprite Status Table") from 0x40 bytes to 0x42 bytes, causing the objects to use substantially more RAM than before. This is actually similar to a modification that was made to Sonic 3's engine, however that modification involved the removal of object IDs entirely, instead replacing them with a pointer to the object's code. With that done, I was also able to pre-multiply the object IDs by 4 to avoid constantly doing so at runtime, which provides a small performance boost.
Level Animation
Because Sonic 2 was made from Sonic 1, porting Sonic 1's levels to it is pretty natural: the engine supports all of the same subsystems that Sonic 1's levels and objects rely on, and in many cases there is leftover or reused code from Sonic 1 in Sonic 2's engine that can be repurposed.
Curiously, while Sonic 2 introduced a new script-based system for handling animated level artwork, it still maintains backwards-compatibility with Sonic 1's code-based system, allowing the animated level artwork of Green Hill Zone, Marble Zone, and Scrap Brain Zone to be ported with ease. However, I did have to convert the ported code to use DMA transfers instead of manually poking VRAM, as this caused issued with the game's two-player mode.
Loops
Another major difference between Sonic 1's and Sonic 2's engines is how loop-de-loops work: in Sonic 1, when Sonic is running through a loop, the entire chunk of level data that makes up the loop is swapped-out for an identical-looking chunk that has different collision data, allowing Sonic to run through parts of the loop that were previously solid. In Sonic 2, however, the loop itself never changes: instead, the level has two 'layers' of collision data, and invisible objects are placed on the loop to make Sonic swap between the two layers when he touches them. Converting Sonic 1's loops to this new system would be easy enough, except Sonic 2's system is actually slightly more limited than Sonic 1's: the two collision layers can only differ in shape, not orientation, while Sonic 1's system allows both to change. Working around this required duplicating some collision shapes and pre-flipping them so that the second collision layer could use them properly.
Object Collision
Another problem with porting objects from Sonic 1 is that Sonic 2 made extensive changes to its object collision system to account for Tails in 'Sonic & Tails' mode. Since there could now be two player characters on-screen at once, objects have to account for the fact that multiple characters can be interacting with them at once. Sonic 2's object collision system is ultimately simpler to use than Sonic 1's, but converting to it without introducing bugs is still tricky because of how invasive the modifications can be. Perhaps the worst object for this is the block in Marble Zone that Sonic has to push around.
Music and Sounds
In the past, Sonic 1's music and sound effects may have been some of the hardest things to port to Sonic 2 due to them being in a game-specific bytecode format, however my recent conversion of the disassemblies to use ASM-formatted music and sounds makes the porting process trivial. Unfortunately, there are two sounds that are still problematic: the block-pushing sound from Marble Zone, and the flowing-water sound from Green Hill Zone and Labyrinth Zone.
The block-pushing sound is made awkward by the fact that it uses a custom sound command that does not exist in Sonic 2's sound driver. However, it did exist in some of Sonic 2's prototypes, so the code can simply be copied from one of those.
The flowing-water sound is much worse: unlike every other sound in the game, it is a 'background sound'. A background sound (also known as a 'special SFX') is a unique type of sound effect, which is lower priority than a regular sound effect and higher priority than music. Since Sonic 2 doesn't use any background sounds, its driver lacks support for them, meaning that, in order to port the flowing-water sound from Sonic 1, the entire background sound system needs to be ported to Sonic 2's sound driver. This is actually something that I had done before for an old April Fools' joke way back in 2015, so I knew how tedious it was. Still, I was able to get it done in about a day and have the sound working as intended.
Rings
Another fun difference between Sonic 1 and Sonic 2 is that rings are regular objects in Sonic 1, but an entirely separate subsystem in Sonic 2. I assume that this was done to save RAM and CPU cycles, since allocating and processing a whole object for something as simple as a ring is just wasteful. Accounting for this required making my conversion tool split rings from the level object placement data to their own special ring placement data. Curiously, Sonic 1's ring spawner object supports several arrangements of rings that Sonic 2's ring spawner does not, so those have to be emulated by manually placing individual rings in the required arrangement.
Sprite Mappings
The data that arranges tiles into sprites is known as 'sprite mappings'. Between Sonic 1 and Sonic 2, the format of these mappings changed. These changes included adding additional data for the game's two-player mode, extending the range of the X coordinate to cover the whole screen, and padding the header so that the mapping data could be safely read as a series of CPU words rather than bytes.
At first, my method of porting mappings from Sonic 1 to Sonic 2 was opening them in my ClownMapEd sprite editor in one format and then saving them in the other, but this was very slow, tedious, and it was also a destructive process: my sprite editor does not preserve the exact structure of the data that it loads, even if it isn't edited. I didn't like the idea of needlessly altering data as it could theoretically introduce bugs in cases where the mapping data, or code that relates to it, is unusually brittle. Because of this, I eventually settled on another way of porting mappings:
On GitHub, there exist alternative branches of the Sonic 1 and Sonic 2 disassemblies, which are named 'MapMacros'. As the name suggests, these are branches where the games' mappings (and Dynamic Pattern Load Cues) are converted to macros, abstracting-away the underlying format differences between games, making the porting process a simple copy-paste job. I integrated support for these macros into my hack, enabling me to use them to quickly, easily, and non-destructively port mappings.
Since integrating them into my hack, I've added support for MapMacros to ClownMapEd and merged the MapMacros branches into the master branches of the two Sonic disassemblies, so that everyone can benefit from the portability that they add.
Labyrinth Zone Gimmicks
Labyrinth Zone features currents that pull the player through tunnels, as well as water slides. The code for these gimmicks was repurposed in Sonic 2, for the wind in Wing Fortress Zone and the oil slides in Oil Ocean Zone. The code was slightly modified, adding sanity checks and changing or removing the associated sound effect. When porting Labyrinth Zone, some of these modifications needed to be disabled to restore the original behaviour in that zone.
Closing
This project has been a lot of fun: it was cool to learn all of the differences between Sonic 1's and 2's engines, and it was satisyfing to restore logic and data that had been crudely ripped-out or repurposed in Sonic 2. The engine feels a lot more 'complete' with Sonic 1's levels restored: suddenly the leftover Sonic 1 objects and background parallax scroll code are no longer just dead code, but useful parts of the codebase once again.
It's also just really cool to play Sonic 1's levels in a newer and more-refined engine: the rings using their own subsystem, the sound driver running on the Z80 CPU, dynamic artwork being loaded via the DMA queue, and so on. It's also pretty neat to see Sonic 1 with so many 'Sonic 2-isms', such as Sonic 2's HUD, Sonic sprites, having Tails as a companion, Sonic 2's monitor sprites, explosion sprites, checkpoints, Special Stages, title cards, loading times, etc. Being able to play Sonic 1 as Tails and Knuckles also kicks ass, especially since I didn't have to port either of them.
Chunks and Tiles
One such format change was level 'chunk' data being resized from 256x256 to 128x128. Splitting the 256x256 chunks into 128x128 chunks is simple enough, but doing so can result in more chunks than the engine supports. Culling unused and duplicate chunks helps with this, but Spring Yard Zone and Labyrinth Zone still use too many chunks even afterwards. To resolve this, I made my tool split chunks between individual levels when necessary, which is a trick that Sonic Megamix also used.
The format of tiles did not change between games, however Sonic 2 has a much tighter VRAM budget due to having both Sonic and Tails together at the same time. Star Light Zone and Scrap Brain Zone use too many tiles for this, so I made my tool split the tile data for those zones as well.
Compatibility Shim
Another interesting thing that I did was implement a compatibility layer for ported Sonic 1 code. This is made necessary by the disassemblies of Sonic 1 and Sonic 2 being wildly different, each using their own naming schemes for variables and functions, and having their own directory structures. Rather than spend a bunch of time converting all of the ported Sonic 1 code to suit Sonic 2's disassembly, I instead implemented a compatibility shim that allows the Sonic 1 code to operate as if it were still in the Sonic 1 disassembly, allowing the ported code to be used almost completely unmodified. This is accomplished by aliasing symbols from the Sonic 1 disassembly to their equivalents in the Sonic 2 disassembly. Here's a snippet of that:
Code:
; RAM
v_screenposx = Camera_X_pos
v_screenposy = Camera_Y_pos
v_player = MainCharacter
v_zone = Current_Zone
v_act = Current_Act
; Functions
Bg_Scroll_X = SwScrl_HPZ_Continued
BGScroll_XY = SetHorizVertiScrollFlagsBG
DeleteChild = DeleteObject2
ObjHitCeiling = ObjCheckCeilingDist
KillSonic = KillCharacter
SmashObject = BreakObjectToPieces
ExplosionBomb = Obj58
; Data
Drown_WobbleData = Obj0A_WobbleData
; Misc. Constants
cWhite = $0EEE
bitUp = button_up
bitDn = button_down
btnABC = button_A_mask | button_B_mask | button_C_mask
; Object IDs
id_BossBall = ObjID_BossBall
id_BossGreenHill = ObjID_GHZBoss
;id_ExplosionBomb = ObjID_BossExplosion ; No longer equivalent
id_GrassFire = ObjID_GrassFire
id_Crabmeat = ObjID_Crabmeat
id_Missile = ObjID_Missile
id_ExplosionItem = ObjID_Explosion
; SFX IDs
sfx_HitBoss = SndID_BossHit
sfx_Spring = SndID_Spring
sfx_Roll = SndID_Roll
sfx_Teleport = SndID_SpindashRelease
sfx_Burning = SndID_Flamethrower
sfx_Basaran = SndID_Basaran
sfx_ChainRise = SndID_ChainRise
sfx_ChainStomp = SndID_Hammer
sfx_Push = SndID_Push
sfx_Fireball = SndID_LavaBall
sfx_WallSmash = SndID_SlowSmash
sfx_Rumbling = SndID_Rumbling
sfx_Door = SndID_DoorSlam
sfx_Flamethrower = SndID_FireBurn
sfx_Saw = SndID_LaserBeam
sfx_Electric = SndID_Zap
sfx_Waterfall = SpecSndID_Waterfall
; PLC IDs
plcid_Boss = PLCID_S1Boss
plcid_FZBoss = PLCID_Fz
; Animation IDs
id_Roll = AniIDSonAni_Roll
id_Hang = AniIDSonAni_Hang
id_Run = AniIDSonAni_Run
; SSTs
;obID: equ id ; No longer equivalent.
obRender: equ render_flags ; bitfield for x/y flip, display mode
obGfx: equ art_tile ; palette line & VRAM setting (2 bytes)
obMap: equ mappings ; mappings address (4 bytes)
obX: equ x_pos ; x-axis position (2-4 bytes)
obScreenY: equ x_sub ; y-axis position for screen-fixed items (2 bytes)
obY: equ y_pos ; y-axis position (2-4 bytes)
obVelX: equ x_vel ; x-axis velocity (2 bytes)Curiously, Sonic 1 uses a slightly different animation script format to Sonic 2, so, in order to be able to use Sonic 1's Badniks and the like unmodified, I had to port Sonic 1's 'AnimateSprite' function and make the ported objects use it instead of Sonic 2's version.
Object ID Limit
One big challenge with making this hack was overcoming the engine's limit on object IDs. Each different type of object in the game has a unique ID, and this ID is stored in a byte, creating a limit of 256 different IDs. By porting many of Sonic 1's objects to Sonic 2, this limit is reached. I could have worked around this by having two sets of 256 IDs that are selected based on which zone the player is currently in, but I found that to be too much of a nasty hack for my tastes, so I opted to do things the 'proper' way by extending the IDs to 16-bit.
This required extending the object state struct ("Sprite Status Table") from 0x40 bytes to 0x42 bytes, causing the objects to use substantially more RAM than before. This is actually similar to a modification that was made to Sonic 3's engine, however that modification involved the removal of object IDs entirely, instead replacing them with a pointer to the object's code. With that done, I was also able to pre-multiply the object IDs by 4 to avoid constantly doing so at runtime, which provides a small performance boost.
Level Animation
Because Sonic 2 was made from Sonic 1, porting Sonic 1's levels to it is pretty natural: the engine supports all of the same subsystems that Sonic 1's levels and objects rely on, and in many cases there is leftover or reused code from Sonic 1 in Sonic 2's engine that can be repurposed.
Curiously, while Sonic 2 introduced a new script-based system for handling animated level artwork, it still maintains backwards-compatibility with Sonic 1's code-based system, allowing the animated level artwork of Green Hill Zone, Marble Zone, and Scrap Brain Zone to be ported with ease. However, I did have to convert the ported code to use DMA transfers instead of manually poking VRAM, as this caused issued with the game's two-player mode.
Loops
Another major difference between Sonic 1's and Sonic 2's engines is how loop-de-loops work: in Sonic 1, when Sonic is running through a loop, the entire chunk of level data that makes up the loop is swapped-out for an identical-looking chunk that has different collision data, allowing Sonic to run through parts of the loop that were previously solid. In Sonic 2, however, the loop itself never changes: instead, the level has two 'layers' of collision data, and invisible objects are placed on the loop to make Sonic swap between the two layers when he touches them. Converting Sonic 1's loops to this new system would be easy enough, except Sonic 2's system is actually slightly more limited than Sonic 1's: the two collision layers can only differ in shape, not orientation, while Sonic 1's system allows both to change. Working around this required duplicating some collision shapes and pre-flipping them so that the second collision layer could use them properly.
Object Collision
Another problem with porting objects from Sonic 1 is that Sonic 2 made extensive changes to its object collision system to account for Tails in 'Sonic & Tails' mode. Since there could now be two player characters on-screen at once, objects have to account for the fact that multiple characters can be interacting with them at once. Sonic 2's object collision system is ultimately simpler to use than Sonic 1's, but converting to it without introducing bugs is still tricky because of how invasive the modifications can be. Perhaps the worst object for this is the block in Marble Zone that Sonic has to push around.
Music and Sounds
In the past, Sonic 1's music and sound effects may have been some of the hardest things to port to Sonic 2 due to them being in a game-specific bytecode format, however my recent conversion of the disassemblies to use ASM-formatted music and sounds makes the porting process trivial. Unfortunately, there are two sounds that are still problematic: the block-pushing sound from Marble Zone, and the flowing-water sound from Green Hill Zone and Labyrinth Zone.
The block-pushing sound is made awkward by the fact that it uses a custom sound command that does not exist in Sonic 2's sound driver. However, it did exist in some of Sonic 2's prototypes, so the code can simply be copied from one of those.
The flowing-water sound is much worse: unlike every other sound in the game, it is a 'background sound'. A background sound (also known as a 'special SFX') is a unique type of sound effect, which is lower priority than a regular sound effect and higher priority than music. Since Sonic 2 doesn't use any background sounds, its driver lacks support for them, meaning that, in order to port the flowing-water sound from Sonic 1, the entire background sound system needs to be ported to Sonic 2's sound driver. This is actually something that I had done before for an old April Fools' joke way back in 2015, so I knew how tedious it was. Still, I was able to get it done in about a day and have the sound working as intended.
Rings
Another fun difference between Sonic 1 and Sonic 2 is that rings are regular objects in Sonic 1, but an entirely separate subsystem in Sonic 2. I assume that this was done to save RAM and CPU cycles, since allocating and processing a whole object for something as simple as a ring is just wasteful. Accounting for this required making my conversion tool split rings from the level object placement data to their own special ring placement data. Curiously, Sonic 1's ring spawner object supports several arrangements of rings that Sonic 2's ring spawner does not, so those have to be emulated by manually placing individual rings in the required arrangement.
Sprite Mappings
The data that arranges tiles into sprites is known as 'sprite mappings'. Between Sonic 1 and Sonic 2, the format of these mappings changed. These changes included adding additional data for the game's two-player mode, extending the range of the X coordinate to cover the whole screen, and padding the header so that the mapping data could be safely read as a series of CPU words rather than bytes.
At first, my method of porting mappings from Sonic 1 to Sonic 2 was opening them in my ClownMapEd sprite editor in one format and then saving them in the other, but this was very slow, tedious, and it was also a destructive process: my sprite editor does not preserve the exact structure of the data that it loads, even if it isn't edited. I didn't like the idea of needlessly altering data as it could theoretically introduce bugs in cases where the mapping data, or code that relates to it, is unusually brittle. Because of this, I eventually settled on another way of porting mappings:
On GitHub, there exist alternative branches of the Sonic 1 and Sonic 2 disassemblies, which are named 'MapMacros'. As the name suggests, these are branches where the games' mappings (and Dynamic Pattern Load Cues) are converted to macros, abstracting-away the underlying format differences between games, making the porting process a simple copy-paste job. I integrated support for these macros into my hack, enabling me to use them to quickly, easily, and non-destructively port mappings.
Since integrating them into my hack, I've added support for MapMacros to ClownMapEd and merged the MapMacros branches into the master branches of the two Sonic disassemblies, so that everyone can benefit from the portability that they add.
Labyrinth Zone Gimmicks
Labyrinth Zone features currents that pull the player through tunnels, as well as water slides. The code for these gimmicks was repurposed in Sonic 2, for the wind in Wing Fortress Zone and the oil slides in Oil Ocean Zone. The code was slightly modified, adding sanity checks and changing or removing the associated sound effect. When porting Labyrinth Zone, some of these modifications needed to be disabled to restore the original behaviour in that zone.
Closing
This project has been a lot of fun: it was cool to learn all of the differences between Sonic 1's and 2's engines, and it was satisyfing to restore logic and data that had been crudely ripped-out or repurposed in Sonic 2. The engine feels a lot more 'complete' with Sonic 1's levels restored: suddenly the leftover Sonic 1 objects and background parallax scroll code are no longer just dead code, but useful parts of the codebase once again.
It's also just really cool to play Sonic 1's levels in a newer and more-refined engine: the rings using their own subsystem, the sound driver running on the Z80 CPU, dynamic artwork being loaded via the DMA queue, and so on. It's also pretty neat to see Sonic 1 with so many 'Sonic 2-isms', such as Sonic 2's HUD, Sonic sprites, having Tails as a companion, Sonic 2's monitor sprites, explosion sprites, checkpoints, Special Stages, title cards, loading times, etc. Being able to play Sonic 1 as Tails and Knuckles also kicks ass, especially since I didn't have to port either of them.
