Self-decompressing COM files?

[Steve Nickolas]

OK, this is going to get weird. The tl;dr of it is that I'm wondering if
there's a tool to create self-decompressing (like what upx does) binaries
for CP/M-80 (i.e., a CP/M executable cruncher). I'll get into the gories.

I've been working on porting some games to a system (Nabu) that runs CP/M
3.1 - the call at 5 is to $BA06. (There are two other unofficial ports of
CP/M, both of 2.2, and one of them I'll probably need to figure out the
equivalent value; the other I don't care about.)

For one of them, I've been working on trying to expand it from the source
version's (MSX) 76 levels to the full 150. I don't think there's memory
to have all 150 and still enough room to load the program from CP/M. (The
level storage format is kind-of sparse and wasteful, but easy to figure
out. Basically, it's BCD.) I decided to include 120 levels.

But I was wondering about whether, using an executable cruncher, I could
try to include the whole set of 150.

(The other method - to store them in a file and load them on the fly, like
the original game on the Apple ][ - would require a lot more refactoring
of the code to move where scratch RAM is located.)

-uso.

[Jacob Nevins]

Post by Steve Nickolas
OK, this is going to get weird. The tl;dr of it is that I'm wondering if
there's a tool to create self-decompressing (like what upx does) binaries
for CP/M-80 (i.e., a CP/M executable cruncher).

PopCom! is one such.

Apparently I've lost track of where CP/M archive mirrors are these days,
but you can download a copy, and documentation in Japanese (looks like
Shift-JIS encoding), from
https://www.asahi-net.or.jp/~am9y-mn/fswlist.htm

Post by Steve Nickolas
I've been working on porting some games to a system (Nabu) that runs CP/M
3.1 - the call at 5 is to $BA06. (There are two other unofficial ports of
CP/M, both of 2.2, and one of them I'll probably need to figure out the
equivalent value; the other I don't care about.)
For one of them, I've been working on trying to expand it from the source
version's (MSX) 76 levels to the full 150. I don't think there's memory
to have all 150 and still enough room to load the program from CP/M.

[...]

Post by Steve Nickolas
But I was wondering about whether, using an executable cruncher, I could
try to include the whole set of 150.

I'm not sure how much PopCom will help you increase your maximum program
size on your CP/M 3.1 system, though?
(Disclaimer: I only ever used a banked 3.1 system with loads of memory,
so I had a massive TPA and never had to understand any of this stuff
properly. So I may have misunderstood. Also I haven't thought about any
of this for a very long time.)

Machine-translating the PopCom docs:

"When a programme compressed with PopCom! is executed, it is not limited
by the TPA size when PopCom! is executed."

Looking at this CP/M 2 description
http://www.gaby.de/cpm/manuals/archive/cpm22htm/ch5.htm
I can see how PopCom can increase effective program size, because the
definition of TPA (limiting the loaded program's size) doesn't include
the CCP, but your program can re-use the CCP's space once it's loaded.
So I guess PopCom lets you increase your program size by the size of the
CCP (however much that is), since PopCom can uncompress into space that
was occupied by the CCP during loading.

However, on CP/M 3, I think that restriction on loaded program size
is reduced to the LOADER (not the whole CCP), which is documented as
being "three pages long" (= 384 bytes IIRC).
(http://www.cpm.z80.de/manuals/cpm3-pgr.pdf pp17-8)
So I think PopCom can only increase your maximum program size by that
much on CP/M 3? (And I don't know how much of that PopCom's own
overheads might eat.)

Other tidbits machine-translated from POPCOM.DOC, in case they avoid
wasting your time:
"PopCom! can be used with CP/M of 47K or more (TPA 42K or more).
However, a CPU equivalent to Z80 or higher is required."
"It also uses only the minimum amount of memory required for
decompression, so if the source programme is one that does not destroy
CCP, the compressed programme will also work without destroying CCP in
most cases."

It looks like your TPA is 46.25 kbyte or thereabouts, so I guess PopCom
will work on your system.

[Steve Nickolas]

Post by Jacob Nevins

Post by Steve Nickolas
OK, this is going to get weird. The tl;dr of it is that I'm wondering if
there's a tool to create self-decompressing (like what upx does) binaries
for CP/M-80 (i.e., a CP/M executable cruncher).

PopCom! is one such.
Apparently I've lost track of where CP/M archive mirrors are these days,
but you can download a copy, and documentation in Japanese (looks like
Shift-JIS encoding), from
https://www.asahi-net.or.jp/~am9y-mn/fswlist.htm

Yeah, it's SJIS. My Japanese isn't great but I do speak it a little... ;p

Post by Jacob Nevins

Post by Steve Nickolas
I've been working on porting some games to a system (Nabu) that runs CP/M
3.1 - the call at 5 is to $BA06. (There are two other unofficial ports of
CP/M, both of 2.2, and one of them I'll probably need to figure out the
equivalent value; the other I don't care about.)
For one of them, I've been working on trying to expand it from the source
version's (MSX) 76 levels to the full 150. I don't think there's memory
to have all 150 and still enough room to load the program from CP/M.

[...]

Post by Steve Nickolas
But I was wondering about whether, using an executable cruncher, I could
try to include the whole set of 150.

I'm not sure how much PopCom will help you increase your maximum program
size on your CP/M 3.1 system, though?
(Disclaimer: I only ever used a banked 3.1 system with loads of memory,
so I had a massive TPA and never had to understand any of this stuff
properly. So I may have misunderstood. Also I haven't thought about any
of this for a very long time.)

I'm actually kicking CP/M out after I load the program. But the program
won't load right if it's big enough to step on the BDOS in the process.

Post by Jacob Nevins
Other tidbits machine-translated from POPCOM.DOC, in case they avoid
"PopCom! can be used with CP/M of 47K or more (TPA 42K or more).
However, a CPU equivalent to Z80 or higher is required."
"It also uses only the minimum amount of memory required for
decompression, so if the source programme is one that does not destroy
CCP, the compressed programme will also work without destroying CCP in
most cases."
It looks like your TPA is 46.25 kbyte or thereabouts, so I guess PopCom
will work on your system.

Nod. It's an unbanked 64K system.

I'm just using CP/M as a loader, so if the decompressor stomps on CP/M, I
don't really care. Might have to make it DI first though, if it doesn't
already.

-uso.

[Steve Nickolas]

Just tested it - had to go into an actual CP/M environment, but that was
fine - crunched the game I was working on (Lode Runner btw) and tried it,
and it worked.

-uso.

[Jacob Nevins]

Post by Steve Nickolas
Just tested it - had to go into an actual CP/M environment, but that was
fine - crunched the game I was working on (Lode Runner btw) and tried it,
and it worked.

\o/

Re running in actual CP/M environment: for what it's worth, I find that
utilities like PMEXT run fine under zxcc, which means I can conveniently
run them in my host (Linux) environment.
https://www.seasip.info/Unix/Zxcc/index.html
POPCOM may work this way too. (In a quick test, it seemed to basically
work, although there was a "cpmredir: Corrupt FCB" glitch on exit.)

[Steve Nickolas]

Post by Jacob Nevins

Post by Steve Nickolas
Just tested it - had to go into an actual CP/M environment, but that was
fine - crunched the game I was working on (Lode Runner btw) and tried it,
and it worked.

\o/
Re running in actual CP/M environment: for what it's worth, I find that
utilities like PMEXT run fine under zxcc, which means I can conveniently
run them in my host (Linux) environment.
https://www.seasip.info/Unix/Zxcc/index.html
POPCOM may work this way too. (In a quick test, it seemed to basically
work, although there was a "cpmredir: Corrupt FCB" glitch on exit.)

Perhaps. I tried the Lopushinsky emulators in DOSBOX. (I also run on a
Linux host.)

-uso.

[Tony Nicholson]

Post by Jacob Nevins
Re running in actual CP/M environment: for what it's worth, I find that
utilities like PMEXT run fine under zxcc, which means I can conveniently
run them in my host (Linux) environment.
https://www.seasip.info/Unix/Zxcc/index.html
POPCOM may work this way too. (In a quick test, it seemed to basically
work, although there was a "cpmredir: Corrupt FCB" glitch on exit.)

I've collected some updates and patches for John Elliot's ZXCC which
you'll find on GitHub at

https://github.com/agn453/ZXCC

Be sure to read the README for the complete list of changes.

Tony