Exomizer 2.0 NEW FEATURES SINCE V1.X: o Totally rewitten handling of the sfx decruncher: 1) Several different decrunch effects. 2) Complete user control over memory and irq config. 3) Can start basic programs properly with built in trampoline by using the command 'sfx basic' for the Commodore and Oric targets. 3) The decruncher code is assembled on the fly for maximum flexibility by an embedded assembler. 4) Will adapt the decruncher to include only features used by the compressed stream to keep the size down. 5) Supports the c128 and the vic20 (in several configurations). 6) The Atari 400/800 XL/XE family is supported as sfx target 168. 7) Can generate relocated sfx targets without basic line. 8) The Apple ][+ (and //e) is supported as sfx target 162. 9) The Oric 8-bit family is supported as sfx target 1. o Reads Atari xex-files and autodetects the RUNAD vector if the sys keyword is given to the sfx command. o Can load plain files using the @

syntax. o Has a literal sequence escape mechanism that detects and handles sequences of uncrunchable data properly. o The level mode of exomizer now crunches each infile separately and appends them to the outfile instead of merging them together like the -r mode did before. o New raw mode that features: 1) Crunches plain files instead of .prg files. 2) Crunches forward or backwards instead of just backwards. 3) Also decrunches. 4) Handles files larger than 64k. The Canterbury and Calgary corpora is now possible to crunch. 5) Comes with two different C-source decrunchers, one thread safe forward pull decruncher and one low-level backwards push decruncher, perfect to base an assembler port on. o New desfx mode that features decrunching of sfx crunched files. o Contributed decruncher source 1) forward 6502 decuncher by Krill (exodecrs/krilldecr.s) 2) z80 decrunchers by Metalbrain, Antonio Villena. (rawdecrs/z80/*) 3) 6809 decruncher by Edouard Forler. (rawdecrs/6809/*) FEATURES PLANNED FOR NEXT STABLE RELEASE: o Multiple memory segments 1) Handle the vic20 target better with regards to memory holes. 2) Handle Atari target better with regards to RAM under OS-ROM. o There is no 6502 assembler decruncher for streaming data crunched forward crunched using literal sequences. However the streaming (backwards) decrunchers will work if the -b and -c flags are used when crunching (with exomizer raw). Please see the examples. Please note that unlike 1.x, only exomizer raw can crunch data intended to be used by the streaming decrunchers. o The sfx command sometimes crunch better with the -c flag. The cruncher doesn't take the sfx decuncher size increase due to enabling the literal sequence escape mechanism into consideration. ------------------------------------------------------------------------------- The command line syntax differs from v1.x. Here's a quick translation guide: old new ------------------ ----------------------------- exomizer -ssys exomizer sfx sys exomizer -s$1234 exomizer sfx $1234 exomizer -r exomizer level exomizer exomizer mem -lnone exomizer -l$1234 exomizer mem -l$1234 exomizer -lauto exomizer mem ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- -- A beginning of something like documentation -------------------------------- ------------------------------------------------------------------------------- A note about addresses: An address in this context is a positive 16-bit value [0 - 65536]. It can be given in decimal, hexadecimal (prefixed with $ or 0x) or octal (prefixed with 0) notation. Examples: 3, 54, $3456, 0x1111, 06543. All addresses that end an interval of any kind, printed by exomizer or given by the user as an argument, are exclusive. In other words they point to the address just following the last byte of the memory area the interval covers. Exomizer has four sub commands: mem, level, sfx and raw. They all compress data. They share the following option flags: ---------------- -c This flag enables 1.x compatibility mode. It disables the use of the literal sequence escape mechanism. It will make the output files compatible with Exomizer v1.x. -C This flag enables imprecise rle matching. It increases crunching speed but reduces the crunch result. -e Uses the given encoding parameters for crunching instead of trying to calculate an optimal set. The encoding must be in the following format: xxxxxxxxxxxxxxxx,xxxx,xxxxxxxxxxxxxxxx,xxxxxxxxxxxxxxxx where each x must be a hexadecimal character, 0-9 or A-F. The Exomizer prints the encoding parameters used when writing the output file in this format in order to allow for easy copy and paste. Please note that this option sets the initial encoding parameters and not the encoding parameters used in the output file. To set the encoding parameters of the output file this option must be used in combination with the -p 1 option. -m Limits the maximum sequence offset used by the cruncher to . The default limit is an offset of 65535 bytes. A smaller offset increases the crunching speed but reduces the crunch result. If a circular buffer decruncher is used it is important that the maximum offset doesn't exceed the buffer size. -M Limits the maximum sequence length used by the cruncher to . The default limit is an length of 65535 bytes. A smaller length increases the crunching speed but reduces the crunch result. -o Sets outfile name. -p Limits the maximum number of optimization passes the cruncher uses when calculating the encoding parameters. default is 65535. However the cruncher will always stop when no further improvement to the crunch result is made. -q Enables quiet mode which will disable display output. -v Displays Exomizer version, email address for support and the usage license. -- Will make all arguments that follow be treated as non-options, even if they begins with a - character. -? Displays a help screen that briefly describes the command line syntax and the valid options. Also for the sub command if one is given. ------------------------------------------------------------------------------- -- Information relevant for the mem, sfx and level sub commands --------------- ------------------------------------------------------------------------------- The mem, level and sfx command all crunches files backwards. The files have to be located in a 16-bit address space and are also limited in size to that 16bit address space. The file format of the input files are prg files, Atari xex files, Oric tap files or located plain files (plain files that are given an address to load to.) The syntax for loading a prg, xex or Oric tap file is simply the filename: $ exomizer mem -o If you want to load a prg file to a different address than it contains you can simply append a comma and the new load address: $ exomizer mem , -o Relocation does not work for xex or tap files. A relocated xex or tap file will simply be treated as if it was a relocated prg file. Plain located files are given using a similar syntax: $ exomizer mem @

-o When given multiple input files the sfx and mem commands loads all files to where they are located and then crunch the covered memory area. Any unused space between files will be zero-filled. The level command loads each input file separately, crunches it and then appends the crunched data to the output file. The autodetection of xex or Oric tap files is not perfect. Prg files that load at $ffff or $1616 will be wrongly detected as xex or Oric tap. To disable the autodetection relocate the prg files to the same address they normally load to. ------------------------------------------------------------------------------- -- The mem sub command -------------------------------------------------------- ------------------------------------------------------------------------------- The mem command generated files that are used for decrunching from memory. Normally these files are linked into the program images either by machine code monitors or by assembler directives like incbin. The mem command writes its output in prg format. It has the following option: -l

|none Sets the load address of the output prg file to

. If the string none is given the output will be a plain file. Defaults to the address that gives the crunched data as much overlap with the decrunched data as possible but still allow for in-place decrunching. However, the file doesn't need to be decrunched from that address. Any address that doesn't cause data being decrunched to overwrite data yet to be decrunched will work. -f Crunch the file forward instead of backwards that is the default. This means that in this mode the read and write pointers move forward in memory while decrunching. The exodecrunch.s does not support forward decrunching. For this feature to work You'll have to use Krill's contributed decruncher. ------------------------------------------------------------------------------- -- The level sub command ------------------------------------------------------ ------------------------------------------------------------------------------- The level command writes its output file so that the crunched bytes are returned in correct order for decrunching in the fly while streaming in the bytes from secondary storage. -f Crunch the file forward instead of backwards that is the default. This means that in this mode the write pointer move forward in memory while decrunching. The exodecrunch.s does not support forward decrunching. For this feature to work You'll have to use Krill's contributed decruncher. ------------------------------------------------------------------------------- -- The sfx sub command -------------------------------------------------------- ------------------------------------------------------------------------------- The sfx command generates a runnable stand alone in memory decrunching file. Its first argument must always be the run address. It may be given as an address or the string sys or the string basic. If the run address is an actual address then it will be used as the target address of a jmp instruction executed after the decrunching. If the run address is the string sys then the run address will be autodetected. Either from the runad vector for the Atari target or by scanning the area of the basic start for a sys basic command. If the run address is the string basic then the computer will run a basic program after decrunch. This is not yet implemented for the Atari target. The Oric target support basic start for the Oric 1 and Atmos computers but not for the Telestrat. The run basic mode takes up to three optional addresses. They are in order: start of basic program, end of basic program and highest address used by basic. $ exomizer sfx basic[,[,[,]]] The start of basic, , is the address where the basic program starts in memory. For many but not all targets it defaults to $0801. The end of basic, , is the address where the basic variables starts in memory. It defaults to the the address where the basic program ends. The highest address used by basic, , is the address where the basic variables end. Its default value depends on the target. The basic interpreter will write all variables, strings and arrays between the address and the address. Normally these addresses are autodetected and/or defaulted. However, if non standard settings are needed they can be changed. Examples of such situations could be when a basic program needs to be run at a non standard address and/or some graphic needs to be protected from basic variables. The output of the sfx command is a runnable program file that decrunches itself in memory when run. The output file format depends on the target used: For all Commodore machines the output is a runnable prg-file that starts with a basic line. For the Atari target the output is a auto running xex-file. For the Apple target the output is an Applesoft basic file without headers. For the Oric family the output is a Oric tap file. Please note that the sfx command doesn't support extended memory since it can't decrunch into banked memory. The sfx command has the following options: -t Sets the decruncher target. The possible targets are: 1: Oric 1, Oric Atmos and Telestrat. 20: Commodore Vic20, unexpanded memory. 23: Commodore Vic20, 3kB memory expansion. 52: Commodore Vic20, 32kB memory expansion. 55: Commodore Vic20 32+3kB memory expansion. 16: Commodore c16 4: Commodore plus4. 64: Commodore c64. (The default target) 128: Commodore c128. 162: Apple ][+ and //e 168: Atari 400/800 XL/XE. -X -x[1-3]| Sets the decrunch effect. If an assembler fragment is given, it must not change the X-register, Y-register or modify the carry flag. Separate instructions with a space. Please note that including spaces in the fragment requires quoting to work. An example of a slow C64 x-scroll-register effect: -X "txa and #07 ora #$c8 sta $d016" Use a value of 1 - 3 to pick a preset effect: 1: Accumulator based fast border flash 2: X-register based fast border flash 3: Y-register based fast border flash Default effect is a slow character flash. The 1-3 presets is only available for targets that supports changing border color. -n No decrunch effect. This option can't be combined with -X or -x -D= Defines symbols for the sfx assembler. More advanced features of the decruncher is controlled by defining symbols. More on this in the decruncher symbol reference section below. -s Specifies an optional assembler fragment to be executed when the decruncher starts decrunching. It must not change the Y-register. This assembler fragment is intended for things like clearing the screen, changing background and/or border color, changing a c128 to 2MHz mode or enable 'Skip internal cycles' and burst mode on a DTV2. The fragment syntax is the same as for the -X and -x options. -f Specifies an optional assembler fragment to be executed when the decruncher has finished decrunching. This assembler fragment is intended for things like restoring a c128 to 1MHz or disabling 'Skip internal cycles' and burst mode on a DTV2. The fragment syntax is the same as for the -X and -x options. ------------------------------------------------------------------------------- -- The desfx sub command ------------------------------------------------------ ------------------------------------------------------------------------------- The desfx command decrunches a file generated by the sfx command and write it as a prg-file. It autodetects the entry point from the in-file runs the embedded decrucher in order to get to the decrunched data. The sfx command has the following options: -e

|load Overrides the entry point of the decruncher with

or if the string load is given, the load address of the input file will be used. ------------------------------------------------------------------------------- -- The raw sub command -------------------------------------------------------- ------------------------------------------------------------------------------- Now let's go back to the command that handles plain files, raw. The raw command can both crunch and decrunch both forward and backward. It also handles generic files without any of the above mentioned restrictions. There's no hardcoded limit in file size but it gets increasingly slow with larger files so there's a practical limit of about 1MB. The output file format of this command is a plain file. It is possible to give an optional offset and length for the infile like this: [,offset[,length]]. Negative offsets are allowed. They will wrap around to the end of the file. To skip the first two bytes but read all the rest of a file do like this: $ exomizer raw ,2 -o To read 10 bytes at offset 13 from the end of the file do like this: $ exomizer raw ,-13,10 -o ------------------------------------------------------------------------------- -- The included decruncher source code with examples -------------------------- ------------------------------------------------------------------------------- There are two directories with source code in the distribution: exodecrs/ This contains the source code of four 6502 assembler decrunchers. One backwards decruncher that decrunches data crunched by the mem sub command. It also contains a forward decruncher contributed by Krill/Plush for data crunched by the mem sub command and the -f option. Call these two decrunchers once to decrunch the data to its targeted location. Also contained in this directory are two streaming forward decrunchers for data crunched with the raw sub command. Both use a circular buffer and has a get_byte like interface that are called repeatedly to get one byte or a block of decrunched data until the end of data is signalled by the call returning with the carry flag set. However they are not capable of handling the literal sequence escape mechanism so data must be cruncher with the -c flag. Also included are a Makefile and some example code and data that use the four decrunchers. The assembler syntax is ca65, the assembler of the cc65 tool chain. This tool chain is needed to build the examples. rawdecrs/ This directory contains two decrunches written in C. One backwards decruncher with a call once and all data are decrunched interface. And one forward streaming decruncher with a get_decrunched_byte interface. Also incuded are example code using the decrunchers and a Makefile that can be used for building them. An ANSI-C compiler are necessary for this. All of the included makefiles have only been tested with Gnu make. ------------------------------------------------------------------------------- -- Credits for Exomizer 2.x --------------------------------------------------- ------------------------------------------------------------------------------- Feature suggestions ------------------- Csabo / LOD Ninja / The Dreams Richard / TND Jack Alien / Remember Krill / Plush Contributed code ---------------- Krill / Plush Metalbrain Bug reports ----------- iAN CooG / HokutoForce Regression testing ------------------ Ninja / The Dreams Csabo / LOD iAN CooG / HokutoForce Lasse ��rni Inspiration ----------- PuCrunch The Oneway crunchers, packers and linkers, Byte Boiler, AB, cruel etc. The Final Super-Compressor by Meidinger & Borris Timecruncher 3.1 by Matcham of Network ------------------------------------------------------------------------------- -- Credits for Exomizer 1.x --------------------------------------------------- ------------------------------------------------------------------------------- Inspiration ----------- The Oneway crunchers: Byteboiler, The Cruncher AB and CruelCrunch. Pasi Ojala's PuCrunch Beta testing ------------ Lasse ��rni Csabo / LOD Overkiller / Hokuto Force / PoL iAN CooG / Hokuto Force Rafal Szyja bubis / Resource Ninja / The Dreams Bug Reporting ------------- Csabo / LOD Soci / Singular 6R6 / Nostalgia Ninja / The Dreams Feedback/Suggestions -------------------- Count Zer0 bubis / Resource Ninja / The Dreams ------------------------------------------------------------------------------- -- Contact information -------------------------------------------------------- ------------------------------------------------------------------------------- Please don't hesitate to contact me if you have any feature/improvement suggestions, bugs to report or perhaps just have something to say about the exomizer program. My name is Magnus Lind and my email address is magli143@gmail.com ------------------------------------------------------------------------------- -- Reference for the sfx decruncher symbols ----------------------------------- ------------------------------------------------------------------------------- The sfx decruncher is tweaked by defining optional symbols to the assembler on the command line using the -D= flag: The following symbol controls the sfx load address: i_load_addr If it is unset then the sfx file will load to the default basic start address of the target machine. For Atari the default load address is $2000. For the Oric the default load address is $0500. If it is set then the sfx file will load to the given address and it will not have any basic line. (The Atari target never has a basic line). The run address is the same as the load address. The Apple target file format will change from an Applesoft basic file without header to an Apple binary file with a four byte header where the first two bytes are 16-bit load address, little endian, and the next two are the length in bytes of the following data, little endian. This is the same 4 byte DOS 3.3 header that cc65 generates. AppleCommander 1.3.5 or later can be used to put these binary files onto DOS 3.3 as well as ProDOS 8 disk images using the -cc65 option. The following symbol controls the sfx basic line number: i_line_number If unset it will default to 20. This symbol can't be combined with i_load_addr. The following symbols control the memory config: i_ram_enter (defaults to standard ROM config) i_ram_during i_ram_exit (defaults to standard ROM config) The i_ram_enter symbol differs from the other two. It isn't used to set the memory config. It tells the exomizer of which memory config to expect when the decruncher is run. This information is used to minimize the decruncher size. For the c64 and the c128 target the value of the above symbols will be stored into $ff00 (c128) or $01 (c64) to set the mem config. For the c16/+4 target the possible values are 0=ROM and 1=RAM. For the Atari the value will be stored in $d301. The Oric target supports 0=ROM and 1=RAM where 1 will enable decrunching into the top 16kB RAM supplied by the microdisc controller. Default is $37 for the c64, $00 for the c128, 0 for the c16/+4 and Oric and $ff for the Atari. The following symbols control irqs. (0=disabled,sei, 1=enabled,cli) i_irq_enter (defaults to 1) i_irq_during i_irq_exit The following symbols control NMIs, this is only implemented for the Atari target where the value will be stored in $d40e i_nmi_enter (defaults to $40) i_nmi_during i_nmi_exit To locate the decrunch table elsewhere than the default tape buffer or $0600 for The Atari target, use the symbol i_table_addr: -Di_table_addr=

The apple target has an option to disconnect DOS. To do this, set the i_a2_disable_dos symbol to any value: -Di_a2_disable_dos=1 Effects are no longer selected by the use of symbols. They are now chosen by the -x or -X flags or deactivated by -n. ------------------------------------------------------------------------------- -- Results for the Canterbury and Calgary corpae for exomizer raw ------------- ------------------------------------------------------------------------------- ------------------------------------------------------ Canterbury in out b/B ratio gained ------------------------------------------------------ alice29.txt 152089 53060 2.79 34.89% 65.11% asyoulik.txt 125179 48530 3.10 38.77% 61.23% cp.html 24603 8352 2.72 33.95% 66.05% fields.c 11150 3177 2.28 28.49% 71.51% grammar.lsp 3721 1294 2.78 34.78% 65.22% kennedy.xls 1029744 162282 1.26 15.76% 84.24% lcet10.txt 426754 138443 2.60 32.44% 67.56% plrabn12.txt 481861 190570 3.16 39.55% 60.45% ptt5 513216 51904 0.81 10.11% 89.89% sum 38240 12316 2.58 32.21% 67.79% xargs.1 4227 1829 3.46 43.27% 56.73% ------------------------------------------------------ total 2810784 671757 1.91 23.90% 76.10% ------------------------------------------------------ Calgary in out b/B ratio gained ------------------------------------------------------ bib 111261 34219 2.46 30.76% 69.24% book1 768771 305695 3.18 39.76% 60.24% book2 610856 198396 2.60 32.48% 67.52% geo 102400 68268 5.33 66.67% 33.33% news 377109 139442 2.96 36.98% 63.02% obj1 21504 10137 3.77 47.14% 52.86% obj2 246814 78902 2.56 31.97% 68.03% paper1 53161 18659 2.81 35.10% 64.90% paper2 82199 29546 2.88 35.94% 64.06% paper3 46526 18309 3.15 39.35% 60.65% paper4 13286 5722 3.45 43.07% 56.93% paper5 11954 5165 3.46 43.21% 56.79% paper6 38105 13536 2.84 35.52% 64.48% pic 513216 51904 0.81 10.11% 89.89% progc 39611 13591 2.74 34.31% 65.69% progl 71646 16198 1.81 22.61% 77.39% progp 49379 11129 1.80 22.54% 77.46% trans 93695 18677 1.59 19.93% 80.07% ------------------------------------------------------ total 3251493 1037495 2.55 31.91% 68.09%