# Copyright(2000): Dutch Forth Workshop Foundation, by GNU Public License # $Id: testreport.txt,v 5.7 2014/06/18 21:29:27 albert Exp $ Test report of generic forth's for Intel 86 family. First part for ISO forth's, in the appendix some figforth's. Summary This is a test report for 12 out of a zillion Forths that can be generated from fig86.gnr and ci86.gnr . Only the hosted version have a a automated regression test. The booting version take more time to test. However, please tell me if any of the following tests fails you, where it doesn't fail me. It looks as if sometimes K' fails. But not for a freshly booted system. If you have observation with regard to this, or even insight, or even a fix, I urge you to inform me. Stable version 5.1.0 Suse ( and XP) Still s| and D| conflivt in the assembler. error 13 retained. Stable version 5.0.0 Suse ( and XP) Stable version 4.0.6 Suse and XP The redirection on XP doesn't work for the command interpreter: it takes over the file from which commands are read unconditionally, and doesn't head the /c option and the subsequent commands. TICK on MSDOS, see 4.0.5 Stable version 4.0.5 Suse and XP MEASURE-PRIME on the Celeron works flawlessly now. (The reason it failed was not the lack of instruction 0F 31.). MEASURE-PRIME on MSDOS fails to the effect that TICK 's is not implemented on MSDOS (yet). Stable version 4.0.4 Suse and XP This stable version was used internally because it had improvements related to timing, and was never officially released and tested. Stable version 4.0.3 Suse and XP The XP version failed the MEASURE-PRIME because it was a Celeron system without the clock count instruction 0F 31 . (Apparently for some other reason because the Celeron has that instruction.) Stable version 4.0.1 Suse Environment 4.0 S.u.S.e. ciforth, an ISO forth. With the same hardware configuration as above. Environment 3.0 ... ciforth, an ISO forth. With the same hardware configuration as above. Environment 2.147 .... 3.0 fig-Forth GNU/Linux 2.2.14 Redhat 6.1 (Pentium 90) development&test MSDOS 3.3 (Pentium III "Coppermine") MASM 1.0 Copyright IBM 1981 (!) nasm 0.98 In this environment the figforth (linux with c input) crashes immediately. Even if an older version RELEASE4 is built. Environment ... 2.147 fig-Forth GNU/Linux 2.0.34 Redhat 5.1 (Pentium 90) development&test GNU/Linux 1.2.13 Slackware (test of lina) MSDOS 3.3 (Pentium III "Coppermine") MASM 1.0 Copyright IBM 1981 (!) nasm 0.98 The following tests are defined: (USED for 4.0.# and 5.#.# After 4.0 WANT A Booting/starting up B make it C List and load the elective screen (8 before 3.223, now 5) D after 2.223 : REQUIRE SIEVE 0 PAUSE ! 100 SIEVE before 2.223 : REQUIRE ERATOSTHENES 0 PAUSE ! 100 SIEVE after 5.62 : WANT ERATOSTHENES 0 PAUSE ! 100 SIEVE primes ending in 99971 99989 99991 E (requires A) KRAAK QUIT (5.x.x SEE QUIT) F REQUIRE MEASURE-PRIME (does nothing for 16 bit, assembly code) After 5.62 : CASE-INSENSITIVE WANT MEASURE-PRIME G (r. A) Editing a screen 0 PAUSE ! 100 SIEVEter 5.62 : ``mywc *.asm''. J Binary equivalent than compiled on linux (for windows Forths and NASM) K BYE L Dos Directory correct M Documentation complete N "ls -l" SYSTEM or "dir" SYSTEM O Built in regression test P Johns Hopkins university test (always with restrictions) R Assemble empty code definition Z The assembly source of this version is equivalent with previous version Before 3.223 The following tests are defined: A Booting B make it C 5 LIST 5 LOAD D (requires A) 25 LOAD 0 PAUSE ! 100 SIEVE primes ending in 99971 99989 99991 E (requires A) KRAAK QUIT F ( 89 LOAD (does nothing for 16 bit, assembly code)) G (r. A) Editing a screen H wc *.asm after 5.x named mywc J Binary equivalent than compiled on linux K BYE L Dos Directory correct / WANT ls ls M Documentation complete N "ls -l" SYSTEM or "dir" SYSTEM O Built in regression test P Johns Hopkins university test (always with restrictions) R Assemble empty code definition Pass criteria A booted OK B I made it through make (version can be built). C plausible content of 8, everything loads fine, shows copyright screen D primes less then 100000 zip accross the screen E plausable decompilation of QUIT F timing of Byte benchmarks, must be 7 ms for a 500 Mhz Coppermine G Effect of cursor (^E ..) word (^A ^F ^T) (un)delete (^Y ^U) H Same as /bin/wc J Binaries are equal (apart from version stamp) or J' disassembly reveals only gratuitous MASM deviations. K successful return to operating system (linux) or K' successful return to "operating system" (msdos) L Dos Directory correct on generated floppy. M .html .texinfo (orps/pdf) and .info file generated correctly. (Not content!) O No difference with built in test output found(lina) O' Expected differences with built in test output found P Output of test has only the expected restrictions R Assembler loads, and the empty definition runs. Test are always mentionned if applicable. A failed test is preceeded by -. Not all tests are always down. The following environments are defined (Only for BOOTFD BOOTHD): I nasm version , Booted from FD or HD II nasm version, started from MSDOS III build with MASM 1.0 , Booted from FD or HD IV build with MASM 1.0 , started from MSDOS III (Other environments following from the executable itself). Notes about ommitted data: - all linux versions use NASM PROTECTED BITS32 - all booting versions use NASM BIOS - the MASM versions all are MSDOS-hosted and real mode This table can be used to decide whether you want to run an older better tested version. Data for older versions is not present if the new version passed all tests the older one does. FAILED TESTS ARE PRECEEDED with -. This test is for the ISO ciforth , version 3.### ONLY THE LATEST VERSION OF lina IS TESTED for snapshots. The latest versions of other configurations are trustworthy, because the changes only relate to the general part. ) Description Latest Configuration Executable Tests test file name hosted_win32 windowsXP 5.1.0 wina.cfg wina ABCDEF-HKMNOPR hosted_linux linux_64 5.1.0 lina64.cfg lina ABCDEFHKmNOP-R hosted_linux linux_32 5.1.0 lina32.cfg lina32 ABCDEFHKmNOPR hosted_linux linux_native 5.0.0 lina.cfg lina BCDEFHKLmNOPR hosted_linux linux_native 4.0.6 lina.cfg lina BCDEFHKMOPR hosted_linux linux_native 4.0.5 lina.cfg lina BCDEFHKMOPR hosted_linux linux_native 4.0.3 lina.cfg lina BCDEFHKMOPR 4.0.1 lina.cfg lina BCDEFHKMOPR hosted_linux linux_native 4.0 lina.cfg lina BCDEFHKMOPR 275 lina.cfg lina BCDEFHKMOPR 223 lina.cfg lina BCDEFHKMOPR 189 lina.cfg lina BCDEFHKMOPR 181 lina.cfg lina BCDEFHKMOPR 176 lina.cfg lina BCDEFHKMOPR 154 lina.cfg lina J 153 lina.cfg lina BCDEFHKMOPR 135 BCDEFHKMOP dpmi ("windows" 32 XP ) 4.0.6 wina.cfg WINA.COM ABCDEFGKLMOPR 4.0.5 wina.cfg WINA.COM ABCDEFGKLMOPR dpmi ("windows" 32) 4.0.1 wina.cfg WINA.COM BCDEFGLMOR 4.0 wina.cfg WINA.COM BCDEFGLMOR 275 wina.cfg WINA.COM BCDEFGLMOR hostedmsdos real modern 4.0.6 mina.cfg mina.com II B CDE -FG K' LMNOPR (tested on dosemu 1.0.2.0) 4.0.5 mina.cfg mina.com II B CDE -FG K' LMNOPR hostedmsdos real modern 4.0.1 mina.cfg mina.com II B CDEG K' O P R 4.0 mina.cfg mina.com II B CDEG K' O P R 275 mina.cfg mina.com II B CDEG K' O P R 275 mina.cfg mina.com II B CDEG K' O P R 224 msdos.cfg msdos.com II B CDEG K' O P R 154 msdos2.cfg msdos.com J' 153 msdos2.cfg forth.com IV B CDEG K' -O -P R hostedmsdos real bios 153 msdos1.cfg msdos.com II B CDEG K' R hosted_linux linux_c 4.0 linux.cfg (ciforthc) BCDEFK O'PR 223 linux.cfg (ciforthc) BCDEFK O'PR 189 linux.cfg (ciforthc) BCDEFK O'PR 154 linux.cfg (ciforthc) J' 153 linux.cfg (ciforthc) BCDEFKMPO'R bootfd real 224 alone1.cfg forth.com B I -A B II ACDEG K' R N.A. alone1.cfg forth.com I B -A II B -A bootfd prot16 4.0 alone.cfg forth.com B I ACDEFGR II L 1) 223 alone.cfg forth.com B I II A-C-DE-G-K'L-R 1) 176 alone.cfg forth.com B I A-CDEGR II ACDEGK'LR 154 alone.cfg forth.com J' 153 alone.cfg forth.com B I ACDEGR II ACDEGK'LR bootfd prot32 4.0 alone3.cfg forth.com B I ACDEFG M R II ACDEF-G K'MR 5) 6) 224 alone3.cfg forth.com Z 223 alone3.cfg forth.com II ABCDEF-GK'LR 5) 153 alone3.cfg forth.com B I ACDEFG R 3) II ABCDEFGK'LR boothd prot32 4.0 alonehd.cfg no filesystem ABCDEFG M R 223 alonehd.cfg no filesystem ABCDEFG M R 189 alonehd.cfg no filesystem ABCDEFG M -R' 4) 177 alonehd.cfg no filesystem ABCDEFGMR 154 alonehd.cfg no filesystem J' 2 153 alonehd.cfg no filesystem ABCDEFGMR hostedmsdos classic N.A. msdos3.cfg msdos.com Not built hostedmsdos modern 32 N.A. msdos9.cfg forth.com II BCDEFGK'MLR 3) Versions are 3.xxx unless a dot is present. N.A. : not available. L* : 1) This requires a special CONFIG.SYS/AUTOEXEC.BAT to free memory <64K 2) O and P not tested, because these version have no redirection. 3) Editor is sometimes unreliable ^U hangs. 4) Only Postscript file fails. 5) Most editor commands work ^T fails or a hangup occurs. 6) Floppy is extremely slow. APPENDIX The test for the latest version that conformed to the fig standard. This table can be used to decide whether you want to run an older better tested version. Data for older versions is not present if the new version passed all tests the older one does. This test applies to the fig-Forth fig86.gnr version 2.### Description Latest Configuration Executable Tests test file name hosted_linux linux_native 147 lina.cfg (lina) BCDEFHKM hosted_linux linux_c native linux.cfg (figforth) BCDEFHK 147 B-C-D-E-F-H-K M bootfd real 102 alone1.cfg forth.com B I ACDEG II CDEG K'L 147 BM bootfd prot16 147 alone.cfg forth.com BM I ACDEG II CDEG K'L hostedmsdos modern 147 msdos.cfg msdos.com BM II CDEG K' L bootfd prot32 116 alone3.cfg forth.com B I ACDEG L 116 II CDE-G K' L boothd prot32 148 alonehd.cfg no filesystem B I ACDEFG hostedmsdos MASM bios 102 msdos1.cfg msdos.com B II CDEG K' L hostedmsdos MASM modern 102 msdos2.cfg msdos.com B II CDEG K' L hostedmsdos MASM classic 102 msdos3.cfg msdos.com B II CDEG K' L* hostedmsdos MASM modern 32 112 msdos9.cfg forth.com B II CDEF-G -K' Version 2.114 apparently in alone3 cannot handle screen deleting lines, otherwise reasonable. Version 2.147 has the same assembly source file than 2.148 whereever it has been tested. (Modification was needed for alonehd.cfg. Afterwards 2.148 has been checked in and it was checked that the source was equivalent to the one tested. (i.e. after removing comment lines the source was the same, or differed at most with respect to strings printed.) All MASM version generate real code. I have to boot into MSDOS to test this so these are really old version. N.A. : not applicable. L* : The classic version has been tested after properly setting OFFSET to compensate for the fact that the BLOCKS.BLK was not simple dumped on the floppy from sector 0. In that case the MSDOS directory would not have remained intact.