# Impetus I fell in love with Forth in the 1980's. I had been writing code since the early 70's, starting by toggling in binaries on the front panel of a derelict Raytheon 0703 that someone had donated to my high school. I dropped out of college to pursue a career in IT at a time when formal education was rare (and often irrelevant) and the door was wide open to people like me who could do hardware design and programming. By the late 70's I had written a compiler, several operating systems, and some vertically-integrated production management systems. I always felt that the tools I had more often got in the way rather than facilitated development. Yes, a good high level language could get you off the ground a lot faster, but when it came to fine-tuning and really getting the job done well, the second 90% of the projects often seemed to be more about fighting the development environment (too abstract, too resource-hungry, too buggy) and I'd find myself wishing I'd just coded the whole damn thing in assembly. Then I discovered Forth, and it fit perfectly with my sense of style. It had a tiny resource footprint, it was trivial to break through abstraction when necessary (which wasn't all that often), and I soon had my own highly-portable implementation which meant that I was responsible for all the bugs. Suddenly I was completing projects in the amount of time my intuition said they should take, instead of the "take your best guess, double it, and move to the next unit of measure" formula which generally seemed to apply. Starting from bare iron, I wrote a complete development environment for a new processor family in two days: it included a multi-tasking OS, "graphical" (character-based) user interface, an assembler, and a Forth environment. The OS and assembler took one day, the rest of the environment was completed the next. The customer had allocated *months* to the job. My code from that era ended up in the Space Shuttle launch systems, in automobile ignitions, in industrial controllers, in embedded telecommunications devices, on mainframes, processing mortgages, running courtrooms. Life was good. Operating systems grew more sophisticated, and the tools I was using were getting left behind. I wasn't happy with the graphical environments for Forth on Mac OS and Windows in the 1980's, and I was doing more punditry than actually coding at the time. When Apple released HyperCard, I thought that might make a fun basis for a Forth-based IDE. Before I really got the project off the ground, however, Life Happened and I ended up going back to college, getting an MD, starting a medical practice, raising children, writing a web-based EMR, and building and maintaining a stealth infosec distributed telecommunications network. But I still am unhappy with development environments. Every time I find myself writing code in Perl or Java or C or C++ or Python or Bash or Swift, I get a nostalgic longing for the days when my own Forth environment was like an extension of my psyche. Now my children are teens, my medical practice is stable and even slightly profitable, and my IT business is stable, I've decided to see whether my fond memories of Forth are just nostalgia for a bygone era, or if a highly-interactive version of Forth built on a web-based interactive platform could really restore the sense of productivity I had working on my 1980's-era Forth IDE. # Ground Rules ## web-based I expect to have this running as a simple web service accessible from a stock-standard HTML5 browser using minimal if any Javascript (my goal is 100% functionality with Javascript disabled, but I might use some Javascript adornments). ## lightweight I want a system that can run reasonably well on embedded systems such as a Raspberry Pi Zero W (I'll be bringing up the prototype on just such a system today). It should be tolerant of slow net connections and require minimal storage. ## portable The old Nostalgiaforth system was insanely portable. I was once developing on a Z-80 based S-100 system, and pulled the 8" floppy disk out and put it in a Motorola Exorcisor (68000-based) system, and it booted and ran. Slowly. Is such a thing necessary in this age when 86-based architectures and ARM together probably cover 98% of use cases? No, and it wasn't even that important back then. But I found that, often, the ability to fall back on that hyper-portable reference implementation was quite useful when chasing down hardware problems or trying to find unusual ways to optimize code. Nostalgiaforth had 17 primitives (*very*-primitives) that required implementation in native code. On a new platform, I'd usually code the primitives in assembler (or even machine code if an assembler wasn't yet available) in a matter of hours. The result was a running, if sluggish, system. Optimization could then proceed using the internal performance monitoring code to identify high-value targets for low-level implementation and unrolling. (I once implemented the 17 primitives in semi-custom silicon to make an actual Forth engine in hardware. Good times.) I would like to preserve a similar level of portability. Partly, it's just an exercise. Like writing a sonnet, I find that sticking to an overly-restrictive set of rules can clarify my thinking. At the same time, I would really like to minimize the impact of my early implementation choices. I've written an inner interpreter in Perl because it was the easiest for me to integrate with my CGI environment (Sacdoc, my EMR, is mostly written in bad Perl so I have a lot of experience with it). But I in no way want 4gi to be a Perl- or Python- or Swift- or C++- s or Java-based implementation. I anticipate fewer than 100 lines (I hope *far* fewer) of native code to get the thing running, and I can translate those to native x86 or ARM machine code as the need arises. I would like the environment to be suitable for smartphone development (though I doubt it will ever meet Apple's App Store requirements), embedded systems, and web applications. ## open It's all open source, all the way. This is how it must be. ## secure IT security has been a focus since I first hacked my way into a PDP-8 running TSS/8 at my school in 1974. I've been hacking ever since, and if there's one thing I've learned it's that you can't implement security as an overlay. 4gi will have security baked in from the start. >Aside: There's nothing new under the sun. The Forth engine in silicon that I mentioned above never went into full production. The reason was that it used a highly parallel architecture that simultaneously followed multiple branches of code; simplistic by today's standards but "speculative execution" nonetheless. My good friend Jim Valesh was reviewing the design, and said "how are you going to prevent cached data from the stale branches from leaking protected memory information?" Sigh. We ended up prototyping with the cache completely disabled which caused a colossal (10x?) performance hit. We ran out of time, money, and interest before we solved that problem. Today, I see people saying that the Meltdown and Spectre processor bugs represent "new science" and that "nobody could have anticipated" those risks of speculative execution. Bullshit. Two podunk nobodies identified the problem *in 1984* and it has come up in other projects since. I'm certain that at least one engineer at each of Intel, AMD, and ARM was screaming "you can't do that!" and being answered with "everybody else is doing it; we can't afford to worry about that purely theoretical risk." If those emails ever surface... # Implementation ## platform Per above, I want to maintain an impossible degree of platform independence. I'm making choices based on expedience and comfort secure in the knowledge that I'm not locking myself in to a given programming environment, web server, operating system, or hardware environment. Here's what I've got going right now: * Raspberry Pi Zero W * Raspbian OS * ngnix * uWSGI * perl v5.24 ## valence One idea that I've toyed with is to assign each defined word a "valence," or net stack behavior. The goal is to allow stack-runaway (overflow or underflow) conditions to be detected at compile time. For me, stack overflow and underflow is the most common runtime failure mode for my Forth code. Is it possible to write Forth definitions in a way that each word will *always* have a fixed, predictable effect on stack depth? I suspect I'll find out that it's impossible or impractical, but I'm going to give it a try. ## stubbing I want to dispel the notion that code can or should be written top-down or bottom-up. I want to be able to take my inspiration for a definition and just type it at full speed. Then when I compile it, I'll get a list of words I used that are not yet defined. With a single click, I can create a stub (typically just the word name, valence, and human-readable stack description). So I can very quickly create a function, fix typos, stub new definitions, then finish the definitions iteratively until the code works. ## debugging I've always been happy with pretty simple debugging. Display the stacks and allow stepping through a definition or diving into a reference. Perhaps some simple breakpoints. ## interface For the first draft, I'm going to make it mimic Nostalgiaforth as closely as I can remember it. (It's possible the actual Nostalgiaforth code is on 8" or 5.25" floppies somewhere in my garage but, really, finding the disks and a way to access them just isn't very appealing.) Once I have that basic functionality, I'll start working on making it look more like a 21st century project. [--Ron Risley](mailto:ronq4gi@risley.net) 03February2018 # Development Log ## 03February2018 I've been thinking about this off-and-on since I first started CGI programming in the mid 1990's, but there always seemed to be so many other things that needed to be done first. Today, the time seemed right. Probably because I've had a bad cold for the past week and have a ton of work to catch up on and income taxes to file, so what better way to procrastinate? So today I wrote this tome to try to solidify the thoughts that have been flitting through my brain. I mocked up what I thought the user interface (web page) might look like: ![4gi interface page](https://4gi.wtf/4gi.png) I've sketched out the design of the inner interpreter in Perl and have some vague notions about what the Perl version of the outer interpreter will look like. The outer interpreter will need to be re-coded in Forth pretty quickly, but I suspect the inner interpreter (and "Native" word definitions) will probably remain Perl-based until performance issues demand that it be tuned. Oh, I registered the [4gi.wtf](https://4gi.wtf) domain and published this document. I've configured a Raspberry Pi Zero W with [nginx](https://www.nginx.com) and [uWSGI](https://uwsgi-docs.readthedocs.io/en/latest/) to serve the environment. I should probably get back to my taxes and paying work. My world tends to run on a fortnightly schedule, so the next update is probably two weeks away. Once I have another day to spend on this, I should have an interactive Forth environment running slowly and roughly. Let the fun begin. ## 14February2018 A pesky, lingering cold has kept me from being very productive, but given me some extra time to think. I realize that time has blurred the memory of how much work went in to creating the Nostalgiaforth model. If I have it at all, it's on obsolete media buried in my garage. It's a testimony to the power of Forth that I really gave little thought to the Nostalgiaforth code once I'd written it. Porting to a new environment meant just worrying about some primitives -- the bulk of the code just ported in source form and ran. I don't want to reinvent the wheel here, though. I went poking around online and found the still-archived [Forth Interest Group](http://www.forth.org) web site. I was able to download a copy of the fig-FORTH glossary, which is what I originally based Nostalgiaforth on. It was nostalgic, indeed, poring through that old stapled-together collection of typewritten pages. It brought back many memories. So it appears that I'll be basing 4gi on the old fig-FORTH code. I don't plan to slavishly copy it -- improvement and innovation are what Forth is all about -- but I have already designed the editor in the original 16 line by 64 character format of the FIG block editor. The (currently Perl-based) inner interpreter uses a 32 bit data and addressing path, though, and uses separate memory spaces for threaded code, native code, stack, and return stack. Doing so will allow me to implement a more robust security model when I transition from Perl to native ARM code. My fortnightly break will occur in a few more days and I hope to get the outer interpreter running and the inner interpreter prototyped over the weekend.