<!-- Forthmacs Formatter generated HTML output -->
<html>
<head>
<title>Vocabularies and Search Order</title>
</head>
<body>
<h1>Vocabularies and Search Order</h1>
<hr>
<p>
This chapter is a tutorial describing vocabularies and search order control.  It 
also describes the contents and purpose of each of the vocabularies that are 
part of the normal Risc-OS Forthmacs system.  
<p>
A vocabulary is a list of Forth words.  The Forth 83 standard defines a 
vocabulary as: 
<p>
A vocabulary is an ordered list of word definitions.  Vocabularies are an 
advantage in separating different word definitions that may have the same name.  
More than one definition with the same name can exist in one vocabulary.  The 
latter is called a redefinition.  The most recently created redefinition will be 
found when the vocabulary is searched.  
<p>
The most important vocabulary is  <code><A href="_smal_BR#209"> forth </A>.</code> 
The  <code><A href="_smal_BR#209"> forth </A></code> vocabulary contains most of 
the familiar words such as  <code><A href="_smal_BP#1d7"> dup </A>,</code>  <code><A href="_smal_AW#2e6"> swap </A>,</code>  <code><A href="_smal_AT#1c3"> do </A>,</code> 
etc.  Another vocabulary is  <code><A href="_smal_AD#153"> assembler </A>,</code> 
containing words used to assemble machine code.  
<p>
When the Forth interpreter receives a word, either one that you have typed at 
the keyboard or one that it gets from a file, it will look for that word in 
several vocabularies.  The ordered list of vocabularies to search is called the 
"search order".  A word will not be found unless it is contained in one of the 
vocabularies in the search order.  The search order may be changed at any time.  
<p>
To display the search order, use  <code><A href="_smal_BE#28c"> order </A>.</code> 
Example: 
<p>
<br><code>                 order</code><br>
<br><code>                 context: forth forth root       current: forth</code><br>
<p>
In this example, the search order is  <code><A href="_smal_BR#209"> forth </A></code>  <code><A href="_smal_BR#209"> forth </A></code>  <code><A href="_smal_AX#2b7"> root </A>,</code> 
as shown after the heading "context:".  The "context" search order is the list 
of vocabularies that is searched when the interpreter is deciding what to do 
with at word.  The reason that  <code><A href="_smal_BR#209"> forth </A></code> 
appears twice in the search order will be explained later.  The significance of 
the  <code><A href="_smal_AX#2b7"> root </A></code> vocabulary will also be 
explained later.  
<p>
After the heading "current:", the "compilation vocabulary" is shown.  The 
"compilation vocabulary", also called the "current" vocabulary, is the 
vocabulary where newly-created definitions appear.  The compilation vocabulary 
need not be the same as one of the vocabularies in the search order, although in 
practice it usually is.  
<p>
<p>
<h2>Setting the Search Order</h2>
<p>
When a vocabulary is executed, it replaces the first vocabulary in the search 
order.  Note the effect of this sequence of commands: 
<p>
<br><code>                 order</code><br>
<br><code>                 context: forth forth root     current: forth</code><br>
<br><code>                 assembler order</code><br>
<br><code>                 context: assembler forth root     current: forth</code><br>
<p>
See how the execution of  <code><A href="_smal_AD#153"> assembler </A></code> 
replaced the first vocabulary.  
<p>
If you want to add vocabularies to the search order, use the word  <code><A href="_smal_BU#14c"> also </A>.</code>  <code><A href="_smal_BU#14c"> also </A></code> 
duplicates the first entry in the search order, thus making the search order 
contain one more vocabulary.  Continuing with our example, if we now execute: 
<p>
<br><code>                 also  order</code><br>
<br><code>                 context: assembler assembler forth root     current: forth</code><br>
<p>
Notice that the search order now contains 4 vocabularies instead of 3, and that 
the  <code><A href="_smal_AD#153"> assembler </A></code> vocabulary appears 
twice.  It is okay to have a vocabulary listed twice in the search order.  The 
Forth word  <code><A href="_smal_BK#202"> find </A>,</code> which uses the 
search order to search through several vocabularies, is smart enough to 
recognise the duplication and to avoid the wasted effort of searching that 
vocabulary twice.  
<p>
If we execute another vocabulary now, it will replace the duplicate  <code><A href="_smal_AD#153"> assembler </A></code> 
entry.  
<p>
<br><code>                 hidden order</code><br>
<br><code>                 context: hidden assembler forth root     current: forth</code><br>
<p>
Don't worry about the significance of the  <code><A href="_smal_AO#21e"> hidden </A></code> 
vocabulary just yet; we will get back to that later.  
<p>
At this point, with the search order being  <code>"<A href="_smal_AO#21e"> hidden </A></code>  <code><A href="_smal_AD#153"> assembler </A></code>  <code><A href="_smal_BR#209"> forth </A></code>  <code><A href="_smal_AX#2b7"> root </A>"</code> 
, whenever you try to execute a word, the interpreter will first look for the 
word in the  <code><A href="_smal_AO#21e"> hidden </A></code> vocabulary.  If it 
doesn't find the word in  <code><A href="_smal_AO#21e"> hidden </A>,</code> next 
it will look in  <code><A href="_smal_AD#153"> assembler </A></code> , then  <code><A href="_smal_BR#209"> forth </A>,</code> 
and finally in  <code><A href="_smal_AX#2b7"> root </A>.</code> Whenever the 
interpreter finds the word, it stops looking.  If it doesn't find the word in 
any of those vocabularies, it checks to see if the word is a number.  If the 
word is a number, the interpreter pushes the number on the stack.  If the word 
is not a number, the interpreter displays a question mark (?) and gives up.  
<p>
Now suppose you want to shorten the search order, instead of adding things to 
it.  The most common way to do this is the word  <code><A href="_smal_BB#289"> only </A>.</code>  <code><A href="_smal_BB#289"> only </A></code> 
shortens the search order all the way, so that it only contains the  <code><A href="_smal_AX#2b7"> root </A></code> 
vocabulary.  
<p>
<br><code>                 only  order</code><br>
<br><code>                 context: root root     current: forth</code><br>
<p>
The  <code><A href="_smal_AX#2b7"> root </A></code> vocabulary contains only a 
very few words, just those words necessary to control the search order.  
<p>
<br><code>                 WORDS</code><br>
<br><code>                 definitions   forth         previous      seal</code><br>
<br><code>                 except        only          also</code><br>
<p>
These are all the words contained in the  <code><A href="_smal_AX#2b7"> root </A></code> 
vocabulary.  As you have probably guessed, the  <code><A href="_smal_AJ#339"> words </A></code> 
command displays the names of all the words in a particular vocabulary.  
Specifically,  <code><A href="_smal_AJ#339"> words </A></code> displays just 
those words in the first vocabulary of the search order.  In this case, we have 
displayed the words in the  <code><A href="_smal_AX#2b7"> root </A></code> 
vocabulary.  
<p>
Once  <code><A href="_smal_BB#289"> only </A></code> has reduced the search 
order to contain just  <code><A href="_smal_AX#2b7"> root </A>,</code> we can 
use  <code><A href="_smal_BU#14c"> also </A></code> and vocabulary names to 
build any search order we want.  
<p>
To summarise what we have seen so far: 
<p>
<p><pre>
        &lt;vocabulary-name&gt;       Makes the specified vocabulary the first
                                one in the search order, replacing the one
                                that was previously first.
        order                   Displays the search order.
        also                    Duplicates the first vocabulary in the
                                search order, increasing the number of
                                vocabularies in the search order by one.
        only                    Reduces the search order to contain only
                                the ROOT vocabulary.
        words                   Displays the names of all the words in
                                the first vocabulary of the search order.
</pre><p>
<p>
<p>
<h2>Compilation vocabulary</h2>
<p>
Now back to the compilation vocabulary, the one displayed after the "current:" 
heading.  Whenever you define a new Forth word, for instance a colon (:) 
definition or a  <code><A href="_smal_BN#325"> variable </A>,</code> the new 
word is entered into the compilation vocabulary.  The compilation vocabulary may 
be changed with  <code><A href="_smal_AH#1b7"> definitions </A>.</code>  <code><A href="_smal_AH#1b7"> definitions </A></code> 
changes the compilation vocabulary to be the same as the first vocabulary in the 
search order.  
<p>
<br><code>                 order</code><br>
<br><code>                 context: root root     current: forth</code><br>
<br><code>                 definitions</code><br>
<br><code>                 context: root root     current: root</code><br>
<p>
Now new definitions will be entered into the  <code><A href="_smal_AX#2b7"> root </A></code> 
vocabulary, as indicated by "current: root".  You probably won't need to put any 
new definitions in the  <code><A href="_smal_AX#2b7"> root </A></code> 
vocabulary, but you could if you wanted to.  
<p>
Let's get back to a more reasonable search order; right now the only vocabulary 
in the search order is  <code><A href="_smal_AX#2b7"> root </A>,</code> which 
doesn't contain very many interesting words.  
<p>
<br><code>                 forth also order</code><br>
<br><code>                 context: forth forth root     current: root</code><br>
<p>
It's frequently a good idea to execute  <code><A href="_smal_BU#14c"> also </A></code> 
after  <code><A href="_smal_BR#209"> forth </A>,</code> so that the next 
vocabulary won't totally remove  <code><A href="_smal_BR#209"> forth </A></code> 
from the search order.  Most of the time, you want  <code><A href="_smal_BR#209"> forth </A></code> 
to stay in the search order because  <code><A href="_smal_BR#209"> forth </A></code> 
contains most of the basic words that you use a lot.  
<p>
We are still not quite in a normal state, because the compilation vocabulary is  <code><A href="_smal_AX#2b7"> root </A>,</code> 
which is generally not a popular place for new words.  To get into the normal 
state, we need: 
<p>
<br><code>                 definitions  order</code><br>
<br><code>                 context: forth forth root     current: forth</code><br>
<p>
Now we are back to the situation that prevails when Forth is first started.  We 
got there by a tedious route, one step at a time, looking at the search order 
after nearly every step.  The easy way to get there is to execute this phrase: 
<p>
<br><code>                 only forth also definitions</code><br>
<p>
This is a very commonly-used phrase.  It gets you back to the "normal" search 
order after you have been mucking around with the search order for some reason 
or another.  It is important to understand how this works; if it isn't clear to 
you, reread this chapter, then try executing the phrase one word at a time, 
executing  <code><A href="_smal_BE#28c"> order </A></code> after each word.  
<p>
<p>
<h2>Creating New Vocabularies</h2>
<p>
So far we have been using vocabularies that are already in the Forth system.  
You can make your own vocabularies too.  But first, lets get a list of all the 
vocabularies already in the system.  
<p>
<br><code>                 vocs</code><br>
<br><code>              bug ignoredom command-completion keys-forth terminals</code><br>
<br><code>                 system assembler hidden root forth</code><br>
<p>
Note the distinction between the search order, which is the list of vocabularies 
that are being searched, and the list displayed by  <code><A href="_smal_BS#32a"> vocs </A>.</code>  <code><A href="_smal_BS#32a"> vocs </A></code> 
displays the names of ALL vocabularies, regardless of whether or not they are in 
the search order right now.  
<p>
We have seen some of these vocabularies before.  Later, I will tell you more 
about the other ones, but for now let's make a new one.  
<p>
<br><code>                 order</code><br>
<br><code>                 context: forth forth root    current: forth</code><br>
<br><code>                 vocabulary myvoc</code><br>
<br><code>                 order</code><br>
<br><code>                 context: forth forth root    current: forth</code><br>
<br><code>                 vocs</code><br>
<br><code>                 myvoc ignoredom command-completion keys-forth disassembler</code><br>
<br><code>                 system assembler hidden root forth</code><br>
<p>
Notice that the creation of the new vocabulary did  <code><A href="_smal_AM#27c"> not </A></code> 
change the search order.  When a vocabulary is created, it is not automatically 
added to the search order.  It does, however, now appear in the list of all 
vocabularies as displayed by  <code><A href="_smal_BS#32a"> vocs </A>.</code> 
How do we get the new vocabulary into the search order? Just execute it, of 
course.  
<p>
<br><code>                 myvoc order</code><br>
<br><code>                 context: myvoc forth root    current: forth</code><br>
<p>
What's in the new vocabulary? Exactly what we have put into it: nothing.  
<p>
<br><code>                 words</code><br>
<p>
Since our new vocabulary is the first thing in the search order,  <code><A href="_smal_AJ#339"> words </A></code> 
displays the words in <strong>myvoc</strong> , and since we haven't put any 
words in it, there are none to display.  
<p>
To put some new words into <strong>myvoc</strong> , we have to make <strong>myvoc</strong> 
the compilation vocabulary.  
<p>
<br><code>                 definitions  order</code><br>
<br><code>                 context: myvoc forth root     current: myvoc</code><br>
<p>
Now any new words we define will go into <strong>myvoc</strong> .  Let's make 
one: 
<p>
<br><code>                 : SSS ." hello" ;</code><br>
<p>
It does not really matter what words you make, we are only interested in seeing 
where they appear.  
<p>
<br><code>                 words</code><br>
<br><code>                 sss</code><br>
<p>
Now we can execute <strong>sss</strong> , because it is contained in the 
vocabulary <strong>myvoc</strong> , which is in the search order.  
<p>
<br><code>                 sss</code><br>
<br><code>                 hello</code><br>
<p>
But suppose <strong>myvoc</strong> were not in the search order? Let's replace <strong>myvoc</strong> 
in the search order with  <code><A href="_smal_BR#209"> forth </A>.</code> 
<p>
<br><code>                 order</code><br>
<br><code>                 context: myvoc forth root     current: myvoc</code><br>
<br><code>                 forth  order</code><br>
<br><code>                 context: forth forth root     current: myvoc</code><br>
<br><code>                 sss</code><br>
<br><code>                 sss ?</code><br>
<p>
This time, <strong>sss</strong> didn't work, because the vocabulary where it 
lives is no longer in the search order.  Notice that myvoc is still the 
compilation vocabulary, (the word to change the compilation vocabulary is  <code><A href="_smal_AH#1b7"> definitions </A>,</code> 
and we haven't executed that recently).  Having <strong>myvoc</strong> as the 
compilation vocabulary doesn't help us find <strong>sss</strong> , because the 
thing that controls whether or not a word is found is the search order.  The 
compilation vocabulary just controls where to put new words.  
<p>
<p>
<h2>More Search Order Words</h2>
<p>
There are a few more words to control the search order.  They generally aren't 
used nearly as often as the ones we have seen so far.  The first is  <code><A href="_smal_BU#29c"> previous </A>,</code> 
which is sort of like the opposite of  <code><A href="_smal_BU#14c"> also </A>.</code> 
Whereas  <code><A href="_smal_BU#14c"> also </A></code> increases the number of 
vocabularies in the search order by duplicating the first one,  <code><A href="_smal_BU#29c"> previous </A></code> 
reduces the number by tossing out the first one.  
<p>
<br><code>                 only forth also definitions order</code><br>
<br><code>                 context: forth forth root    current: forth</code><br>
<br><code>                 previous order</code><br>
<br><code>                 context: forth root    current: forth</code><br>
<p>
 <code><A href="_smal_BU#29c"> previous </A></code> need not follow  <code><A href="_smal_BU#14c"> also </A></code> 
; it will happily throw out the first vocabulary in the search order even if 
that vocabulary is not duplicated in the search order.   <code><A href="_smal_BU#29c"> previous </A></code> 
is not a part of the Forth 83 Standard, but it is in the popular F83 Forth 
implementation.   <code><A href="_smal_BU#29c"> previous </A></code> is not 
frequently used in most code that I have seen.  Most people prefer to rebuild 
the search order from scratch with  <code><A href="_smal_BB#289"> only </A></code> 
and  <code><A href="_smal_BU#14c"> also </A>,</code> rather than trying to keep 
track of adding and removing vocabularies from the search order.  
<p>
Another search order tool is  <code><A href="_smal_AN#1ed"> except </A>.</code>  <code><A href="_smal_AN#1ed"> except </A></code> 
takes an argument from the input stream, which is the name of a vocabulary to 
remove from the search order.  
<p>
<br><code>                 only forth hidden also forth also  order</code><br>
<br><code>                 context: forth forth hidden root     current: forth</code><br>
<br><code>                 except hidden  order</code><br>
<br><code>                 context: forth forth root     current: forth</code><br>
<p>
 <code><A href="_smal_AN#1ed"> except </A></code> will go through the search 
order and remove any occurrence of the vocabulary which is its argument  <code>(<A href="_smal_AO#21e"> hidden </A></code> 
in this case).   <code><A href="_smal_AN#1ed"> except </A></code> is unique to 
Forthmacs; I don't know of any other Forth that has it.  
<p>
Finally, there is the standard word  <code><A href="_smal_BR#2c9"> seal </A>.</code>  <code><A href="_smal_BR#2c9"> seal </A></code> 
removes all occurrences of  <code><A href="_smal_AX#2b7"> root </A></code> from 
the search order.  You may have noticed that  <code><A href="_smal_AX#2b7"> root </A></code> 
tends to remain in the search order no matter what you do; well,  <code><A href="_smal_BR#2c9"> seal </A></code> 
is the way to get rid of it.  You will hardly ever want to do this.  It does 
come in handy for implementing "turnkey" systems, where the user is not supposed 
to have access to any Forth words other than the ones that you explicitly 
provide.  With  <code><A href="_smal_AX#2b7"> root </A></code> removed from the 
search order, you can make it impossible to change the search order.  For 
example (don't try this or you will have to restart Forth): 
<p>
<br><code>                 only forth myvoc seal</code><br>
<p>
Right before  <code><A href="_smal_BR#2c9"> seal </A></code> is executed, the 
search order is: 
<p>
<br><code>                 context: myvoc root     current: forth</code><br>
<p>
Right after, the search order is: 
<p>
<br><code>                 context: myvoc     current: forth</code><br>
<p>
Since <strong>myvoc</strong> does not contain any words which modify the search 
order, there is no way to get out.  
<p>
<p>
<h2>Predefined Vocabularies</h2>
<p>
Risc-OS Forthmacs contains several built-in vocabularies.  We saw their names 
when we executed  <code><A href="_smal_BS#32a"> vocs </A>.</code> To get a list 
of the words in a vocabulary, put that vocabulary in the search order by typing 
its name, then execute  <code><A href="_smal_AJ#339"> words </A>.</code> Here 
are the Risc-OS Forthmacs vocabularies and their contents: 
<p>
<p><pre>
        root                    Contains words used to control the search
                                order.  Selected by executing ONLY.
        forth                   Contains most of the important words.
                                This is the most important vocabulary.
        hidden                  Contains a lot of words which help to
                                implement other words.  The words in
                                HIDDEN are not intended for use by
                                applications programs, because they are
                                not documented and not guaranteed to remain
                                the same or even exist in future releases.
                                If the decompiler shows a word that is not
                                in the FORTH vocabulary, this is a good
                                place to look.
        assembler               Contains words for assembling machine
                                code.  See the "Assembler" chapter.
        system                  Contains words which are direct interfaces
                                to certain TOS system calls.  See the
                                "System Calls" chapter.
        disassembler            Contains words used to implement the
                                disassembler.
        keys-forth              Contains words which associate keystrokes
                                with the editing functions of the command
                                line editor.  The command line editor
                                searches KEYS-FORTH directly, without
                                using the search order.  You can change
                                the keys used by the command line editor
                                by redefining the words in this
                                vocabulary.
        bug                     Contains words needed for the assembler level
                                debugger or other debugging tools.
</pre><p>
<p>
<p><pre>
        command-completion      Contains words used to implement the
                                command completion feature of the command
                                line editor.
        ignoredom               Gee, I liked this name when I thought it
                                up.  It seems pretty awful now.  Anyway,
                                IGNOREDOM contains words used to
                                implement the condition compilation words
                                IFTRUE, OTHERWISE, IFEND, etc.
</pre><p>
<p>
<p><pre>
        TERMINALS               Contains words used to describe the
                                characteristics of various types of
                                terminals which can be connected to the
                                system.
</pre><p>
<p>
<p>
<h2>One More Thing</h2>
<p>
When you started a colon definition in elderly Risc-OS Forthmacs versions or 
other Forth-83 implementations, the first vocabulary in the search order was 
changed to be the same as the compilation vocabulary.  For example: 
<p>
<br><code>                 only forth also hidden definitions  forth order</code><br>
<br><code>                 context: forth forth root     current: hidden</code><br>
<br><code>                 : qqq  [ order ]</code><br>
<br><code>                 context: hidden forth root     current: hidden</code><br>
<br><code>                 ;</code><br>
<br><code>                 order</code><br>
<br><code>                 context: hidden forth root     current: hidden</code><br>
<p>
See how the first vocabulary in the search order changed from  <code><A href="_smal_BR#209"> forth </A></code> 
to  <code><A href="_smal_AO#21e"> hidden </A>.</code> This behavior is built in 
to : so that the compilation vocabulary will be part of the search order while 
the colon definition is being completed.  
<p>
Note that the original search order is  <code><A href="_smal_AM#27c"> not </A></code> 
restored after the colon definition is finished.  Thus a colon definition can 
alter the search order.  Usually this is not a problem, because most of the time 
the compilation vocabulary is the first one in the search order anyway.  But 
don't say I didn't warn you.  
<p>
<strong>This is not the case in current Forthmacs or ANS Forth versions.</strong> 
The search order is  <code><A href="_smal_AM#27c"> not </A></code> changed by 
colon any longer!) 
<p>
<p>
Code definitions do a funny thing too: 
<p>
<br><code>                 only forth also hidden definitions  forth order</code><br>
<br><code>                 context: forth forth root     current: hidden</code><br>
<br><code>                 code rrr   order</code><br>
<br><code>                 context: assembler forth root     current: hidden</code><br>
<p>
First, note that the first vocabulary in the search order is  <code><A href="_smal_AD#153"> assembler </A>,</code> 
not  <code><A href="_smal_BR#209"> forth </A></code> or  <code><A href="_smal_AO#21e"> hidden </A>.</code> 
Second, note that I did not have to put brackets ([ ]) around  <code><A href="_smal_BE#28c"> order </A></code> 
this time, because code definitions are not created in compile state like colon 
definitions.  If fact, if I had used brackets, it wouldn't have worked right, 
because the closing bracket (]) would have thrown me into compile state which is 
not what I want.  
<p>
Now watch what happens when I finish the code definition: 
<p>
<br><code>                 end-code</code><br>
<br><code>                 order</code><br>
<br><code>                 context: hidden forth root     current: hidden</code><br>
<p>
Weird, huh? Why did it restore the first vocabulary to  <code><A href="_smal_AO#21e"> hidden </A>,</code> 
instead of back to  <code><A href="_smal_BR#209"> forth </A>?</code> It is 
reasonable to restore it to something other than  <code><A href="_smal_AD#153"> assembler </A>,</code> 
something, because you hardly ever want the  <code><A href="_smal_AD#153"> assembler </A></code> 
vocabulary in the search path except when you are actually in the middle of 
assembling something.  The question is, what do you restore it to? The 
reasonable answer is to restore it to its previous state, but that is not the 
way it works.  For historical reasons,  <code><A href="_smal_BV#1dd"> end-code </A></code> 
restores the first vocabulary in the search path to the compilation vocabulary.  
Since the compilation vocabulary is usually the same as that first vocabulary in 
the search path, this does the right thing most of the time.  It is only a 
problem if you don't know about it.  
<p>
</body>
</html>
