Becoming an STx Guru

The following text is based on an STx workshop of 2005, which took place in the Acoustics Research Institute (ARI), and which was described "our weekly time-wasting meeting" in a colleague's blog. It explains the internal structure and the features of the STx 3.7 release, and has been carefully adapted to the current STx version (4.0) in 2014.

Whom this is for

This manual is a general introduction to the programming language that is part of the STx environment. Reading this manual will allow you to implement procedural functions and then to proceed to object-oriented classes for a wide range of tasks, with a natural focus on numerical applications, sound processing, and visualization. As a reader of this manual, you should be slightly familiar with programming in general, and it would be great if you were an avid user of STx (on the other hand, being faintly familiar with starting up STx, loading sound files, and maybe even starting the spectrogram function should suffice).

This manual is intended for reading from start to end (not necessarily without interruptions). It is not a reference manual (there is such a thing, too: the STx Quick Reference), meaning that it will abstract from, you might even bluntly say: omit, many a detail in order not to depress the reader with a seemingly abundant amount of material. Instead, a careful selection has been taken on what is, and what is not, necessary for achieving common goals. Of course this selectivity (or, if you prefer, these omissions) try hard not to give any false impressions of what can, and what can't, be done with STx (and how). We are well aware that writing such an introductory programmer's (or, as we hope, programmers') manual is a slippery slope, and we hope for the reader's (or, as we hope, the readers') pardon if the depth covering each topic is too shallow, or to deep, or both (we are not quite sure whether the latter is logically possible, but one never knows). We appreciate any comments or criticism, both on this manual and on STx.

If you want to dig deeper into any specific topic, or if you are looking for information on a specific issue, we recommend having a look either at the online help of STx (just start STx and select the "Help" menu, or press the "F1" key), or, even better, at the STx Programmer Guide. Both are a vast and ever-growing collection of deep, sometimes even exhausting wisdom. They are, in fact, the official reference manual.

When new to STx programming, we recommend first reading this manual (you might skip paragraphs you find particularly uninteresting), probably trying out all the examples by yourself, preferably even modifying and improving them. Afterwards, you will have a basis firm enough for using the STx Programmer Guide, or even the STx Quick Reference, as a reference manual. And you will surely have a sound basis for solving all your future programming tasks with STx, disposing of the need for any other programming languages or environments.

How to write an STx Script: The Easy Part

This chapter deals with quite a basic fact, answering the first question that may come to your mind: How can I create and edit a script, and what need I do to submit it to STx for execution?

Well, the first part is easy: Just use a text editor of your choosing. Since Windows does not come equipped with the editor of your choosing, you should probably get it from www.vim.org, unless, of course, you already did so.

There is a faint convention to use the filename extension ".STS" for STx scripts, but everything else will really work equally well. So: just start editing.

After crafting your STx script, you still have STx to notice its presence, and to execute it. The simplest way for working on a script is pushing the large button labelled "Script file" (on the top of the main STx window, labelled (and called) "Workspace". After pressing this button, a file dialog titled "Select script file" will open, allowing you to, well, select your shiny new script file.

To the right of the "Script file" area, there is another area labelled "Macro". If your script file contains more than one macro function, this drop-down thingy will allow you to select which macro to execute.

There is a third field, labelled "Arguments", where you can supply arguments for your macro. And there are two awesome checkboxes labelled "Debug" and "Console", respectively. When checking "Debug", STx will start your amazing macro program in the gorgeous STx macro debugger. When checking "Console", your macro will run in the interactive STx console (an interactive version of the STx command interpreter). When checking both, either may happen, or both.

The awesome buttons, fields and checkboxes mentioned here build up what we call the gorgeous Script Controller. You will not find much more information in Script Controller.

Note: If you do not see any of the fields mentioned here (bluntly put: if you do not see the Script Controller at all), it is likely that it is not there. In this case, try using the Workspace menu item "Scripts > Show Scripts" to unhide it.

Constants

Our STx does not strictly discern between string constants and numerical constants. Normally each constant is considered a string, there only being exceptions dependent on the context where the constant occurs. If, for example, a constant occurs as part of a numerical expression, STx tries to get its numerical value.

Generally, you may, but you need not, put single quotes around constants. With a few exceptions depending on context, this will not change the way STx handles the constant. So each of the following strings is a valid STx constant:

string1
'string1'
-12.34
'-12.34'

Regardless of the presence or absence of single quotes, both the first and the second argument will be considered string constants. Also regardless of the presence or absence of single quotes, both the third and the fourth constant will be considered numerical constants when occurring in a numerical context, or string constants when occurring in a string context.

If a string constant contains whitespace characters, it depends on the context whether STx considers it one string constant or more than one string constant. If you want to make sure that a string constant is considered one constant, you should always put single quotes around the whole affair:

'Hello World'
Hello World

While the first string is always considered one string constant, the second one may, depending on where it is occurring, be considered one string constant denoting the string "Hello World", or two string constants, denoting the strings "Hello" and "World", respectively.

These issues will be dealt with in more detail below. For the moment, the curious reader may have a look at the following assignment statements:

#a := num '5' * '3' // value of #a will be 15
#b := set 5 * 3 // value of #a will be "5 * 3"

The first statement is a numerical assignment (denoted by the keyword "num"). Both constants, 5 and 3, will be considered numerical constants, even if surrounded by quote characters. On the other hand, the second statement is a string assignment (denoted by the keyword "set"). So the argument will, logically, be interpreted as one string constant, "5 * 3", even though it contains whitespace and lacks any quote character. What happens physically is that all separate words will be concatenated to the one string constant expected, inserting exactly one blank between each pair of words. The following statements show the consequences of this procedure:

#b := set 5 * 3 // value of #b will be "5 * 3"
#b := set 5        *      3 // value of #b will be "5 * 3", too
#b := set '5 * 3' // value of #b will also be "5 * 3"

In the second statement, though the words "5", "*", and "3" are separated by more than one whitespace character, the one string that will be built up from them is "5 * 3" – when concatenating them, they get separated by exactly one space character.

You may influence the way concatenation works by quoting some, or all, of the words to concatenate. Quoting a word will prevent STx from automatically inserting a blank before and after that word. So, compare the above statements with the statements below:

#b := set 5 '*' 3 // #b is "5*3" (no space)
#b := set '5' * 3 // #b is "5* 3" (one space)
#b := set 5 ' * ' 3 // #b is "5 * 3" (three spaces)

With the first statement, the word in the middle, "*", is quoted. This indicates STx on concatenation not to insert a space character either before or after this word, resulting in #b being set to "5*3" (no intervening whitespace).

With the second statement, the first word is quoted and will, hence, be concatenated to its right successor (there is no left predecessor) without inserting space. The second and the third word are not quoted and will be concatenated with an additional space in between them. This results in #b being assigned "5* 3" (no whitespace between "5" and "*", one blank between "*" and "3").

With the third statement, concatenation will not add any additional blanks either before or after the word in the middle. The whitespace that is part of the word, i.e. part of the quotation (three blanks before and after the asterisk, each), will be unaltered, though. So what results it #b being assigned the string "5 * 3" (exactly three blanks both before and after the asterisk).

Within a constant, you may alter the meaning of special characters by using the STx escape character "`", the backwards single quote, sometimes called back-tick. At the current stage, we can only use this feature for defining a string constant that contains single quote characters themselves:

#a := set 'Rome is a city but `'Rome`' is a four-letter word'
#a := set Rome is a city but `'Rome`' is a four-letter word

Both statements will assign the string "Rome is a city but 'Rome' is a four-letter word" to a variable called #a (although it may not always be easy later to retrieve the value of this variable). We have to leave these issues open for later discussion.

Variables

The names of STx variables start with an optional one-character prefix indicating the scope of this variable (the lack of such a prefix indicating shell-global scope, see below). Besides this prefix, they may consist of letters and digits, although their first actual character must be a letter. Names are not case-sensitive, meaning that e.g. „freq", „Freq", and „FREQ", are names of the same variable.

Our STx discerns four kinds of scope and, hence, four kinds of variables:

Scope	Prefix	Description
Global	@	Global variables are known to, and may be changed by, every shell instance, both running or yet to start. Global variables are the only kind of variables guaranteed to be persistent over interactive macro calls during one STx run.
Shell	no prefix	"Shell-global" variables are global to the running shell. This implies that they are known to, any may be changed by, any macro invoked by a normal macro call. The variable will not, though, be known to other running shells or to shells yet to start. So, in general, shell-global variables will not be persistent through interactive calls to a user-defined macro.N.B.: Do not mix up shell-global variables with item handles that will be dealt with in chapter 5 (page 1). For the time being, suffice it to say that item handles look like shell-global variables, but that they reside in different namespaces and that they behave differently.
Local	#	Local variables are valid only during the runtime of one invocation of a shell macro. On each invocation of a shell macro, a separate namespace containing all its local variables is being created. This namespace is destroyed as soon as the macro finishes. Note that this implies that on invocating a macro recursively, it will find itself starting with a fresh, empty copy of all its local variables, while the calling instance will find its namespace, i.e. its local variables, untouched.
Member	&	When adhering to the object-oriented programming paradigm, you will define classes and instantiate them. Each instance of a class will have its own set of variables called member variables. The introductory chapters will stick to the clean ole' procedural way of programming.

As a rule of thumb, most of the time you will be using local variables whose names start with "#", e.g. variables like #i, #depth or #title.

Variables need neither be declared nor to be explicitly initialized. If you want to introduce a variable, just start using it. Note that a variable is empty, i.e. contains the empty string, when being referred to without having been explicitly set to a value different from the void.

Typing

There is really not much about typing in STx. All variables store strings, like it is the case with most scripting languages. Of course these strings are at liberty only to consist of digits, a comma, and an optional sign character, making them look like numbers, smell like numbers, taste like numbers, and being treated like numbers by numerical STx functions like addition, multiplication, or even the controversial subtraction.

Setting Variables

Setting a variable, i.e. assigning a value to the variable, is done with the ":=" operator. Its general format is

variable := expression

You should never use this kind of assignment with STx, though. Instead, always use one of the following assignments:

variable := SET string_expression       // typed string assignment
variable := INT numerical_expression    // typed integer assignment
variable := NUM numerical_expression    // typed numerical assignment
variable := EVAL amazing_expression     // awesome assignment using EVAL

See the following chapters for the reasons of this recommendation.

Simple string assignment

In its simplest form, this "expression" is a simple string constant, just like in the following example:

#adress := 'Reichsratsstrasse 17'     // discouraged, see further below - always use SET

Or even:

#adress := Reichsratsstrasse 17       // discouraged, see further below - always use SET

Although generally valid, either usage is strongly discouraged, because it may lead to often surprising, seldom desired results if or when the string to be assigned starts with the name of a built-in function or a user-defined macro (note that words so far not reserved may become reserved words any time now, and that, when in a large software-building project, you never know how your colleagues call their helper macros today).

Typed string assignment

You may indicate your expressed desire for the expression to be a string constant by prefixing it with the type-selector statement "set":

#address := set Reichsratsstrasse 17        // this is better

or, even better:

#address := set 'Reichsratsstrasse 17'      // and this is the way to go

In this case, the assignment will even work if one day the STx macro language should be added a built-in function "Reichsratsstrasse" (which is, we dare to admit, unlikely) – or if one of your colleagues' macros happens to be likely called.

Typed numerical assignment

Although the value assigned is invariably a string, this string may be the result of a numerical computation. You may indicate your desire for it to be so by using one out of the following type-selector statements: "int", "num", and "eval".

The "int" type-selector will cause your expression being evaluated as an integer expression. More precisely (more precisely less wrongly), the expression will be evaluated numerically, and the result will be converted to an integer whose textual representation will be the string to be assigned to the destination variable. The calculation itself will be done with the point floating, though (see the below examples for what that means).

The "num" type-selector will cause your expression being evaluated as a numerical expression, provided it is such. The textual representation of the numerical result will be the string to assign to the destination variable.

The "eval" type-selector is the most powerful of them all. Firstly, it does everything the "num" type-selector does. So, when evaluating a plain numerical expression, it is your free choice whether to use "num" or "eval" (we might one day choose spontaneously to fade out the "num" type-selector, but do not allow this to bias your choice). But, secondly: The "eval" type-selector is capable of much, much more: It does vector and matrix operations of the most sophisticated kind, calculates averages, converts units, dances the Fourier transform, and so on. The actual number of functions available to "eval" expressions is more than 70 and counting (see the interactive STx help topics "eval").

For an example, compare the following statements:

#a := num 3*3.4 // result is 10.19999999999
#a := int 3*3.4 // result is 10 (!)
#a := num 3*int(3.4) // result is 9
#a := int 3*int(3.4) // result is 9
#a := num int(3*3.4) // result is 10 (!)
#a := num int(3*int(3.4)) // result is 9

Here the first statement will cause #a to be assigned the result of the floating point multiplication of 3 and 3.4, i.e. about 10.2. In the second statement, the same multiplication will be calculated, and only the result of this calculation, i.e. 10.2 (roughly...), will get truncated to an integer – this integer in turn being 10. Only the third and the fourth statement (both!) will cause the second factor, 3.4, to be broken down to an integer before multiplication. This usage, though, is strictly not a feature of the type selector-based assignment, but of the numerical function (we will come to these later) "int". (Note that since the product of two integers is an integer itself, in the third and fourth statement it does not make any difference whether we use the "num" or the "int" type selector.)

Do not mix up the type selector of the assignment with any built-in type-conversion functions: Whereas the type-selector always immediately follows the assignment operator, ":=", type-conversion functions never do. Furthermore, the arguments of type-conversion functions are always enclosed in brackets, whereas the type selector never uses brackets.

So in the third to sixth example, the type selectors are "num", "int", "num", and again "num", whereas the strings "int(3.4)", "int(3*3.4)", and "int(3*int(3.4))" are calls to the built-in type conversion function, "int". (If this sounds confusing for the moment, you should not worry: In practice, things are much easier, and it is not normally necessary to think these things over).

Function calls

Syntactically, every built-in STx function, every macro, and every method of a user-defined class may be the source of an assignment. If this happens to be the case, the respective function or macro is executed, and its result is assigned to the destination variable. See it for yourself:

#i := word 2 one two three four // #i will be "three"

The built-in "word" function takes an integer index and a list of strings as its arguments. From this list, it selects and returns the string with the respective index. Hence, the expression "word 2 one two three four" will select, and return, the third word, „three". When part of an assignment as in the above example, this very string "three" will be assigned to the respective destination variable, in this case: to the local variable #i.

The same holds true if the source of the assignment is the name of a class method. In this case, the respective method is being called, and its result (which may as well be the empty string) gets assigned to the destination variable.

The issue of function calls will be dealt with in full detail in chapter Calls to STx Macros and Built-in Functions.

Accessing Variables

If and when you want to access the value of a variable (bluntly put: to read out its content), you need to put a dollar sign in front of the variable name. What happens internally is that, before actually executing a line from the macro file, STx replaces all occurrences of variable names that are prefixed by a dollar sign by the content of the respective variables. See for yourself:

#i := int 7
writelog 'The current value of variable #i is $#i'
#i := int $#i + 1 // #i will be set to 8
#heading := set 'This is page $#curpage out of $#totpage'

The first line in this example will assign the value 7 to a local variable called #i. This is nothing new; note that we are using the type-selector "int" to make sure the assigned value is interpreted as an integer, though in this case this is strictly redundant because 7 cannot help being an integer anyway.

The second line will print out the text "The current value of variable #i is 7". This should not be surprising, for the name of a variable is only replaced by its content if preceded by a dollar sign. Hence, the first occurrence of "#i" – although we know that there is a variable called #i – does not get replaced by that variable's value. The second, though, does, because it is preceded by the dollar sign. STx is one reliable piece of software, strictly and indiscriminately following the instructions laid out by its master.

What the third line does is increase the value of the local variable #i by 1. What's more interesting is how it does so. First, STx replaces all dollar-prefixed variable names by the contents of the respective variables. Hence, STx will replace the string "$#i" by the current value of #i which, in our example, happens to be 7. This replacement will change the current statement from "#i := int $#i + 1" to "#i := int 7 + 1". Now this is one fine integer expression that in turn gets evaluated to 8, thereby causing 8 to be the string that is finally assigned to the destination variable.

The fourth and last line demonstrates that substitution also works within string constants, and that it does so even if they are put under quotation marks (in our case, apostrophes). Some macro languages, e.g. the well-known UNIX shells, let certain kinds of quotation marks prevent substitution. Users familiar with such shells should bear in mind that STx is a kind of its own. (Note that if you really want to suppress the special meaning of a character like the dollar sign, you may precede it with the STx escape character "`", the so-called back-tick.)

The aspiring STx guru may find it instructive to consider the following example (anyone else will find no harm in completely skipping this example):

#var := set 'one'
writelog '#var now containing "$#var"'

#var := set 'two'
writelog '#var now containing "$#var"'

$#var := set 'three' // N.B.: substitution will make this
                     // "two := set 'three'"
writelog '#var still containing "$#var"'

writelog '...but there suddenly is a variable called two'
writelog '...and its value is "$two"'

The first line is well familiar. It assigns the string "one" to a local variable called "#var". Consequently, the second line will print exactly the following string:

#var now containing "one"

The third line changes the value of #var to "two", hence the third line will print out the following string:

#var now containing "two"

No surprises yet. But what will the fourth line do? Well, not to be surprised about the answer to this semi-rhetorical question, we must analyze the statement carefully. It reads "$#var := set 'three'" – did you notice that the assignment target is preceded by a dollar sign? Alas, this instructs STx to replace the variable name by its content; but the content of variable "#var" is "two." Hence the statement gets, by substitution, altered to "two := set 'three'". This is a perfectly valid assignment statement, only that the target of the assignment is a shell-global variable called "two". So we assign the string "three" to a variable called "two". The next statements only illustrate this fact by printing out the respective values.

Read Functions

The "read" family of functions supply a means for parsing the contents of a string or a variable, that is for splitting them into several pieces, and for storing some or all of these pieces into one or more other variables (or even in the same variable). That being said, it should be noticed that everything is much easier than this description implicates. See for yourself:

readstr 'one two three four five' #a #b #c /Delete
// #a is now "one", #b is now "two", #c is "three four five"
writelog 'now a="$#a", b="$#b", and c="$#c".'

The readstr command parses its first argument into (at most) as many blank-separated strings as there are variables. If the number of variables is higher than the number of available words, the remaining variables will either be cleared (if supplying the /Delete option), or they will be left untouched (if omitting the /Delete option). If, on the other hand, the number of variables is lower than the number of available words, the last variable gets all the remaining words. Note that if there is more than one whitespace character between two words, this will not do any harm.

So what the above example does is parse the string "one two three four five" into three substrings (there are three variables supplied, #a, #b, and #c). The first substring will be the first word, "one". The second substring will be the second word, "two". Since there are more words than variables, the third substring will catch all the rest, that is, the string "three four five". It's really simple, isn't it?

There is one additional feature you may, or may not, find convenient. You may as well parse strings that are separated by exactly one character of your choice. If, for some reason, you prefer semicolons over blanks, you might have done the above example as follows:

readstr 'one;two;three;four;five' #a ';' #b ';' #c /Delete
// #a is now "one", #b is now "two", #c is "three;four;five"
writelog 'now a="$#a", b="$#b", and c="$#c".'

The syntactical difference between those two examples is that the latter explicitly names the separator character between each pair of variables. The difference in semantics is that now there must be exactly one separator character between two words. So this variant of the readstr command empowers you to read empty words, two. Consider the following statement (we may abbreviate the /Delete option to /D, if we do not care for the reduced legibility):

readstr 'one;;two;three;four;five' #a ';' #b ';' #c /D
// #a is "one", #b is empty, #c is " two;three;four;five"
writelog 'now a="$#a", b="$#b", and c="$#c".'

Looking awfully identical, doesn't it? Well, instead of one semicolon, there are now two semicolons between the first two words, "one" and "two". Believe it or not, this is making all the difference in the world: STx will consider these successive semicolons three separate words, the first being "one", the second one being empty, and the third one being, in general, "two" (in our case where there are only three variables supplied, the third word will get the rest of the string). So, with this example, variable #a will get the string "one", variable #b will be cleared, and variable #c will get the rest, that is the string, "two;three;four;five". Cool, isn't it?

Note that if you omit the /Delete option, target variables corresponding to empty words will not be cleared, that is, they will keep whatever value that had before calling readstr. That being said, you can easily foretell the results of the following statements:

b := set 'old value before calling readstr'
readstr 'one;;two;three;four;five' #a ';' #b ';' #c
writelog 'now a="$#a", b="$#b", and c="$#c".'

As you rightly foretold, the readstr command in this example will not change the value of the second variable, #b, since the second word in this string is empty.

Although technically a consequence of the above, it may not immediately be clear that the space character, too, may explicitly named as separating arguments – and that this does cause a difference to the default readstr behaviour. Look for yourself (and notice that there are two space characters between the "a" and the "b" in the first string constant):

readstr 'a b c' #a #b #c /Delete
// #a is now "a", #b is "b", #c is "c"
writelog 'a="$#a", b="$#b", c="$#c"'
readstr 'a b c' #a ' ' #b ' ' #c /Delete
// #a is now "a", #b is empty, #c is "b c"
writelog 'a="$#a", b="$#b", c="$#c"'

As you already know, there is one important difference between both readstr variants: When explicitly naming the separation character, STx considers consecutive occurrences of the separation character to separate empty strings. When not naming a separation, consecutive occurrences of whitespace are considered one single separator character, thereby causing no empty word to be read. So, in the above example, the first readstr command will read "a" into #a, "b" into #b, and "c" into #c, whereas the second readstr command will read "a" into #a, the empty word into #b, and the remaining string, "b c", into the last variable, #c.

Of course the string argument supplied to readstr may even be the result of variable substitution. Less prosaically put, you might as well supply code like the following:

#three := set 'THREE'
readstr 'one two $#three four' #a #b #c #d
writelog 'now a="$#a", b="$#b", c="$#c", d="$#d"'

Before executing the command, STx will, as usual, look for any variable name prefixed with a dollar sign. If there happens to be any, they will be replaced by the contents of the respective variables. So in the above example, "$#three" will get replaced by the contents of variable #three, thereby causing the readstr command actually to be processed to be the following:

readstr 'one two THREE four' #a #b #c #d

Only the strong survive the following example (anyone else will find no harm in skipping it).

readstr 'x y z' #a #b #c 
writelog 'now a="$#a", b="$#b", c="$#c"'
readstr 'one two three' $#a $#b $#c
writelog 'gee, now a="$#a", b="$#b", c="$#c"'
writelog 'but mysteriously, x="$x", y="$y", z="$z"'

When listening very carefully, you might hear the above example speak for itself. If not, you will find the key in the third line that contains the second readstr command. Did you notice the variables being prefixed by a dollar sign each? Now the dollar sign indicates STx that variable substitution is desired (and required) before the command is to be processed. So all STx does is replace the two strings "$#a", "$#b", and "$#c" by the contents of the respective variables. Since, at this stage, their contents are "x", "y", and "z", the command actually to get executed will look as follows:

readstr 'one two three' x y z

Now this clearly is a request to parse the string "one two three" into the three shell-global variables "x", "y", and "z".

When reading directly from one variable, you might prefer a variant of readstr, the readvar command. readvar works similar to readstr, as the following example shows:

#var := set 'one two three'
readvar #var #a #b #c /Delete

The first argument to readvar is one variable to read from. The remaining arguments are the variables where to store the words read. Otherwise that readvar reads from a variable as opposed to reading from a literal string, there is no difference between readvar and readstr.

You will undoubtedly ask why there is such a thing as a readvar command. After all, all that

readvar #var #a #b #c

does may as well be done using the following command:

readstr '$#var' #a #b #c

You are, of course, right in principle. The difference between the two commands is that the latter depends upon variable substitution which introduces a second step when evaluating and executing the command. Hence, the former is simply faster.

Note that there is a readtable function, too. We will come to that much later when dealing with the versatile (both are) STx table feature.

Special Variables

There are a number of reserved variables that serve special purposes. At this stage, it suffices to give a short overview of the most important special variables.

Variable	Description
rc	After executing an STx statement or built-in function (not a user-defined macro!), this variable contains its numerical return code, 0 indicating success, and values different from 0 indicating different kinds of failure. "rc" gets set after each invocation of an STx statement or built-in function, meaning that an error code will get reset when executing the next statement. If you later need to map a numerical error code to a textual error message (e.g. for presenting it to the user), you may always use the EMSG command.
emsg	This variable contains a textual description of the value of the "rc" variable. It, too, gets reset with each new STx statement. If you have saved the value of rc e.g. in a local variable and you later need to map it to a textual error message (e.g. for presenting it to the user), you may always use the EMSG command. (And don't confuse the emsg variable and the EMSG command - they are not the same).
#argc, #argv	On macro invocation, "argc" contains the number of arguments supplied to the respective macro, while "argv" contains the actual arguments (all of them). See below.
result	After returning from a user-defined macro call, the variable "result" contains the value returned by the respective macro, i.e. the value the macro supplied as an argument to the "exit" call. If the macro was left without returning an argument, "result" is empty (this is not considered an error – honestly, there is not much that is ever considered an error as far as STx is concerned).
#read	After using one of the STx READ commands (read, readstr, readvar, readtab), this variable contains the number of arguments actually read.
#new	When allocating an STx shell item, its item name will be stored in a variable called "#new". On the issue of shell items, please be patient until chapter Shell Items: An Overview, or divert to the Shell Items chapter in the Reference Manual.

In general, even the reserved variable may be target of an assignment (this is sometimes used with the "#argv" variable for implementing default macro arguments). You should not be surprised, though, that assigning a value to either "rc" or "emsg" will not have the desired outcome: Since the assignment statement is built-in STx statement itself, executing it will reset "rc" to 0 and "emsg" to the empty string, thereby indicating that the assignment statement itself was successful (which it was).

Control Structures

Calls to Macros and Built-In Functions

You might argue that we have been through that already, but, unfortunately, this is only part true: There is much more to passing an argument to a function than we have explored when calling the built-in "word" function, or the built-in "writelog" command. This chapter tells why and what.

Macros

Semantically, an STx macro is like a procedure or function of any procedural programming language of your choice. Syntactically, a macro starts with a macro definition surrounded by square brackets, e.g. the following line:

[Macro mymacro]

There is no clear end to a macro. Execution continues until the control flow reaches an "exit" statement, until a new macro starts, or until the file comes to an end (whichever happens first).

Calling a macro

For calling a macro, you simply type the macro name. In the above case, if STx encounters a line starting whose first word is the string "mymacro", it will execute the like-named macro. STx supports recursive macro calls.

There are a few commands providing different, not always cleaner ways of calling a macro, namely the MACRO, the MACROX and the SHELL command. Please do not use these commands unless there is very good reason to (there normally isn't).

Returning From a Macro

Macro execution will stop when STx encounters the "exit" statement. Normally, control flow resumes at the line immediately following the statement that caused the macro to execute. You may supply the following optional arguments to an exit statement:

exit level result

When calling "exit" without any arguments or when calling "exit 1", control flow will, as said above, resume with the next statement after the macro call.

In general, the first argument to "exit" indicates how many call levels to skip. When calling "exit 2", control flow will not resume at the line after the statement calling the macro, but at the line after that statement calling whichever macro was in turn calling our macro. When calling "exit 3", control flow will resume at the line after the statement calling the macro calling the macro calling our macro. When calling "exit 4", control flow will resume at the line after the statement calling the macro calling the macro calling the macro calling our macro, while on "exit 5" execution will resume with the line after the statement calling the macro calling the macro calling the macro calling the macro calling our macro, and so on. There is limited use to this feature, and you are at the safe side when, at least in the beginning, always using "exit 1".

There are two special cases, one when supplying 0 for the exit level, the other when supplying a negative number.

When supplying 0 for the exit level, i.e. when executing exit 0, the executing shell will be terminated, effectively ending execution of the whole user script (both the running macro and all calling macros).

When supplying a negative level argument (e.g. exit -2), STx will return from as many macro levels as needed to find a macro level where there is a non-empty local variable #onexitall defined. Since this is a bit complicated and not very clean, you probably should try not to use this feature.

Of more interest than the first is the second argument to "exit". It is the result of the macro, that is the number or string the macro is to return. Like with assignment operations, this result may, and should, start with a type selector, one out of "set", "int", "num", and "eval". Consider the following "exit" statements:

exit 1 set 'Hallo Welt'
exit 1 int 5/3
exit 1 num 3/5

The first statement will exit the running macro and return the friendly string "Hello World". The second statement will exit the running macro and return whichever integer results on dividing 5 by 3. Finally, the third statement will exit the running macro and return the quotient of 3 and 5.

There are several interchangeable ways for the caller to retrieve whatever value the macro has returned. First of all, the result is stored in the reserved shell variable "result", rendering the following code snipped a working example:

greetings // call the "greetings" macro
writelog '$result' // ...and print its results
exit // ...and rest after a long day's work


[Macro greetings]
   exit 1 set 'Hello World'

An even more elegant feature is using command substitution, a concept both known and feared from the UNIX shells:

#var := set '$(greetings)' 
writelog '$(greetings)'

The first statement executes the macro "greetings" and assigns its result to a variable called "var". The second statement, too, calls the "greetings" macro, but directly pastes its result into a "writelog" statement.

Though command substitution is the most general way, a macro may be directly called, too – just like any built-in STx function:

#var := greetings

The latter format greatly improves legibility and is therefore he recommended way of calling a user-defined macro.

Supplying and Retrieving Arguments

There are, they say, many ways leading to Rome, and this surely holds true for retrieving and parsing the arguments a user-defined macro has been supplied with. The easiest way is disposing of the problem in the macro definition, i.e. by writing something like this:

[Macro multiply #mand #mor]
   #prod := eval $#mand * $#mor
   writelog 'multiplying $#mand by $#mor results in $#prod'
   exit

As you may have noticed, the macro name may be followed by one or more variable names. In this case the arguments supplied to the macro are being automatically parsed and stored to the like-named variables. We call this the implicit argument parsing feature. Hence, when calling "multiply 7 9", the starting macro will find its local variable #mand set to 7 and its local variable #mor set to 9. This will cause the above macro to print the message "multiplying 7 by 9 results in 63". If there are fewer arguments than variables, the remaining variables will be empty. If there are more arguments than variables, the whole remaining part of the arguments will be stored to the last variable. When you think it over, this is completely analogous to the readstr and readvar functions already known.

What's also analogous to readstr and readvar is the possibility explicitly to name a separator character of one's choice. See for yourself:

[Macro multiply #mand';'#mor]
   #prod := eval $#mand * $#mor
   writelog 'multiplying $#mand by $#mor results in $#prod'
   exit

While the first variant of the user-defined macro "multiply" will expect its arguments to be separated by an arbitrary number of whitespace characters, the second version will expect them to be separated by exactly one semicolon. This comes in handy when passing arguments that may contain, or contain, or must contain, or should contain, or would contain were it not for the fact that this is impossible with the default way of argument parsing, whitespace characters.

A further and generally slightly more flexible, though a little more laborious way of parsing one's arguments is using the special variables "#argc" and "#argv". On start-up of a macro, these variables get set to the number of arguments and to the actual argument list, respectively. So the above example may be re-written for explicit argument parsing as follows:

[Macro multiply]
   readvar #argv #mand #mor
   #prod := eval $#mand * $#mor
   writelog 'multiplying $#mand by $#mor results in $#prod'
   exit

Why is this more flexible in general? – Because of the greater versatility of the readvar and readstr commands that even allow parsing argument lists built up from a variable number of arguments. We will come to that with the macro countSTXProgrammers of chapter 4.1.1.3 on page 1.

That being said, we may well continue with a real-world example. Consider e.g. the following macro:

[Macro declare #person #attribute]
   writelog 'I hereby declare $#person an $#attribute'
   exit

Now, if you would like to declare, say, Toni an STx hero, you might call the macro as follows:

declare Toni STx hero

This will work fine because the first word, "Toni", gets stored to the first variable, #person, and the rest of the arguments, "STX hero", gets stored to the second, and last, variable, #attribute. But what if you want to declare Christian Gottschall an STx beginner? See for yourself:

declare Christian Gottschall STx beginner

This will invariably lead to the first word, "Christian", being stored to #person, and the whole rest, "Gottschall STx beginner", being stored to #attribute, hence causing Christian being declared a (sic!) "Gottschall STx beginner", which he surely isn't.

You might be tempted to attacking this problem by using quotes, but in fact this is utterly impossible. You will find any conceivable combination (and even most unconceivable combinations) of quotes and escape characters to have a meaning very different from that intended. So the only general solution to this problem is using a different separator character:

[Macro declare #person';'#attribute]
   writelog 'I hereby declare $#person an $#attribute'
   exit

Or:

[Macro declare]
   readvar #argv #person ';' #attribute
   writelog 'I hereby declare $#person an $#attribute'
   exit

Now you may declare whomever you want whatever you please:

declare Toni ; STx hero

declare Christian Gottschall ; STx beginner

Though looking unusual from the standpoint of several other programming languages, it is good STx practice to quote the macro arguments, either as a whole, or argument-wise, or even (don't read this loudly!) on a per-word basis:

declare 'Jonnie White ; STx guru'
declare 'Jonnie White' ; 'STX guru'
declare 'Jonnie' ' White' ; 'STX ' 'guru'

In general, neither form of quotation will do any harm. Remember, though, that, with STx, writing several quoted strings will cause them to be implicitly concatenated with any intervening whitespace removed. Hence, although the above examples work as desired (note the quotations of the fourth statement containing space characters), the following won't:

declare 'Jonnie' 'White' ; 'STX' 'guru'

The latter statement will print out the text "I hereby declare JonnieWhite an STXguru": Although there is a whitespace character between each pair of quoted words, the quotations themselves do not contain any blank character, thereby causing the respective words to be concatenated without any intervening space. Compare this with the third statement in the previous example where there is a space character at the beginning of the second, and at the end of the third quoted word. (We've already had a few words on concatenation in chapter on page 1.)

Note that when using the implicit argument parsing feature (or when using readvar with the default separation characters (whitespace), you need to be careful not to let the concatenation feature come in your way. Consider the following macro call:

usermacro 'one' 'two' 'three'

Here, by string concatenation, the two strings "one", "two," and "three" will get concatenated to one single string, "onetwothree". Consequently, the macro will, whatever way of parsing it uses, get only one argument – the string "onetwothree." If you want the macro actually to be called with three arguments, you will need to use one of the following statements:

usermacro one two three
usermacro 'one two three'
usermacro 'one ' 'two ' ' three'

It will work either way, additional whitespace characters never doing any harm.

As a rule of thumb, the easiest thing would be always to quote the whole arguments to a macro call, like is done in the second statement of the above example.

Combined example

The following example builds up several things we are at this stage familiar with (and several others we are not). You need not fully understand the macro at this stage, but you might notice several familiar issues.

[Macro countSTXProgrammers]
   // read 0 both into #count and #totcount
   readstr '0 0' #count #totcount // (1)

   // the "forever" loop will never terminate by itself.
   // it will only stop at a "break" statement
   forever // (2)

      // read the first person into #person, and the
      // remaining persons into #argv
      readvar #argv #person ';' #argv /Delete // (3)

      // return both the total number of persons counted and
      // the number of STx persons
      if '$#read' == 0 then // (4)
         [[{Programmer_Guide/Command_Reference/EXIT|exit]] 1 set '$#totcount;$#count' // (5)
      end

      #totcount := int $#totcount+1
      #index := keyword '$#person' Toni Jonnie Christian // (6)

      if '$#index' >= 0 then
         #count := int $#count+1
      end

   end

The countSTXProgrammers macro gets an arbitrary number of arguments separated by semicolons. Each argument is considered the name of a person. What the macro does, slightly arbitrarily, is count both the total number of persons supplied, and the number of STx programmers amongst them (STx programmers being considered Toni, Jonnie, and Christian only). The most important statements are the following (see the respective numbers commented in the macro source):

(1)

This shows an idiomatic use of the readstr command for initializing multiple variables. Here the string "0 0" is parsed into two variables, #count and #totcount, effectively setting them both to zero. Of course you might as well use the two separate assignment statements "#count := int 0", and "#totcount := int 0".

(2)

The forever keyword starts kind of an eternal loop running until a break statement will be met. Both issues are dealt with in separate chapters (Forever and Ever and Ending a Loop Prematurely: Break and Continue).

(3)

The readvar statement will be utterly familiar to you. Note, though, that one of the destination variables is the same as the source variable. This is not a problem at all. Parsing will proceed normally, and the assignment will take place only after parsing has finished. So this readvar statement will result in the first word of #argv being parsed into #person. The remaining contents of #argv will, in turn, be parsed to%… #argv itself(!), effectively removing the first word from #argv. So with each pass of the loop, the next "first" name from the list will both be stored in #person and be removed from #argv. Isn't that cool?

(4)

This line introduces the if command. You need not worry about this command; it will be dealt with properly in chapterc 4.1.3.

(5)

The exit statement will return from the macro, and it will do so – due to the if statement – on the condition that #read is zero which is the case as soon as there are no more names to read. The first argument to the exit statement is 1, indicating that only the current macro is to end (and that execution shall continue with the calling macro). The second argument to the exit statement is the result of the macro, in this case: the string to return (indicated by the string assignment type selector, set). This string is built up from #totcount, a semicolon, and #count, and will hence contain the total number of persons and the number of STx programmers, separated by a semicolon.

(6)

This statement uses the built-in keyword function to investigate whether the current person is on the list of STx programmers.

Statements and Built-in Functions

Arguments to STx statements and to built-in functions differ in several respects from macro calls:

1. One quoted string is always considered one argument, even if it contains whitespace characters.
2. Hence there is no automatic string concatenation either.
3. If (and only if) the statement or the function expects a numerical argument, you may supply a numerical expression instead of a number. This expression will be evaluated before executing the statement, or calling the built-in function, respectively.

The first two issues are easily demonstrated by the following example:

#a := word 2 'a b c' d 'e' f
writelog '#a="$#a"' // value of #a is "e"

Both quoted strings will be considered one argument each, and no concatenation will take place. Hence, the built-in word function will return the string "e", "e" being the third argument (index 2).

More surprising is the third issue, but it, too, can be demonstrated by a simple example:

#a := word 1+1 'a b c' d 'e' f
writelog '#a="$#a"' // value of #a is "e", too

Since the "word" built-in expects its first argument to be a number, any numerical expression occurring at the respective position will be evaluated before calling "word". Hence, this example will return "e", too, because 1+1 equals 2. Compare this with the following statements:

#a := word 1+1 'a b c' d '2+2' f
writelog '#a="$#a"' // value of #a is "2+2"

Here what the third argument of the "word" built-in will return is the string "2+2". This string does not get evaluated, because "word" expects a string argument (and not a numerical argument) at this position.

Note that if the numerical expression is to contain whitespace (or if it cannot be precluded that it does, e.g. when it is built up using variable substitution), you need to quote the whole expression. This quite naturally leads to the following rule of thumb: Always quote numerical expressions that are arguments to a statement or to a built-in function. See the following example:

#i := int 3-2
#a := word '$#i+2' a b c d e
writelog '#a="$#a"' // value of #a is "d"

First of all, #i gets assigned the difference between 3 and 2, that is 1. String substitution will change the string "$#i+2" to "1+2", "1" being the value of #i. Finally, due to the fact that the "word" function expects its first argument to be numerical, "1+2" will be evaluated as a numerical expression, resulting in 3. So the fourth string argument, "d", will be returned.

If Conditions are to be Tested

If conditions are to be tested, the "if" statement comes in handy. It actually does so in two flavours:

if condition statement

And:

if condition then

   many statements

else

   many more statements

end

Before going into more details about what conditions look like, a few simple examples should make things more clear:

if '$#a' == '' [[writelog 'Variable #a is empty.'

if '1+2' != 3 then
   // note the use of the STx escape character, the back-tick,
   // for building up a string that contains a quote character 
   writelog Wohllebenstraße, we`'re having a problem. 
else
   writelog All systems nominal.
end

if a > c then // character-wise comparison
   writelog 'Detecting an unusual character set'
else 
   writelog 'Detecting a usual character set'
end

if '2+3' != '3+2' writelog 'STX addition is not commutative'

Note that with the first, simple form of the "if" command, the statement to be executed in case of the condition being true must not be another "if" command. (In simpler wording: Simple if statements must not be nested.) Hence, it is not allowed to construct a statement like the following:

// BAD! NOT ALLOWED! ERRONEOUS! DON'T DO IT!
// COMBINING TWO SIMPLE "IF" STATEMENTS IS FORBIDDEN!

if '$a' == '$b' if '$c' != '$d' writelog a and b are equal, but c and d are not

If two or more "if" statements are to be combined, use their complex form instead:

if '$a' == '$b' then
   if '$c' != '$d' then
      writelog 'a and b are equal, but c and d are not'
   end
end

Conditions, Formally Revisited

Simple Comparison

Conditions basically compare two entities for being equal, or for one of them being less than, or higher than, or not higher than, or not less than, or quite unlike the other. All this is done with the comparison operators "=", "<", ">", "<=", ">=", and "!=". The arguments to these operators may either be strings, in which case a Unicode-based string comparison will take place, or numerical expressions, in which case a numerical comparison will take place. If the argument to a comparison operator looks like a numerical expression, it will be treated as such, even if the programmer did not mean that to happen.

Note that unless the argument to a comparison operator is quoted, it must be separated from the operator by at least one whitespace character. This means that while "'$#a'=='1'" (no whitespace, but quotes) and "$#a == 1" (blanks between the operator and its arguments) are well-formed expressions, "$#a==1" is not. (Yet another reason for always quoting everything, one might feel inclined to say.)

See a number of simple examples:

if '$#a' > '$#b*2' writelog '#a is more than twice as much as #b'

if 'int($#a)' == '$#a' writelog '#a is an integer'

if '$#a' == '0+$#a' then
   writelog '#a is a number.'
else
   writelog '#a is not a number.'
end

All these examples show that numerical expressions occurring in comparisons will be evaluated. Hence, the first example will compare the value of #a with twice the value of #b. The second example will compare #a with its integer part. If they are the same, we know that #a must store an integer.

The most interesting of these examples is the third one. What it does is check if the variable #a is numeric, i.e. if it contains a number. For understanding how it does so you need to remember that STx evaluates numerical expressions only. Now, should #a contain a number, say, 42, variable substitution will alter the expression "if '$#a' == '0+$#a'" to "if '42' == '0+42'". Now since "0+42" is a numerical expression (remember that single quotes do not matter), it will, in turn, be evaluated to 42 (0+42 equalling 42). This leads to the final statement "if 42 == 42". Since the number 42 is equal to 42, the condition will come out true, causing the "then" branch to be executed.

If, on the other hand, #a contains a value that is not a number, say, the string "Hello", variable substitution will alter the "if" statement to "if 'Hello' == '0+Hello". Due to "0+Hello" not being a numerical expression, there will be no evaluation, resulting in the string "Hello" to be compared against the string "0+Hello". Since they are not identical, the "else" branch will be executed.

Note that there is one potential pitfall (and note the alliteration, too). Should #a consist of digits separated by whitespace, it will be considered a number (one number) anyway. So e.g. the string "123 440 . 12" will be considered the number "123440.12". This is a feature allowing for separating numbers into groups of digits (e.g. a group of ones, tenths, and hundreds, and a group of thousands, ten-thousands, and hundred-thousands).

Pattern Matching

Besides these comparison operators, there are matching operators, too. Matching may be considered a more powerful way of comparing strings that is able not only to find out if two strings are strictly identical, but also to find out whether they are similar in a way yet to be defined. With a matching operation, its first, left-hand side argument must be the string to match. Its second, right-hand side argument must be the pattern against which to match the string.

Wildcard Pattern Matching

Wildcard patterns are similar to strings with the exception that they may contain wildcards (hence the name). With STx, these wildcards are the asterisk, "*", for matching a string of arbitrary length, and the question mark, "?", for matching exactly one character.

There are the following wildcard matching operators:

Wildcard string matching

=SI	The string argument does match the pattern. Case will be ignored.
!SI	The string does not match the pattern, not even regardless of case.
=SR	The string argument matches the pattern case-sensitively.
!SR	The string argument does not match the pattern when respecting case.

Wildcard name matching

Name matching is similar to string matching, the difference being that the string to match is expected to be a well-formed STx name. If this is not the case, the match will fail, even if the pattern would string-match the string. More precisely, matching a non-name string with =N will always return false, whereas matching a non-name string with !N will always succeed, regardless of the pattern used.

=NI	The name argument matches the pattern ignoring case.
!NI	The name argument does not match the pattern, not even regardless of case.
=NR	The name argument matches the pattern including case.
!NR	The name argument does not match the pattern when respecting case.

Have a glance at the following examples:

if 'hallo' =SI ha* writelog 'OK' // 1
if 'hallo' =SI *ha* writelog 'OK' // 2
if 'hallo' =SI h*o writelog 'OK' // 3
if 'hallx' !NI ha*o writelog 'OK' // 4
if 'test' !SI ha* writelog 'OK' // 5
if 'test' !SI *ha* writelog 'OK' // 6
if 'test' !SI h*o writelog 'OK' // 7
if 'h-o' !NI h*o writelog 'OK' // 8
if 'test' =SI t*t* writelog 'OK' // 9

Each of these comparisons should come out true, hence causing each statement to print the string "OK". Let's have a look at a few of these examples in detail:

The pattern, "ha*", will match any string starting with "ha" – "hallo" does. So "=SI" will guarantee a positive outcome.

The pattern "h*o" will match any string whose first character is an "h" and whose last character is an "o". Now, "hallx" is no such string, but since we are using a negative comparison operator, "!NI", the whole thing comes out true since the pattern does not match. (Using "!NI" instead of "!SI" presupposes the string to be a well-formed STx name which, by chance, "hallx" is. But compare statement 8.)

The pattern "h*o" normally matches any string starting with "h" and ending with "o". That it does not match in this expression is because the "NI" operator presupposes its argument to be a well-formed STx name. Now, "h-o" is no such name because it contains a minus sign, "-". So, the match fails by necessity, hence causing the negative expression "!NI" to become true.

If you are shaken by statement 8 of the previous example, you might want to try out for yourself the following macro:

if 'h-o' !NI h*o writelog 'OK1' // 1
if 'h-o' !SI h*o writelog 'OK2' // 2
if 'h-o' =NI h*o writelog 'OK3' // 3
if 'h-o' =SI h*o writelog 'OK4' // 4

What this will do is print out "OK1" and "OK4", but neither "OK2" nor "OK3". Do you see why?

Well, first of all, the string "h-o" is no valid STx name due to its containing the minus sign, "-". So every name-based comparison will fail before even considering the pattern argument. So the third comparison, "=NI", will fail though "h-o" is a string starting with "h" and ending with "o". Consequently, the expression inverting that condition (using "!NI" instead of "=NI") will come out true, causing the first comparison to come out true.

With the string-based comparison operators, things are different. Stringly put, the pattern "h*o" does match the string "h-o", that string actually starting with an "h" and ending with an "o". So the positive operator, "=SI", will come out true, while, consequently, its negative form will come out false.

Regular-Expression Pattern Matching

Aside from the simple wildcard-patterns, STx also supports full POSIX regular-expression pattern matching, using the open-source TRE library. Here is not the place for introducing POSIX regular-expressions, but if you are familiar with that concept, you will find the following STx matching operations useful:

Regular-expression string matching

=RSI	The string argument does match the regular expression. Case will be ignored.
!RSI	The string does not match the regular expression, not even regardless of case.
=RSR	The string argument matches the regular expression case-sensitively.
!RSR	The string argument does not match the regular expression when respecting case.

Regular-expression name matching

This, too, is similar to regular-expression string matching, the difference being that the string to match is expected to be a well-formed STx name. If this is not the case, the match will fail, even if the regular expression would string-match the string.

=RNI	The name argument matches the regular expression ignoring case.
!RNI	The name argument does not match the regular expression, not even regardless of case.
=RNR	The name argument matches the regular expression including case.
!RNR	The name argument does not match the regular expression when respecting case.

Building up Complex Expressions

You may combine more than one comparison either conjunctively, using the "&&" operator, or disjunctively, using the "||" operator. There is, though, neither precedence nor grouping, meaning that (a) complex expressions will be evaluated strictly from left to right and that (b) you cannot use brackets for grouping sub-expressions. While the former is not strictly an offence once one gets used to it (other programming languages like APL behave very similarly), the latter may sometimes cause some programmer inconvenience. We will try addressing this issue a wee bit later.

Evaluation stops as soon as the outcome is clear, but with STx, this is only a performance issue (there is no such thing as built-in functions with side-effects, whereas command substitution, on the other hand, will always be done regardless of where the "$(...)" expression actually occurs).

See the following examples:

if 1 > 2 || 3 > 1 writelog 'condition 1 is true'

if 1 > 2 || 3 > 1 && 0 == 1 writelog 'condition 2 is true'

if '$#a+1' < '$#b*3' || '$#c-7' == '$#d' writelog 'condition 3 is true'

Whereas the first statement should be fairly clear, fully to understand the second statement requires being aware of STx strictly evaluating from left to right: For STx, the condition "1 > 2 || 3 > 1 && 0 == 1" will mean the same as, for a human reader, "(1 > 2 or 3 > 1) and 0 == 1" would. The "and" ("&&") does not – as it normally does in logic and mathematics – take precedence over the "or" ("||").

What you type	What STx does
$a > $b && $b < $c || $c < $b && $b < $a	(($a > $b && $b < $c) || $c < $b) && $b < $a
$a < $b && $a > 0 || $b == -1	($a < $b && $a > 0) || $b == -1
$a < $b || $b > $a && $invert == 1	($a < $b || $b > $a) && $invert == 1
$a < $b || ( $b > $a && $invert == 1 )	Syntax error (brackets are not allowed!)

From the theoretical standpoint, the lack of grouping (i.e. brackets) allows for a much faster evaluation. It does, though, sometimes force the programmer either to combine several "if"-statements, or to alter his or her expression – that is why support for bracketing in logical expressions is under active consideration for STx versions to come. For the time being, here are a few examples on how to rewrite complex expressions:

What you mean to do	What you need to do to do what you mean to do
if ($a > $b && $b < $c) || ($c < $b && $b < $a) then    statements… end	#flag := int 0 if $a > $b && $b < $c #flag := int 1 if $c < $b && $b < $a || $#flag == 1 then  statements… end
if $a > 0 || ( $a < 0 && $b == int($b) ) then    statements… end	if $a < 0 && $b == int($b) || $a > 0 then    statements… end
if ($a > $b) && $b < $c) || ($c < $b && $b < $a) ||    ($b < $c && $c < $a) ||    ($b < $a && $a < $c) then    Statements… end	#flag := int 0 if $a > $b && $b < $c #flag := int 1 if $c < $b && $b < $a #flag := int 1 if $b < $c && $c < $a #flag := int 1 if $b < $a && $a < $c || $#flag == 1 then    statements… end

Reference

• Conditional Expressions Reference in the STx Programmers' Guide

While Loops Loop

While loops model our everyday concept of repeating a task until it finally has the desired outcome. With STx, this is done as follows:

while expression

   many statements…

end

With this code, STx will first evaluate the expression supplied to the "while" statement. If this expression happens to be true, all statements between the "while" and the corresponding "end" statement are being executed once. Afterwards, STx again evaluates the very same expression. If it is still true, all the statements between the "while" and the "end" statement are being executed once more. Then, again, the expression is evaluated… and so on, the effect being that all statements between the "while" and the "end" statement are being executed "while" (i.e. as long as) the condition happens to be true.

For Loops to Be Organized

The "for" statement is a more organized loop statement. Its syntax is the following:

for startcmd to whilecondition step changecmd
   many statements…
end

First of all, "startcmd" – some arbitrary command – gets executed. Next, the expression "whilecondition" gets evaluated. If it is false, the control flow will continue with the line immediately following the "end" statement. If, on the other hand, it is true, all the statements between "for" and "end" will be executed. Next, the "changecmd" – again, any command of your choice – will be executed. After that, things will start all over, with "condition" being evaluated again.

This is functionally equivalent to the following code snippet:

startcmd
while whilecondition 
   many statements…
   changecmd
end

So the for statement does not introduce any new functionality. What it does, though, is saving space and making this certain kind of loop more readable.

Note that the for statement is modelled after the like-named statement of fashionable programming languages like C, C++, Java, or the author's all-time favourite AWK. From a semantical point of view, the keyword "to" may seem slightly misleading. Suffice it to say that there is a lesser-known (and these days virtually extinct) Klingon dialect where the word "t'o" means the same as the English imperative phrase "do this while" (at least we trust this statement to be hard to disprove).

As an example, counting from 1 to 10, may serve the following "for" loop:

for #a := int 1 to $#a <= 10 step #a := int $#a + 1
   writelog 'at this stage, #a=$#a'
end
exit

Forever and Ever

People often feel uneasy with the rapid change and the lack of durability in today's fast-paced environment. STx addresses these issues with the "forever" statement that allows for enduring, long-term code execution. In fact, "forever" will start a flow of execution that never ends:

forever
   eternal statements…
end

This kind of loop will cause the statements between "forever" and "end" to be processed for eternity (or until your computer breaks down, whichever happens first).

Ending a Loop Prematurely: Break and Continue

If STx encounters the "break" statement within the body of a loop, this very loop gets at once terminated. Control flow will resume at the statement immediately following the "end" statement at the end of the loop.

The "continue" statement will have a different effect: Whenever STx encounters this kind of statement, it will directly pass to the next run of this loop, ignoring any statements between the "break" statement and the "end" statement at the end of this loop.

Consider the following example:

// print odd numbers less than 20 that are not a multiple of 3

#i := 0 
forever
   #i := int $#i + 1
   if '$#i' >= 20 break

   // note that "int" here is an in-place function truncating the
   // fractional part off a number
   if '$#i/3' == 'int($#i/3)' || '$#i/2' == 'int($#i/2)' continue

   writelog '$#i is neither a multiple of 3 nor even'
end
writelog 'Done.'
exit

Though not at all an example in style, the above loop surely demonstrates the use of both the "break" and the "continue" statement. The basic idea of this loop is to run forever, with each pass increasing #i by one. Immediately after increasing #i is where the "break" statement finds its use: #i is being compared against twenty, and after having reached this upper limit, the whole loop will be terminated. Execution will continue with the first statement after the "end" statement of the respective loop. In our example, this first statement is the call to the built-in "writelog" statement printing out the string "Done."

The next issue demonstrated here is the "continue" statement. After having ensured that #i has not reached its desired upper limit, #i is being checked for being either a multiple of 3, being a multiple of 2, or both. If so happens, STx will "continue" the loop, i.e. omit its remaining part (in our case, the "writelog" statement), and continue with its first statement.

Should the loop to be "continued" be a "for" or "while" loop, the next run of the loop will begin normally, that is with testing the expression supplied to the "for" or "while" statement. Should it prove false, the loop will terminate. In the case of a "for" loop, its "changecmd" (see above), too, will be executed normally, i.e., before checking the condition.

Statements Considered Useful: goto, gosub, and gosubx

Each statement in an STx script may be labelled. The name of a label may consist of letters, digits, and underscores (though it must not start with a digit). The label must be immediately followed by a colon (that's necessary because otherwise there would be no way to discern between labels and macro calls). Note that labels are local to the macro they are part of. It is hence not possible to jump to a label that is part of a macro different from the executing macro (and it is possible to use labels of the same name in more than one macro).

When using a label, you may at any time divert the flow of control to that label. This is done with the "goto" command. Though it has come out of fashion recently, the "goto" command allows for very efficient diversions in the control flow that may be really hard to follow.

Contrary to most other programming languages, STx allows "goto" to be supplied not only one, but even two labels. If this is the case, STx first tries to go to the first label. Only if there happens to be no such label, STx falls back to the second label and tries to go there. If the second label is not present either, the "goto" command will do nothing (it will leave an error code in the variable rc, though, but most programmers do not check for errors anyway).

The following loop will count from 1 to 10 (compare this with the "for" loop of the last example in the previous chapter):

#a := int 0
looping:
   if '$#a' >= 10 goto endloop
   #a := int $#a + 1
   writelog 'at this stage, a=$#a'
   goto looping
endloop:
   writelog 'let it be'
   exit

A different issue is the "gosub" command. It is very similar to calling a macro: A new execution environment will be created, and control will resume at the label referred to by the "gosub" command. When (and if) the control flow reaches an "exit" command, this execution environment will be destroyed, and control will resume with the line immediately following the "gosub" command that caused all that to happen. Summarizing things up, the code called by the "gosub" statement will see all variables empty. Furthermore, any changes made during the statements invoked using "gosub" will get lost on returning. This is a very powerful feature.

The "gosubx" statement differs from the "gosub" command in that no separate execution environment is being created. Hence, the code will see all variables having their current value, and, conversely, any changes in variables done by the called code will remain in effect after returning. This, too, is always a powerful, sometimes a desired feature, and seldom easy to follow.

Compare the effect of the "gosub" and the "gosubx" commands in the following example:

#a := 5
gosub subroutine
writelog 'a after gosubbing subroutine: $#a (no change!)'
gosubx subroutine
writelog 'a after gosubxing subroutine: $#a (changed!)'
exit

subroutine:
   // should #a be unset, numerical evaluation will fail
   #a := int $#a + 5
   writelog 'value of a in subroutine: $#a'
   exit

Note that we do not actually dare to recommend using any of these statements, and that we strongly encourage structured programming (using "for" and "while" loops and breaking program code into macros). There may, however, be cases where the real programmer finds that he or she can do many interesting things with the help of the statements considered harmful since Dijkstra's famous letter.

Shell Items: An Overview

Shell items are a kind of their own. Each shell item has a handle – actually kind of an internal name – that is not normally directly known to the programmer. Shell items may be created using the "new" command, and they may – and should – be disposed of properly after use which is done with the "delete" command. The handle to a shell item is normally stored in an STx variable.

There are different sorts of shell items, the most important being file items (representing disk files), graph items (representing graphs visible to the user), dialog items (representing GUI dialogs, menus, windows, and the like), table items (representing in-memory databases, numerical vectors, and matrices), value items (providing things as diverse as timers, and fast matrix operations), wave items (for handling soundfiles), and instances of user-defined classes. We will come to each of them later (or at least to most of them).

Beginners occasionally find it difficult to grasp the difference between a variable and a shell item. It is often helpful to remember the following properties:

1. Shell items are very complex objects like files or tables that normally consist of many parts (e.g. the rows and columns of a table, or the records of a file) most of which are themselves complex objects.
2. Variables, on the other hand, are very simple and dumb things: They store one string each – nothing complex in that.
3. Shell items are referred to by their handle, variables by their name.
4. You usually store the handle of a shell item in a variable.
5. You usually do not access a shell item directly, but by using the variable storing its item handle.
6. For using a variable, i.e. accessing its contents, you need the dollar sign ("$"). Opposed to this there is no dollar sign with item handles, because there is no such thing as a simple content of an item.

You may at any time request some action from an existing shell item. For doing so, you use the following command:

set itemhandle actionrequest

Here "itemhandle" is the internal handle of the respective STx item, and the exact nature of "actionrequest" depends on what kind of item you are dealing with; you may, e.g., request a table item to add or remove one or more rows, or a file item to write or to read data.

Since requesting some item to do something is such common a task, you may abbreviate it by completely omitting the "set" command. This means that instead of "set itemhandle actionrequest", you may at any time type the following command:

itemhandle actionrequest

So there would actually be no need of even knowing about the "set" command, were it not for the fact that with the STx online help, you find the actions each kind of item supports under the keyword "set itemtype" (set table, SET DIALOG, SET GRAPH, and so on).

Let's have a first simple example on how shell items are normally used:

// create a table whose handle is called "myhandle"
new table myhandle

// set third line (first and second line will be created empty)
set myhandle 2 'third line' // explicitly using "set"

// replace empty first line with a string
myhandle 0 'first line' // not using "set" works just as well

// now replace the second line, too
myhandle 1 'second line'

// show the table
showitem myhandle ; Showing table "myhandle"

// now, just for the fun of it, delete the second row – note that, again,
// either using "set" or omitting it makes no change at all
myhandle myhandle 1 /Delete

// finally, show the reduced table
showitem myhandle; Showing table myhandle" after deleting its second row

delete myhandle // don't forget to delete the item after use

In this example, we create a table item whose handle is the string "myhandle". Any further access to this item is done with this handle. Note that there is no dollar sign when using "myhandle", because "myhandle" is not a variable, but an item handle. If, by mistake, we would use the expression "$myhandle", STx would replace this expression by the contents of the shell-global variable "myhandle" – an empty string in case of there being no such variable, and surely an undesired result. Bluntly put: Item handles and variables reside in different namespaces, and you may well have like-named variables and item handles (of course this is not at all recommended because it may lead to some degree of confusion.)

All that being said, it is, common practice not to choose a handle of one's own, but to let STx do the choosing. In this case, the automatically chosen handle is normally stored in an STx variable:

// create a table item with a handle of STX's choice (e.g. T#27)
// (the asterisk instructs STx to choose a handle on its own).
// the handle will be stored in a variable called "#tab"
#tab := new table *

// set third line (first and second line will be created empty)
$#tab 2 'third line'

// replace empty first line with a string
$#tab 0 'first line'

// now replace the second line, too
$#tab 1 'second line'

// show the table
showitem $#tab ; Showing table "#tab"

// now, just for fun, delete the second row
$#tab 1 /Delete

// finally, show the reduced table
showitem $#tab ; Showing table "#tab" after deleting second row

// old-style deletion
// delete $#tab    // important: delete after use
// #tab := set ''  // clean the variable

// new-style deletion - deletes the shell item whose name is stored in a variable,
//                      and clears the variable
delete /Var #tab   // no dollar here!

N.B.: Whenever actively allocating a shell item (normally with the "new" command), you should always delete it as soon as you no longer need it. If you don't, your macro(s) will consume an unnecessarily high amount of memory which may, for large macro applications, impair performance. There is no such thing as automatic garbage collection in STx.

Shell Tables in Detail

There are three kinds of tables: (a) plain, or simple, tables, (b) parameter tables, and (c) extended tables (I do not mention directory tables here, and I do not even mention that I do not mention directory tables here). plain, or simple, tables are easiest to handle, and they are most useful for storing lists of strings, each table entry being one such string. Extended tables are such that tables more versatile than them cannot be conceived: Extended tables tables may consist of an arbitrary number of columns each of which may be a string, an STx name, an integer, or a general number. Access to each column may be done, at the programmer's discretion, either by its index or by its symbolic name. Parameter tables, in turn, are a somewhat reduced variant of extended tables, in that they store numbers only. On the other hand, parameter tables do so most efficiently, both memory-wise and runtime-wise. This makes parameter tables the preferred means of storing numerical vectors and matrices, and for doing numerical operations of the utmost complicated kinds. The following chapters will deal with each kind of table in detail.

Plain, or Simple, Tables

Plain tables are mainly useful for storing lists of strings. As always, these strings may be of arbitrary content, allowing for e.g. logically storing multiple values per row that are, in turn, retrieved by the "readstr" or even the "readtab" command (the READ family of commands). Be this as it may, with the availability of the more efficient extended tables for database-like structured data, and of parameter tables for structured numerical data like vectors and matrices, storing structured data in plain tables is no longer recommended.

The "new table *" command creates a plain table item. The name of this table item will be stored in the reserved variable "#new". Furthermore, it will be the result of the "new table *" command. The asterisk, though strictly instructing STx to choose an internal name on its own, should be considered a part of the command and should not normally be altered.

Have a look at the following example using simple tables. The macro "tablewrite" will create a table, fill it with a hundred strings, delete every other row, and, finally, save the table to a file. The macro "tableread" will, in turn, re-read the table from the file.

[Macro tablewrite]
   // create the table
   #tab := new table *

   // the "em" command will, after displaying its second argument in an error
   // dialog, immediately terminate the macro, using its first argument as the
   // result of the macro
   if '$#tab[?]' != table em '-1 ; Cannot create table'

   // fill the table with a hundred strings
   for #i := 1 to $#i <= 100 step #i := int $#i+1
      $#tab $#i 'This is string $#i out of 100'
   end

   // now we prove of changing mind and delete every other row
   for #i := 98 to $#i >= 0 step #i := int $#i-2
      $#tab $#i /Delete
   end

   // show the contents of the table
   showitem $#tab ; See Our Table

   // now save the table as a text file called "table_file.txt".
   // the file will be stored in the STx script directory (whose path
   // is available in the shell-global variable "scriptDirectory")
   #fileName := set '$scriptDirectory\table_file.txt'

   // create a text file item – we will come to this in a later chapter,
   // here it is shown only as kind of cliffhanger
   #fileItem := new file * '$#fileName' /Text /Write
   if '$#fileItem' == * em  '-1 ; failed to open the file $#fileName'

   // save the table to the file
   $#fileItem save $#tab

   if '$rc' > 0 em  '-1 ; failed to save table to file $#fileName'

   // dispose of the file item (not the file on disk!) and the table
   delete $#fileItem $#tab

   exit 1 int 0


[Macro tableread]
   // create an empty table
   #tab := new table *
   #fileName := set '$scriptDirectory\table_file.txt'

   // now, create a file item for reading a text file
   #fileItem := new file * '$#fileName' /Text /Read
   if '$#fileItem' == * em -1 ; failed to open the file $#fileName

   $#fileItem load $#tab
   if '$rc' > 0 em  -1 ; failed to load table from file $#fileName

   delete $#fileItem

   // show the loaded table
   showitem $#tab ; See Our Loaded Table

   delete $#tab
   exit 1 int 0

The above macro already makes use of file items we need to leave for later discussion. As you can see, using them for saving and loading tables is fairly easy. You might even at this stage start saving and loading tables of your own without having looked up any details about file items.

Parameter Tables

Parameter tables store numerical data only. They are optimized for numerical throughput and, hence, are the means of choice when it comes to modelling vectors and matrices.

Technically, parameter tables are a variant of the extended tables yet to be mentioned. The reason why we deal with parameter tables first is their optimal fitness for vector and matrix arithmetic. Note that extended tables, when they are all numerical, support exactly the same operations as parameter tables – in fact, numerical-only extended tables and parameter tables are completely interchangeable. Nevertheless, parameter tables are more efficient both runtime-wise and memory-wise. So, as a rule of thumb, always use parameter tables whenever doing vector and matrix arithmetic.

There are two ways of creating a parameter table:

1. explicitly, by using the NEW TABLE … /Parameter command; or
2. implicitly, by using the matrix or vector result of a call to eval.

Here's a simple and semantically quite meaningless example of working with parameter tables created both ways:

// create a 20-element vector initialized with all zeroes
#v0 := eval fill(20,0,0) // 20 elements, first value 0, increment 0

// create and fill a vector
#v1 := eval fill(30,2,4) // 30 elements, first value 2, increment 4

// create and fill a second vector
#v2 := eval fill(30,3,9) // 30 elements, first value 3, increment 9

// add up both vectors
#v3 := eval $#v1 + $#v2

showitem $#v3 ; v3 // show the sum vector

// create a vector with 30 elements each of which is the sine
// of its index (not a useful example, but useful for an example)
#v4 := new table * * number:x /Parameter
for #i := 0 to $#i < 30 step #i := int $#i + 1
   // store the sine of the index, #i, in #a
   #a := eval sin($#i)

   // set the #i-th value of the vector to the value of #a
   $#v4 $#i $#a
end

// transpose a matrix

// create the matrix to transpose – here we must explicitly create the
// matrix because we are going to assign values to its
// individual columns
#mat1 := new table * * number:x:4 /Parameter // 4 columns

// fill each of its individual columns with some values
$#mat1[*,0] := eval fill(30,10,4) // fill the first column
$#mat1[*,1] := eval fill(30,2,1) // fill the second column
$#mat1[*,2] := eval fill(30,0.1,1.2) // fill the third column
$#mat1[*,3] := eval fill(30,1.2,1.2) // fill the fourth column

// now transpose mat1 and store the transposed matrix to mat2
#mat2 := eval trn($#mat1)

// show the transposed matrix
showitem $#mat2 ; transposed matrix

// now, multiply the matrix by the vector and see what happens
#mat3 := eval $#mat2 * $#v3

showitem $#mat3 ; Transposed matrix multiplied by a vector

// create and initialize a matrix in a single step
#mat4 := eval init(30,10,1.2) // 30 rows, 10 columns, value 1.2

showitem $#mat4 ; A matrix containing all 1.2

// important: delete all items not to leak memory
delete /Var #v0 #v1 #v2 #v3 #v4 #mat1 #mat2 #mat3 #mat4

exit

STx Pitfalls

When invocating a macro for the second time, third time, or generally the n-th time, n being any finite positive integer, it will find its namespace empty. This is what you would expect from a full-featured programming language, but it is not what you get from many macro languages inferior to STx, or even from classical languages like the venerable FORTRAN, Basic, or – God forbid – COBOL.

One important implication of this fact is that local variables are not a means for storing data persistently (bluntly put, local variables are no such thing as the local "static" variables of Kernighan's and Ritchie's fine C programming language).

This consistently holds true even when invocating a macro recursively. Its new instance will find all its variables empty, while the calling instance will find its local variables untouched by the macro call.

Hence, local variables are no means at all for passing information to and fro a macro. For passing data to a macro, you should use macro arguments. For a macro to return results, you should use the return mechanism (using the verbose form of the "exit" command). Technically, another means of passing data to and from a macro is using shell-global (or even global) variables, but in the era of structured and modular programming, this is considered harmful to one's career.

Using one and the same variable name in different scopes is technically valid. You may, e.g., have a global variable called @i, a shell-global variable i, and a local variable called #i. Allow for it to be misleading and a source of programmer error, though.

Misspelled variable names tend to be the cause for great confusion. What makes it worse is that there is no way of automatically detecting misspelled variable names: Since variables need not be explicitly declared nor initialized, any syntactically well-formed variable name will be valid at any stage – this is an issue all programming languages without explicit variable declarations share, among them illustrious names like FORTRAN, Basic, or PROLOG. There is no real remedy for this issue other than placing debug statements in the macro source that occasionally print out the values of some important variables, and to check the printed values against one's expectations.

More than the occasional pitfall results from the fact that the expression in an assignment is a rather complex issue. Though this expression is, in last consequence, a string (remember that with STx, all variables are being created equal, i.e. as string variables), the way this string is computed may be a surprisingly complex, heart-warmingly powerful, and sometimes even error-prone affair, bearing the potential for unexpected side-effects. That being said, preventing most kinds of trouble is easy by following two rules of thumb: (a) put everything under apostrophes; and (b) use the appropriate expression type selector when directly assigning a value (as opposed to the result of a function call or a built-in function). So, instead of writing "name := hugo", always type "name := set 'hugo'".

Users, especially those familiar with other programming languages, but not with macro languages like the UNIX shells, often confuse when to prefix the name of a variable with the dollar sign and when not to. The rule is easy: If you want to mention a variable, you do not use the dollar sign. If you want to use a variable, that is, if you want its occurrence be replaced by its actual contents, you do use the dollar sign. Compiled to a rule of thumb: If the variable is the target of an assignment, you do not use a dollar sign. If the variable is the source of an assignment, or if it is the argument to a function or macro call, you do use the dollar sign.

When using string constants, you should generally use quotation, i.e., place them between single quotes.

Where two or more quoted string constants meet (but see the below exception), they will get concatenated without inserting any whitespace characters. This is a standard feature with many programming languages, especially macro languages, but it is sometimes met with surprise. So please be aware that if you want two string constants to be separated by a blank, you will either need to quote the whole string, or to add the whitespace character at the beginning or at the end of the respective string literal. Hence, the statement "#var := set 'abc' 'def'" will set #var to "abcdef", whereas each of the statements "#var := set 'abc ' 'def'", "#var := set 'abc' ' def'", and "#var := set 'abc def'" will set #var to "abc def".

If a quoted string constant is an argument to a statement or to a built-in function (i.e. not to a user macro), there will be no string concatenation. Hence, each quoted string and each unquoted string is exactly one argument to the function.

If a statement or a built-in function expects a numerical argument, this argument may either be a numerical constant or a numerical expression. Either way, it is more secure to quote even a numerical argument (don't quote me on that, though).

It is easy to mix up the reserved variables "rc" and "result". The former ("rc") contains the numerical return code of the STx statement or built-in function most-recently executed (or the reason code why some STx statement could not be executed). The latter ("result") contains whatever information the most recently called user-defined macro (or class method) chose to return.

Unlike most other programming languages and many macro languages, STx evaluates logical expressions strictly from left to right. Furthermore, it does not support grouping (i.e. using brackets). Although this is not a bad thing in general, it may be cause for error and misunderstanding. When writing down a logical expression, be aware that no rules of precedence will apply. If in doubt, use a second (third, fourth…) "if" statement within the scope of each other.

Note that whitespace characters may at any time freely be interspersed within a number. This means that e.g. the string "123 456", although containing a blank character, will be interpreted as the number "123456". This is not a bad thing in itself, but you should be aware of this fact.

Never forget to free an object (both shell-items and user-defined classes) after use. If you don't, and automatic cleanup will only be done at the termination of the running shell which, when using a complex set of macros, may be very late.

Be careful not to mix up internal STx item handles on the one hand and variable names on the other hand. Remember that accessing an STx item by its name means just using this name – there is no dollar sign involved. If, on the other hand, your item handle is stored in an STx variable, you will need to refer to the contents of this variable, i.e. use this variable – here there is a need for using the dollar sign.