Translating Descriptions to C Code
The translator code used to extend Nyquist resides in the trnsrc
directory. This directory also contains a special init.lsp
, so if
you start XLisp or Nyquist in this directory, it will automatically read
init.lsp
, which in turn will load the translator code (which resides
in several files).
Also in the trnsrc
directory are a number of .alg
files, which
contain the source code for the translator (more on these will follow), and
a number of corresponding .h
and .c
files.
To translate a .alg
file to .c
and .h
files, you start
XLisp or Nyquist in the trnsrc
directory and type
(translate "prod")where
"prod"
should really be replaced by the filename (without a
suffix) you want to translate. Be sure you have a saved, working copy of
Nyquist or Xlisp before you recompile!
Note: On the Macintosh, just run Nyquist out of the runtime
directory and then use the Load
menu command to load init.lsp
from the trnsrc
directory. This will load the translation code and change Nyquist's current directory to trnsrc
so that commands like (translate "prod")
will work.
Rebuilding Nyquist
After generating prod.c
and
prod.h
, you need to recompile Nyquist. For Unix systems, you will
want to generate a new Makefile. Modify transfiles.lsp
in your main
Nyquist directory, run Xlisp or Nyquist and load makefile.lsp
.
Follow the instructions to set your machine type, etc., and
execute (makesrc)
and (makefile)
.
Accessing the New Function
The new Lisp function will generally be named with a snd-
prefix,
e.g. snd-prod
. You can test this by running Nyquist. Debugging is
usually a combination of calling the code from within the interpreter,
reading the generated code when things go wrong, and using a C debugger to
step through the inner loop of the generated code. An approach I like is to
set the default sample rate to 10 hertz. Then, a one-second sound has only
10 samples, which are easy to print and study on a text console.
For some functions,
you must write some Lisp code to impose
ordinary Nyquist behaviors such as stretching and time shifting. A
good approach is to find some structurally similar functions and see how
they are implemented. Most of the Lisp code for Nyquist is in
nyquist.lsp
.
Finally, do not forget to write up some documentation. Also, contributions are
welcome. Send your .alg
file, documentation, Lisp support
functions for nyquist.lsp
, and examples or test programs
to rbd@cs.cmu.edu
. I will either put them in the next release or
make them available at a public ftp site.
Why Translation?
Many of the Nyquist signal processing operations are similar in form, but
they differ in details. This code is complicated by many factors: Nyquist
uses lazy evaluation, so the operator must check to see that input samples
are available before trying to access them. Nyquist signals can have
different sample rates, different block sizes, different block boundaries,
and different start times, all of which must be taken into account. The
number of software tests is enormous. (This may sound like a lot of
overhead, but the overhead is amortized over many iterations of the inner
loop. Of course setting up the inner loop to run efficiently is one more
programming task.)
The main idea behind the translation is that all of the checks and setup
code are similar and relatively easy to generate automatically. Programmers
often use macros for this sort of task, but the C macro processor is too
limited for the complex translation required here. To tell the translator
how to generate code, you write .alg
files, which provide many
details about the operation in a declarative style. For example, the code
generator can make some optimizations if you declare that two input signals
are commutative (they can be exchanged with one another). The main part of
the .alg
file is the inner loop which is the heart of the signal
processing code.
Writing a .alg File
To give you some idea how functions are specified, here is the
specification for snd-prod
, which generates over 250 lines of C code:
(PROD-ALG (NAME "prod") (ARGUMENTS ("sound_type" "s1") ("sound_type" "s2")) (START (MAX s1 s2)) (COMMUTATIVE (s1 s2)) (INNER-LOOP "output = s1 * s2") (LINEAR s1 s2) (TERMINATE (MIN s1 s2)) (LOGICAL-STOP (MIN s1 s2)) )A
.alg
file is always of the form:
(name (attribute value) (attribute value) ... )There should be just one of these algorithms descriptions per file. The name field is arbitrary: it is a Lisp symbol whose property list is used to save the following attribute/value pairs. There are many attributes described below. For more examples, see the
.alg
files in
the trnsrc
directory.
Understanding what the attributes do is not
easy, so here are three recommendations for implementors. First, if there is
an existing Nyquist operator that is structurally similar to something you
want to implement, make a copy of the corresponding .alg
file and
work from there. In some cases, you can merely rename the parameters and
substitute a new inner loop. Second, read the generated code, especially
the generated inner loop. It may not all make sense, but sometimes you can
spot obvious errors and work your way back to the error in the .alg
file. Third, if you know where something bad is generated, see if you can
find where the code is generated. (The code generator files are listed in
init.lsp
.) This code is poorly written and poorly documented, but in
some cases it is fairly straightforward to determine what attribute in the
.alg
file is responsible for the erroneous output.
Attributes
Here are the attributes used for code generation. Attributes and values may
be specified in any order.
(NAME "string")
.c
and
string.h
, and the XLisp function generated will be
snd-
string.(ARGUMENTS arglist)
(type1 name1) (type2
name2) ...
, where type and name are strings in double quotes,
e.g. ("sound_type" "s") specifies a SOUND parameter named s
. Note that arglist is not surrounded by parentheses. As seen
in this example, the type names and parameter names are C identifiers. Since
the parameters are passed in from XLisp, they must be chosen from a
restricted set. Valid type names are: "sound_type"
, "rate_type"
, "double"
,
"long"
, "string"
, and "LVAL"
.(STATE statelist)
ARGUMENTS
above), and has the form: (type1 name1
init1 [TEMP]) (type2 name2 init2 [TEMP]) ...
.
The types and names are as
in arglist, and the "inits" are double-quoted initial values. Initial
values may be any C expression. State is initialized in the order implied by
statelist when the operation is first called from XLisp. If TEMP
is omitted the state is preserved in a structure until the sound computation
completes. Otherwise, the state variable only exists at state
initialization time.(INNER-LOOP innerloop-code)
INNER-LOOP-LOCALS
attribute instead. Within innerloop-code,
each ARGUMENT of type sound_type must be referenced exactly one
time. If you need to use a signal value twice, assign it once to a
temporary and use the temporary twice. The inner loop must also assign
one time to the psuedo-variable output. The model here is that the
name of a sound argument denotes the value of the corresponding signal at
the current output sample time. The inner loop code will be called once for
each output sample. In practice, the code generator will substitute some
expression for each signal name. For example,
prod.alg
specifies
(INNER-LOOP "output = s1 * s2")
(s1
and s2
are ARGUMENTS
.) This expands to the
following inner loop in prod.c
:
*out_ptr_reg++ = *s1_ptr_reg++ * *s2_ptr_reg++;
In cases where arguments have different sample rates, sample interpolation
is in-lined, and the expressions can get very complex. The translator is
currently very simple-minded about substituting access code in the place of
parameter names, and this is a frequent source of bugs. Simple string
substitution is performed, so you must not use a parameter or state name
that is a substring of another. For example, if two sound parameters were
named s
and s2
, the translator might substitute for "s" in two
places rather than one. If this problem occurs, you will almost certainly
get a C compiler syntax error. The fix is to use "more unique" parameter
and state variable names.(INNER-LOOP-LOCALS "innerloop-code")
(SAMPLE-RATE "expr")
ARGUMENTS
list. You can also write (SAMPLE-RATE (MAX name1 name2 ...))
where names are unquoted names of arguments.(SUPPORT-HEADER "c-code")
.h
file. The code typically contains
auxiliarly function declarations and definitions of constants.(SUPPORT-FUNCTIONS "c-code")
.c
file. The code typically contains
auxiliarly functions and definitions of constants.(FINALIZATION "c-code")
(CONSTANT "name1" "name2" ...)
CONSTANT
declaration can
eliminate the overhead of storing these registers.(START spec)
(MIN name1 name2 ...)
means the start
time is the minimum of the start times of input signals name1,
name2, .... Note that these names are not quoted.(TERMINATE spec)
(MIN name1 name2 ...)
means
the terminate time is the minimum of the terminate times of input arguments
name1, name2, .... Note that these names are not quoted. To
terminate at the time of a single argument s1
, specify (MIN
s1)
. To terminate after a specific duration, use (AFTER "c-expr")
,
where c-expr is a C variable or expression. To terminate at a
particular time, use (AT "c-expr")
. spec may also be
COMPUTED
, which means to use the maximum sample rate of any input
signal.(LOGICAL-STOP spec)
TERMINATE
. If no
LOGICAL-STOP
attribute is present, the logical stop will coincide
with the terminate time.(ALWAYS-SCALE name1 name2 ...)
ALWAYS-SCALE
attribute.(INLINE-INTERPOLATION T)
INLINE-INTERPOLATION
is not set,
then much less code is generated and interpolation is performed as necessary
by instantiating a separate signal processing operation.(STEP-FUNCTION name1 name2 ...)
(DEPENDS spec1 spec2 ...)
STEP-FUNCTION
list. Consider a filter coefficient that depends upon
an input signal such as bandwidth. In this case, you want to compute the
filter coefficient only when the input signal changes rather than every
output sample, since output may occur at a much higher sample rate. A
spec is of the form
("name" "arg" "expr" [TEMP "type"])
which is interpreted as follows: name depends upon arg; when arg
changes, recompute expr and assign it to name. The name must be
declared as a STATE
variable unless TEMP
is present, in which
case name is not preserved and is used only to compute other state.
Variables are updated in the order of the DEPENDS
list.(FORCE-INTO-REGISTER name1 name2 ...)
#define
'd macro or referenced in a DEPENDS
specification. (NOT-REGISTER name1 name2 ...)
(NOT-IN-INNER-LOOP "name1" "name2" ...)
(MAINTAIN ("name1" "expr1") ("name2" "expr2") ...
)
MAINTAIN
attribute. This suppresses a store of the value of the named variable,
making it a dead variable. Where the store would have been, the expression
is computed and assigned to the named variable. See partial.alg
for
an example. This optimization is never necessary and is only for
fine-tuning.(LINEAR name1 name2 ...)
snd-prod
(signal multiplication) are "linear." The inner loop has
a single multiplication operator to multiply samples vs. a potential 3
multiplies if each sample were also scaled. To handle scale factors on the
input signals, the scale factors are automatically multiplied and the
product becomes the scale factor of the resulting output. (This effectively
"passes the buck" to some other, or perhaps more than one, signal
processing function, which is not always optimal. On the other hand, it
works great if you multiply a number of scaled signals together: all the
scale factors are ultimately handled with a single multiply.)(INTERNAL-SCALING name1 name2 ...)
snd-recip
operation computes
the reciprocal of the input samples by peforming a division. The simple
approach would be to specify an inner loop of output = 1.0/s1
, where
s1
is the input. With scaling, this would generate an inner loop
something like this:
*output++ = 1.0 / (s1_scale_factor * *s1++);
but a much better approach would be the following:
*output++ = my_scale_factor / *s1++
where my_scale_factor
is initialized to 1.0 / s1->scale
.
Working backward from the desired inner loop to the .alg
inner loop
specification, a first attempt might be to specify:
(INNER-LOOP "output = my_scale_factor / s1")
but this will generate the following:
*output++=my_scale_factor/(s1_scale_factor * *s1++);
Since the code generator does not know that scaling is handled elsewhere,
the scaling is done twice! The solution is to put s1
in the
INTERNAL-SCALING
list, which essentially means "I've already
incorporated scaling into the algorithm, so suppress the multiplication by a
scale factor."(COMMUTATIVE (name1 name2 ...))
prod.alg
.(TYPE-CHECK "code")
downproto.alg
for
an example where a check is made to guarantee that the output sample rate is
not greater than the input sample rate. Otherwise an error is raised..c
file defines a number of procedures. The procedures
that do actual sample computation are named something like
name_interp-spec_FETCH, where name is the NAME
attribute from the .alg
file, and interp-spec is an interpolation
specification composed of a string of the following letters: n, s, i, and r.
One letter corresponds to each sound argument, indicating no interpolation
(r), scaling only (s), ordinary linear interpolation with scaling (i), and
ramp (incremental) interpolation with scaling (r). The code generator
determines all the combinations of n, s, i, and r that are necessary and
generates a separate fetch function for each.
Another function is name_toss_fetch
, which is called when sounds
are not time-aligned and some initial samples must be discarded from one or
more inputs.
The function that creates a sound is snd_make_
name. This is
where state allocation and initialization takes place. The proper fetch
function is selected based on the sample rates and scale factors of the
sound arguments, and a sound_type
is returned.
Since Nyquist is a functional language, sound operations are not normally allowed to
modify their arguments through side effects, but even reading samples from a
sound_type
causes side effects. To hide these from the Nyquist
programmer, sound_type
arguments are first copied (this only copies a small structure. The samples themselves are on a shared list). The function
snd_
name performs the necessary copies and calls
snd_make_
name. It is the snd_
name function that is
called by XLisp. The XLisp name for the function is SND-
NAME.
Notice that the underscore in C is converted to a dash in XLisp. Also,
XLisp converts identifiers to upper case when they are read, so normally,
you would type snd
-name to call the function.
Scalar Arguments
If you want the option of passing either a number (scalar) or a signal as
one of the arguments, you have two choices, neither of which is automated.
Choice 1 is to coerce the constant into a signal from within XLisp. The
naming convention would be to DEFUN
a new function named
NAME or S-
NAME for ordinary use. The NAME function tests
the arguments using XLisp functions such as TYPE-OF
, NUMBERP
,
and SOUNDP
. Any number is converted to a SOUND
, e.g. using
CONST
. Then SND-
NAME is called with all sound arguments.
The disadvantage of this scheme is that scalars are expanded into a sample
stream, which is slower than having a special inner loop where the scalar
is simply kept in a register, avoiding loads, stores, and addressing
overhead.
Choice 2 is to generate a different sound operator for each case. The
naming convention here is to append a string of c's and v's, indicating
constant (scalar) or variable (signal) inputs. For example, the
reson
operator comes in four variations: reson
,
resoncv
, resonvc
, and resonvv
. The resonvc
version implements a resonating filter with a variable center frequency (a
sound type) and a constant bandwidth (a FLONUM
). The RESON
function in Nyquist is an ordinary Lisp function that checks types and calls
one of SND-RESON
, SND-RESONCV
, SND-RESONVC
, or
SND-RESONVV
.
Since each of these SND-
functions performs further selection of
implementation based on sample rates and the need for scaling, there are 25
different functions for computing RESON! So far, however, Nyquist is
smaller than Common Lisp and it's about half the size of Microsoft Word.
Hopefully, exponential growth in memory density will outpace linear (as a
function of programming effort) growth of Nyquist.