# KEHOME/doc/MKRtoken.html
# Mar/9/2007 Apr/10/2009
MKR tokens

MKR tokens
version 7.3

# The MKR parser has three phases
#
# 1. token.icn: get_word(): identify low-level word classes, e.g.
#	B	white space
#	s	separator
#	w	word
# 2. token.icn: get_token(): identify special words and operators, e.g.
#	A	at
#	H	has isin
#	D	do
#	o	od
#	d	done
#	R	relation: isa isa* isa**n ...
#	P	preposition: out of with od from to
#	=	assignment operator: =,+=,-=,*=
# 3. symbol.icn: get_symbol(): identify phrases, sentences, ..., e.g.
#	www;	word phrase
#	wHw=w;	subject has attribute=value
#	wRw;	subject rel object
#	wiwPw=w;subject is  object with attribute=value


#------------------------------------#
# Here's the complete list of tokens #
#------------------------------------#

B	white space: blank tab linereturn newline newpage
N	non separator: ~ / \ ? . $  (included in w)
W	weak separator: + - * :     (included in x)
S	strong separator: , ; = ! & | [ ] { } ( ) < > " ' #
w	word: consecutive bytes not B nor S nor W
x	extended word: consecutive w or W

"	quote mark: " or '
Q	double quote "..." map to q
q	single quote '...'
a	<...> RDF-style comment = multi-line quote
b	(...) balanced parentheses = quote
#	comment mark: #
c	comment

# NOTE:
# For high level parsing (phase 3 in symbol.icn),
# the types above are remapped to
#	B	=> deleted
#	W	=> individual characters
#	S	=> individual characters
#	w	=> w
#	x	=> w
#	"	=> not used
#	q	=> w
#	#	=> not used
#	c	=> deleted


,	list separator: phrase
;	terminator: sentence, proposition

~	OK in word: not
:	OK in word: view:concept, class:number, unit
+	OK in word
-	OK in word
*	OK in word: wildcard

A	at: context -- space,time,view
B	user-defined verb (binary relation)
C	rdfs:Class
D	do can hdo ido vdo happens
d	done
E	else
F	fi
G	generator: every,while,until,when
H	has isin haspart isapart
I	if
i	is isa iss isu isd isi (and inverses: isc isg isp)
J	conjunction: ";" iff implies causes means
j	conjunction: and or
L	rel,urel,brel,trel,nrel
l	HAS
M	!DOCTYPE !ENTITY
N	complement: not non
P	preposition: at out of with od from to
p	rdf:Property
Q	quantifier: a some the no all any either
R	relation verb: isa* isa** ... isc isc* isc** ... in ...
S	assignment: let vlet unlet return
T	then
U	unknown
V	:= production (product := sentence)
w	word not predefined by MKR
x	xml,rdf
Y	group definition: begin
Z	group definition: end
z	exit break

()	grouping: not currently used
[]	begin/end of comma list: phrase
{}	begin/end of semicolon list: proposition
<>	embedded HTML

/	down one level in hierarchy
\	up one level in hierarchy
?	question
!	sh command
$	value of variable
${...}	value of variable
$(...)	product (a la KornShell) -- not currently used
..	dot variable: current genus
.	dot variable: current concept
...	dot variable: current species

=	assignment operator =
=	Set add operator +=
=	Set delete operator -=
=	Set intersection operator *=
V	production operator :=
V	proposition label ::
=	Backus-Naur Form grammar operator ::=


&	and, asynchronous execution (not implemented)
|	or,  pipe (not implemented)
~	not, login directory


##### possible additions #####
m	method
o	od: direct object
u	unit

MKE home