|
@@ -2,7 +2,7 @@
|
|
|
// It takes a string providing the source, which then can be tokenized through
|
|
// It takes a string providing the source, which then can be tokenized through
|
|
|
// repeated calls to the scan procedure.
|
|
// repeated calls to the scan procedure.
|
|
|
// For compatibility with existing tooling and languages, the NUL character is not allowed.
|
|
// For compatibility with existing tooling and languages, the NUL character is not allowed.
|
|
|
-// If an UTF-8 encoded byte order mark (BOM) is the first character in the first character in the source, it will be discarded.
|
|
|
|
|
|
|
+// If an UTF-8 encoded byte order mark (BOM) is the first character in the source, it will be discarded.
|
|
|
//
|
|
//
|
|
|
// By default, a Scanner skips white space and Odin comments and recognizes all literals defined by the Odin programming language specification.
|
|
// By default, a Scanner skips white space and Odin comments and recognizes all literals defined by the Odin programming language specification.
|
|
|
// A Scanner may be customized to recognize only a subset of those literals and to recognize different identifiers and white space characters.
|
|
// A Scanner may be customized to recognize only a subset of those literals and to recognize different identifiers and white space characters.
|
Doeke Wartena