18 lines
966 B
Markdown
18 lines
966 B
Markdown
|
|
# tokens in Partial Concrete Syntax
|
|
What sort of tokens should I track in the syntax?
|
|
- delimiters like e.g. in `[a, b, c]`?
|
|
- more significant separators like the `:` in `{ "foo" : a }`?
|
|
- What about groupin symbols like `{ ... }`?
|
|
- What about keywords? e.g. `fn` or `for` or `while`?
|
|
|
|
Can these questions be answered universally, or is this application dependent?
|
|
- For example, maybe when building a compiler, we don't need to track so much stuff.
|
|
- But for formatter, we probably need to track a bit more.
|
|
- But what about something like a library in a IDE that handles various transformations of the code?
|
|
|
|
# delimiter confusion
|
|
I just realized that I've been misunderstanding the word `delimiter`.
|
|
I thought that a delimiter was like the `,` in `[ a, b, c]`. But that's called properly called a separator! Or item-separator.
|
|
I thought separator and delimiter where synonyms. But it seems like a `delimiter` is actually the grouping symbols like `[` or `]`.
|
|
|