Make space for tree sitter syntax highlighting #212592
Conversation
| private grammarTokens: GrammarTokens | undefined; | ||
| private treeSitterTokens: TreeSitterTokens | undefined; |
There was a problem hiding this comment.
I'd suggest to not have a separate field treeSitterTokens, but have a tree sitter implemenation of GrammarTokens (and introducing an abstract class).
There was a problem hiding this comment.
Sounds good, I will address in an upcoming commit.
| export class TreeSitterTokenizationService implements ITreeSitterTokenizationService { | ||
| readonly _serviceBrand: undefined; | ||
|
|
||
| constructor() { | ||
| // Eventually, this should actually use an extension point to add tree sitter grammars, but for now they are hard coded in core | ||
| this._addGrammar('placeholder-language'); | ||
| } | ||
|
|
||
| private _addGrammar(languageId: string) { | ||
| TreeSitterTokenizationRegistry.register(languageId, {}); | ||
| } | ||
| } |
There was a problem hiding this comment.
I think we should really try to make this available in the monaco-editor as well.
Both to make development easier (with the playground) and to invest into the monaco-editor (which doesn't even have official textmate support, which is very annoying).
|
|
||
| private _onDidChangeContent() { | ||
| // TODO @alexr00 this is an extremely naive implementation and should absolutely not be pushed to main, it's just a placeholder. | ||
| this._tree?.delete(); |
There was a problem hiding this comment.
@hediet this is just temporary. I will implement proper content change handling in an upcoming commit.
| return tokens; | ||
| } | ||
|
|
||
| private findMetadata(captureName: string): number { |
There was a problem hiding this comment.
@hediet, do you know if there's a better way to retrieve the token metadata given a scope? Relying on the ColorThemeData is layer breaking. I started trying to move things around to fix the layer breaker, but it quickly got complicated.
This PR takes the first steps towards adding tree sitter highlighting. I don't expect that there's much that's very interesting in here as there's no actual tree sitter based support in this PR, it's just about getting some of the uninteresting logistics out of the way to make reviewing subsequent PRs easier.