tree_sitter test coverage + DI seam, ratchet floor to 95

Add `colorForRole` switch-arm tests (every role → token mapping plus
the unknown-role fallback). Introduce a DI seam in `TreeSitterService`
and `TreeSitterLib` so tests can substitute the FFI surface and asset
loaders without dlopen'ing `libtree-sitter.so` —
`TreeSitterLib.testing(...)` takes named per-function overrides with
safe no-op defaults, and `TreeSitterLib.fromDynamicLibrary(...)` lets
the smoke test load the vendored library explicitly. Production
paths (`TreeSitterService.shared`, `TreeSitterLib.instance`) are
unchanged.

Fake-FFI tests walk every branch of `_init`, `_loadGrammar`,
`highlight`, and `dispose`. The smoke test catches FFI-signature
regressions the fakes can't, by exercising the real native library
end-to-end on Linux. Together this takes `tree_sitter_service.dart`
from 17% to 96% and crosses the global 95% target — closing out the
D-66 line-coverage epic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-17 20:21:05 +02:00
co-authored by Claude Opus 4.7
parent e430a87569
commit ab2e5e618b
7 changed files with 691 additions and 15 deletions
+18 -2
View File
@@ -56,6 +56,16 @@ heading, and (b) bumping `pubspec.yaml` `version:` in the same commit.
getters), and `widgets/src/clide_markdown.dart` (h3h6, tables,
strikethrough, default block fallback, record-link tap). Crosses
the 93% line-coverage threshold (T-91).
- `kernel/src/syntax/tree_sitter_service.dart` test sweep — every
`colorForRole` switch arm plus fake-FFI coverage of
`_init`/`_loadGrammar`/`highlight`/`dispose` branches, taking the
file from 17% → 96%. Real-library smoke test
(`test/kernel/src/syntax/tree_sitter_smoke_test.dart`) dlopen's the
vendored `native/linux-x64/libtree-sitter.so`, loads the bundled
`dart` grammar end-to-end, and verifies highlight emits sane spans —
catches FFI signature drift the fake-driven tests can't. Skips on
non-Linux and when the vendored library isn't present. Drives total
line coverage across the 95% target (T-91).
- Staged `dart doc` CI job — generates and uploads an HTML API
reference for the public `lib/` surface. The step wraps
`dart doc --validate-links` and grep-fails the build on any warning,
@@ -109,8 +119,14 @@ heading, and (b) bumping `pubspec.yaml` `version:` in the same commit.
### Changed
- Pre-push line-coverage floor unfrozen and ratcheted to 93% (was held
at 90 on 2026-05-14 by mistake). 95% target restored per D-66.
- Pre-push line-coverage floor ratcheted to 95% — D-66 target hit.
- `TreeSitterService` accepts injectable `TreeSitterLib` and grammar /
query loaders so tests can substitute a fake FFI surface without
dlopen'ing `libtree-sitter.so`. `TreeSitterLib.testing(...)` exposes a
named-parameter constructor with safe no-op defaults for every native
function; `TreeSitterLib.fromDynamicLibrary(...)` lets the smoke test
load the vendored `.so` directly. Production paths
(`TreeSitterService.shared`, `TreeSitterLib.instance`) unchanged.
- Tidied test imports — dropped redundant `dart:ui` / `dart:typed_data`
/ barrel-redundant package imports flagged by `unnecessary_import`.
- Terminal panes now render bold attributes with a real bold weight —
@@ -4,6 +4,7 @@ import 'dart:ffi';
import 'dart:io' show File, Platform;
import 'package:ffi/ffi.dart';
import 'package:flutter/foundation.dart';
// -- Opaque handles ----------------------------------------------------------
@@ -146,6 +147,60 @@ class TreeSitterLib {
wasmEngineNew = lib.lookupFunction<_WasmEngineNew, DWasmEngineNew>('wasm_engine_new'),
wasmEngineDelete = lib.lookupFunction<_WasmEngineDelete, DWasmEngineDelete>('wasm_engine_delete');
/// Constructs a [TreeSitterLib] from caller-supplied Dart closures. Used by
/// tests to substitute the FFI surface without dlopen'ing the real library;
/// each unspecified function defaults to a safe no-op (pointers return
/// `nullptr`, ints return `0`, bools return `false`). Tests override the
/// few entries they exercise.
@visibleForTesting
TreeSitterLib.testing({
DTsParserNew? parserNew,
DTsParserDelete? parserDelete,
DTsParserSetLanguage? parserSetLanguage,
DTsParserSetWasmStore? parserSetWasmStore,
DTsParserParseString? parserParseString,
DTsTreeDelete? treeDelete,
DTsTreeRootNode? treeRootNode,
DTsNodeStartByte? nodeStartByte,
DTsNodeEndByte? nodeEndByte,
DTsQueryNew? queryNew,
DTsQueryDelete? queryDelete,
DTsQueryCaptureCount? queryCaptureCount,
DTsQueryCaptureNameForId? queryCaptureNameForId,
DTsQueryCursorNew? queryCursorNew,
DTsQueryCursorDelete? queryCursorDelete,
DTsQueryCursorExec? queryCursorExec,
DTsQueryCursorNextMatch? queryCursorNextMatch,
DTsWasmStoreNew? wasmStoreNew,
DTsWasmStoreDelete? wasmStoreDelete,
DTsWasmStoreLoadLanguage? wasmStoreLoadLanguage,
DWasmEngineNew? wasmEngineNew,
DWasmEngineDelete? wasmEngineDelete,
}) : parserNew = parserNew ?? (() => nullptr),
parserDelete = parserDelete ?? ((_) {}),
parserSetLanguage = parserSetLanguage ?? ((_, __) => false),
parserSetWasmStore = parserSetWasmStore ?? ((_, __) {}),
parserParseString = parserParseString ?? ((_, __, ___, ____) => nullptr),
treeDelete = treeDelete ?? ((_) {}),
// Leaks a zeroed TSNode allocation — only hit when the test supplies
// a non-null parserParseString without also supplying treeRootNode.
treeRootNode = treeRootNode ?? ((_) => calloc<TSNode>().ref),
nodeStartByte = nodeStartByte ?? ((_) => 0),
nodeEndByte = nodeEndByte ?? ((_) => 0),
queryNew = queryNew ?? ((_, __, ___, ____, _____) => nullptr),
queryDelete = queryDelete ?? ((_) {}),
queryCaptureCount = queryCaptureCount ?? ((_) => 0),
queryCaptureNameForId = queryCaptureNameForId ?? ((_, __, ___) => nullptr),
queryCursorNew = queryCursorNew ?? (() => nullptr),
queryCursorDelete = queryCursorDelete ?? ((_) {}),
queryCursorExec = queryCursorExec ?? ((_, __, ___) {}),
queryCursorNextMatch = queryCursorNextMatch ?? ((_, __) => false),
wasmStoreNew = wasmStoreNew ?? ((_, __) => nullptr),
wasmStoreDelete = wasmStoreDelete ?? ((_) {}),
wasmStoreLoadLanguage = wasmStoreLoadLanguage ?? ((_, __, ___, ____, _____) => nullptr),
wasmEngineNew = wasmEngineNew ?? (() => nullptr),
wasmEngineDelete = wasmEngineDelete ?? ((_) {});
final DTsParserNew parserNew;
final DTsParserDelete parserDelete;
final DTsParserSetLanguage parserSetLanguage;
@@ -181,6 +236,13 @@ class TreeSitterLib {
return true;
}
/// Constructs a [TreeSitterLib] from an already-loaded [DynamicLibrary].
/// Used by the smoke test to dlopen the vendored `libtree-sitter.so`
/// directly without going through the global `init()` / `_instance`
/// dance, so the test stays isolated from the singleton.
@visibleForTesting
static TreeSitterLib fromDynamicLibrary(DynamicLibrary lib) => TreeSitterLib._(lib);
static DynamicLibrary? _openLibrary() {
final libName = Platform.isLinux
? 'libtree-sitter.so'
+55 -12
View File
@@ -8,8 +8,17 @@ import 'package:clide/kernel/src/syntax/language_map.dart';
import 'package:clide/kernel/src/syntax/tree_sitter_ffi.dart';
import 'package:clide/kernel/src/theme/tokens.dart';
import 'package:ffi/ffi.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart' show rootBundle;
/// Loads grammar WASM bytes for [language] (e.g. "dart" → `dart.wasm`).
/// Throws on missing or unreadable assets.
typedef GrammarBytesLoader = Future<Uint8List> Function(String language);
/// Loads the highlight query (`.scm` source) for [language], or returns
/// null if no query is bundled for it.
typedef GrammarQueryLoader = Future<String?> Function(String language);
class SyntaxSpan {
const SyntaxSpan({
required this.start,
@@ -42,8 +51,37 @@ class _LoadedGrammar {
}
class TreeSitterService {
static final TreeSitterService shared = TreeSitterService._();
TreeSitterService._();
static final TreeSitterService shared = TreeSitterService();
/// Production constructor: uses the dlopen'd [TreeSitterLib.instance] and
/// the Flutter [rootBundle]. Tests pass [lib] / [grammarBytes] /
/// [grammarQuery] to substitute a fake FFI surface and in-memory assets.
TreeSitterService({
TreeSitterLib? lib,
GrammarBytesLoader? grammarBytes,
GrammarQueryLoader? grammarQuery,
}) : _injectedLib = lib,
_grammarBytes = grammarBytes ?? _defaultGrammarBytes,
_grammarQuery = grammarQuery ?? _defaultGrammarQuery;
final TreeSitterLib? _injectedLib;
final GrammarBytesLoader _grammarBytes;
final GrammarQueryLoader _grammarQuery;
TreeSitterLib? get _lib => _injectedLib ?? TreeSitterLib.instance;
static Future<Uint8List> _defaultGrammarBytes(String language) async {
final data = await rootBundle.load('assets/grammars/$language.wasm');
return data.buffer.asUint8List();
}
static Future<String?> _defaultGrammarQuery(String language) async {
try {
return await rootBundle.loadString('assets/queries/$language.scm');
} catch (_) {
return null;
}
}
final Map<String, _LoadedGrammar> _grammars = {};
final Set<String> _unavailable = {};
@@ -58,7 +96,7 @@ class TreeSitterService {
if (_initDone) return _parser != null;
_initDone = true;
final lib = TreeSitterLib.instance;
final lib = _lib;
if (lib == null) return false;
final engine = lib.wasmEngineNew();
@@ -92,12 +130,11 @@ class TreeSitterService {
return null;
}
final lib = TreeSitterLib.instance!;
final lib = _lib!;
try {
// Load grammar WASM bytes.
final wasmData = await rootBundle.load('assets/grammars/$language.wasm');
final wasmBytes = wasmData.buffer.asUint8List();
final wasmBytes = await _grammarBytes(language);
// Load into WASM store.
final nameNative = language.toNativeUtf8();
@@ -126,10 +163,7 @@ class TreeSitterService {
calloc.free(error);
// Load highlight query.
String? querySource;
try {
querySource = await rootBundle.loadString('assets/queries/$language.scm');
} catch (_) {}
final querySource = await _grammarQuery(language);
Pointer<TSQuery> query = nullptr;
List<String> captureNames = [];
@@ -200,7 +234,7 @@ class TreeSitterService {
return SyntaxResult.empty;
}
final lib = TreeSitterLib.instance!;
final lib = _lib!;
final parser = _parser!;
final cursor = _cursor!;
@@ -253,7 +287,7 @@ class TreeSitterService {
}
void dispose() {
final lib = TreeSitterLib.instance;
final lib = _lib;
if (lib == null) return;
for (final grammar in _grammars.values) {
@@ -273,6 +307,15 @@ class TreeSitterService {
_unavailable.clear();
}
/// Resets the service to a pre-init state. Tests use this to re-exercise
/// `_init()` without constructing a new singleton; production code never
/// needs it.
@visibleForTesting
void resetForTests() {
dispose();
_initDone = false;
}
static Color colorForRole(String role, SurfaceTokens tokens) {
return switch (role) {
'keyword' || 'repeat' || 'conditional' || 'include' || 'exception' || 'operator' => tokens.syntaxKeyword,
+1 -1
View File
@@ -18,7 +18,7 @@ repository: https://github.com/postmeridiem/clide
# Pre-push line-coverage floor. Ratchets up only — see D-66.
# Reading: `awk -F: '/^coverage_floor:/ {gsub(/ /,"",$2); print $2}' pubspec.yaml`.
coverage_floor: 93
coverage_floor: 95
# Project metadata (was project.yaml, folded in per D-056).
# version: above is the single source of truth. The Makefile reads
@@ -0,0 +1,377 @@
/// Fake-FFI tests for `TreeSitterService` — exercises every branch of
/// `_init`, `_loadGrammar`, `highlight`, and `dispose` by substituting
/// `TreeSitterLib.testing(...)` and in-memory grammar/query loaders, no
/// real `libtree-sitter.so` required.
library;
import 'dart:ffi';
import 'dart:typed_data';
import 'package:clide/kernel/src/syntax/tree_sitter_ffi.dart';
import 'package:clide/kernel/src/syntax/tree_sitter_service.dart';
import 'package:ffi/ffi.dart';
import 'package:flutter_test/flutter_test.dart';
void main() {
// Fake addresses used to stand in for opaque handles. Never dereferenced —
// the fake FFI surface just compares pointer identity.
Pointer<TSWasmEngine> engineHandle() => Pointer<TSWasmEngine>.fromAddress(0x1000);
Pointer<TSWasmStore> storeHandle() => Pointer<TSWasmStore>.fromAddress(0x2000);
Pointer<TSParser> parserHandle() => Pointer<TSParser>.fromAddress(0x3000);
Pointer<TSQueryCursor> cursorHandle() => Pointer<TSQueryCursor>.fromAddress(0x4000);
Pointer<Void> languageHandle() => Pointer<Void>.fromAddress(0x5000);
Pointer<TSQuery> queryHandle() => Pointer<TSQuery>.fromAddress(0x6000);
Pointer<TSTree> treeHandle() => Pointer<TSTree>.fromAddress(0x7000);
/// Builds a lib that succeeds through `_init` (engine + store + parser +
/// cursor) and lets each test customize what happens afterwards.
TreeSitterLib initOkLib({
DTsWasmStoreLoadLanguage? wasmStoreLoadLanguage,
DTsQueryNew? queryNew,
DTsQueryCaptureCount? queryCaptureCount,
DTsQueryCaptureNameForId? queryCaptureNameForId,
DTsParserParseString? parserParseString,
DTsTreeRootNode? treeRootNode,
DTsQueryCursorNextMatch? queryCursorNextMatch,
DTsNodeStartByte? nodeStartByte,
DTsNodeEndByte? nodeEndByte,
DTsQueryDelete? queryDelete,
DTsParserDelete? parserDelete,
DTsWasmStoreDelete? wasmStoreDelete,
DTsQueryCursorDelete? queryCursorDelete,
DTsParserSetLanguage? parserSetLanguage,
}) {
return TreeSitterLib.testing(
wasmEngineNew: engineHandle,
wasmEngineDelete: (_) {},
wasmStoreNew: (_, __) => storeHandle(),
parserNew: parserHandle,
parserSetWasmStore: (_, __) {},
queryCursorNew: cursorHandle,
wasmStoreLoadLanguage: wasmStoreLoadLanguage,
queryNew: queryNew,
queryCaptureCount: queryCaptureCount,
queryCaptureNameForId: queryCaptureNameForId,
parserParseString: parserParseString,
treeRootNode: treeRootNode,
queryCursorNextMatch: queryCursorNextMatch,
nodeStartByte: nodeStartByte,
nodeEndByte: nodeEndByte,
queryDelete: queryDelete,
parserDelete: parserDelete,
wasmStoreDelete: wasmStoreDelete,
queryCursorDelete: queryCursorDelete,
parserSetLanguage: parserSetLanguage ?? ((_, __) => true),
);
}
Future<Uint8List> okBytes(String _) async => Uint8List.fromList(const [0, 1, 2, 3]);
Future<String?> noQuery(String _) async => null;
Future<String?> okQuery(String _) async => '(identifier) @keyword';
group('TreeSitterService._init — FFI failure branches', () {
test('wasmEngineNew returning nullptr → hasGrammar(.dart) is false', () async {
final svc = TreeSitterService(
lib: TreeSitterLib.testing(),
grammarBytes: okBytes,
grammarQuery: noQuery,
);
expect(await svc.hasGrammar('foo.dart'), isFalse);
});
test('wasmStoreNew returning nullptr → hasGrammar is false', () async {
final lib = TreeSitterLib.testing(
wasmEngineNew: engineHandle,
wasmEngineDelete: (_) {},
// wasmStoreNew default → nullptr
);
final svc = TreeSitterService(lib: lib, grammarBytes: okBytes, grammarQuery: noQuery);
expect(await svc.hasGrammar('foo.dart'), isFalse);
});
test('parserNew returning nullptr → hasGrammar is false', () async {
final lib = TreeSitterLib.testing(
wasmEngineNew: engineHandle,
wasmEngineDelete: (_) {},
wasmStoreNew: (_, __) => storeHandle(),
// parserNew default → nullptr
);
final svc = TreeSitterService(lib: lib, grammarBytes: okBytes, grammarQuery: noQuery);
expect(await svc.hasGrammar('foo.dart'), isFalse);
});
test('_init failure is cached — second call still false without re-trying', () async {
var engineCalls = 0;
final lib = TreeSitterLib.testing(
wasmEngineNew: () {
engineCalls++;
return nullptr;
},
);
final svc = TreeSitterService(lib: lib, grammarBytes: okBytes, grammarQuery: noQuery);
expect(await svc.hasGrammar('foo.dart'), isFalse);
expect(await svc.hasGrammar('foo.dart'), isFalse);
// Second hasGrammar hits the _unavailable cache before _init runs again.
expect(engineCalls, 1);
});
});
group('TreeSitterService._loadGrammar — branches', () {
test('grammarBytes throwing is caught → grammar marked unavailable', () async {
final svc = TreeSitterService(
lib: initOkLib(wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle()),
grammarBytes: (_) async => throw StateError('bundle missing'),
grammarQuery: noQuery,
);
expect(await svc.hasGrammar('foo.dart'), isFalse);
});
test('wasmStoreLoadLanguage returning nullptr → grammar marked unavailable', () async {
final svc = TreeSitterService(
lib: initOkLib(),
grammarBytes: okBytes,
grammarQuery: noQuery,
);
expect(await svc.hasGrammar('foo.dart'), isFalse);
// Once marked unavailable, languageFor also returns null.
expect(await svc.languageFor('foo.dart'), isNull);
});
test('grammarQuery returning null → grammar loads with query=nullptr', () async {
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
),
grammarBytes: okBytes,
grammarQuery: noQuery,
);
// languageFor returns the language name when the grammar loaded — even
// though there's no highlight query.
expect(await svc.languageFor('foo.dart'), 'dart');
// hasGrammar likewise returns true.
expect(await svc.hasGrammar('foo.dart'), isTrue);
// highlight returns empty because grammar.query is nullptr.
final h = await svc.highlight('foo.dart', 'x');
expect(h.spans, isEmpty);
});
test('grammarQuery loads + queryNew fails → grammar still cached with query=nullptr', () async {
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
// queryNew default → nullptr; capture reflection block skipped.
),
grammarBytes: okBytes,
grammarQuery: okQuery,
);
expect(await svc.languageFor('foo.dart'), 'dart');
expect((await svc.highlight('foo.dart', 'x')).spans, isEmpty);
});
test('grammarQuery loads + queryNew + captures reflected → loadedLanguages reports it', () async {
// Allocate a static capture name buffer that the fake returns for every
// capture id. Leaks for the duration of the test; not freed.
final nameNative = 'keyword'.toNativeUtf8();
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
queryNew: (_, __, ___, ____, _____) => queryHandle(),
queryCaptureCount: (_) => 1,
queryCaptureNameForId: (_, __, lenOut) {
lenOut.value = nameNative.length;
return nameNative;
},
),
grammarBytes: okBytes,
grammarQuery: okQuery,
);
expect(await svc.languageFor('foo.dart'), 'dart');
expect(svc.loadedLanguages, contains('dart'));
});
test('a second call for the same language returns the cached grammar', () async {
var byteLoads = 0;
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
),
grammarBytes: (lang) async {
byteLoads++;
return Uint8List.fromList(const [0, 1, 2]);
},
grammarQuery: noQuery,
);
await svc.languageFor('foo.dart');
await svc.languageFor('foo.dart');
expect(byteLoads, 1);
});
test('once marked unavailable, repeat calls do not re-attempt the bundle load', () async {
var byteLoads = 0;
final svc = TreeSitterService(
lib: initOkLib(), // wasmStoreLoadLanguage default → nullptr
grammarBytes: (lang) async {
byteLoads++;
return Uint8List.fromList(const [0]);
},
grammarQuery: noQuery,
);
expect(await svc.hasGrammar('foo.dart'), isFalse);
expect(await svc.hasGrammar('foo.dart'), isFalse);
expect(byteLoads, 1);
});
});
group('TreeSitterService.highlight — parse + cursor loop', () {
test('parserParseString returning nullptr → empty spans', () async {
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
queryNew: (_, __, ___, ____, _____) => queryHandle(),
// parserParseString default → nullptr
),
grammarBytes: okBytes,
grammarQuery: okQuery,
);
final h = await svc.highlight('foo.dart', 'whatever');
expect(h.spans, isEmpty);
});
test('cursor produces a match with one capture → SyntaxSpan emitted', () async {
// Build a leaky TSNode for the fake to return — start/end byte
// closures decide the span coordinates.
final rootNode = calloc<TSNode>();
final capNode = calloc<TSNode>();
final captures = calloc<TSQueryCapture>(1);
captures[0].index = 0;
captures[0].node = capNode.ref;
final nameNative = 'keyword'.toNativeUtf8();
var matchCalls = 0;
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
queryNew: (_, __, ___, ____, _____) => queryHandle(),
queryCaptureCount: (_) => 1,
queryCaptureNameForId: (_, __, lenOut) {
lenOut.value = nameNative.length;
return nameNative;
},
parserParseString: (_, __, ___, ____) => treeHandle(),
treeRootNode: (_) => rootNode.ref,
queryCursorNextMatch: (_, match) {
if (matchCalls > 0) return false;
matchCalls++;
match.ref.captureCount = 1;
match.ref.captures = captures;
return true;
},
nodeStartByte: (_) => 4,
nodeEndByte: (_) => 11,
),
grammarBytes: okBytes,
grammarQuery: okQuery,
);
final h = await svc.highlight('foo.dart', 'void hello();');
expect(h.spans, hasLength(1));
expect(h.spans.single.start, 4);
expect(h.spans.single.end, 11);
expect(h.spans.single.role, 'keyword');
});
test('captures with an out-of-range index are skipped', () async {
final rootNode = calloc<TSNode>();
final capNode = calloc<TSNode>();
final captures = calloc<TSQueryCapture>(1);
captures[0].index = 99; // way beyond captureNames.length=1
captures[0].node = capNode.ref;
final nameNative = 'keyword'.toNativeUtf8();
var matchCalls = 0;
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
queryNew: (_, __, ___, ____, _____) => queryHandle(),
queryCaptureCount: (_) => 1,
queryCaptureNameForId: (_, __, lenOut) {
lenOut.value = nameNative.length;
return nameNative;
},
parserParseString: (_, __, ___, ____) => treeHandle(),
treeRootNode: (_) => rootNode.ref,
queryCursorNextMatch: (_, match) {
if (matchCalls > 0) return false;
matchCalls++;
match.ref.captureCount = 1;
match.ref.captures = captures;
return true;
},
),
grammarBytes: okBytes,
grammarQuery: okQuery,
);
final h = await svc.highlight('foo.dart', 'void main() {}');
expect(h.spans, isEmpty);
});
});
group('TreeSitterService.dispose', () {
test('dispose calls queryDelete + parserDelete + wasmStoreDelete + queryCursorDelete', () async {
var queryDeletes = 0;
var parserDeletes = 0;
var storeDeletes = 0;
var cursorDeletes = 0;
final svc = TreeSitterService(
lib: initOkLib(
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
queryNew: (_, __, ___, ____, _____) => queryHandle(),
queryDelete: (_) => queryDeletes++,
parserDelete: (_) => parserDeletes++,
wasmStoreDelete: (_) => storeDeletes++,
queryCursorDelete: (_) => cursorDeletes++,
),
grammarBytes: okBytes,
grammarQuery: okQuery,
);
// Load one grammar so dispose has something to walk.
await svc.hasGrammar('foo.dart');
svc.dispose();
expect(queryDeletes, 1);
expect(parserDeletes, 1);
expect(storeDeletes, 1);
expect(cursorDeletes, 1);
});
test('dispose is a no-op when no lib was ever injected and TreeSitterLib.instance is null', () {
// No injected lib + no dlopen'd native lib in the test runner.
final svc = TreeSitterService(grammarBytes: okBytes, grammarQuery: noQuery);
svc.dispose(); // must not throw
});
test('resetForTests re-arms _init for the next call', () async {
var engineCalls = 0;
final lib = TreeSitterLib.testing(
wasmEngineNew: () {
engineCalls++;
return engineHandle();
},
wasmEngineDelete: (_) {},
wasmStoreNew: (_, __) => storeHandle(),
parserNew: parserHandle,
parserSetWasmStore: (_, __) {},
queryCursorNew: cursorHandle,
wasmStoreLoadLanguage: (_, __, ___, ____, _____) => languageHandle(),
);
final svc = TreeSitterService(lib: lib, grammarBytes: okBytes, grammarQuery: noQuery);
await svc.hasGrammar('foo.dart');
expect(engineCalls, 1);
svc.resetForTests();
await svc.hasGrammar('bar.dart');
expect(engineCalls, 2);
});
});
}
@@ -6,8 +6,11 @@
library;
import 'package:clide/kernel/src/syntax/tree_sitter_service.dart';
import 'package:clide/kernel/src/theme/tokens.dart';
import 'package:flutter_test/flutter_test.dart';
import '../../../helpers/kernel_fixture.dart';
void main() {
TestWidgetsFlutterBinding.ensureInitialized();
@@ -63,4 +66,99 @@ void main() {
expect(s.role, 'keyword');
});
});
group('TreeSitterService.colorForRole', () {
late SurfaceTokens tokens;
setUpAll(() async {
final f = await KernelFixture.create();
tokens = f.services.theme.current.surface;
await f.dispose();
});
test('every keyword-flavoured role maps to syntaxKeyword', () {
for (final role in ['keyword', 'repeat', 'conditional', 'include', 'exception', 'operator']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxKeyword, reason: role);
}
});
test('type-flavoured roles map to syntaxType', () {
for (final role in ['type', 'type.builtin', 'constructor']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxType, reason: role);
}
});
test('string-flavoured roles map to syntaxString', () {
for (final role in ['string', 'string.special']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxString, reason: role);
}
});
test('number-flavoured roles map to syntaxNumber', () {
for (final role in ['number', 'float', 'boolean']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxNumber, reason: role);
}
});
test('comment maps to syntaxComment', () {
expect(TreeSitterService.colorForRole('comment', tokens), tokens.syntaxComment);
});
test('function-flavoured roles map to syntaxMethod', () {
for (final role in ['function', 'function.builtin', 'function.method', 'method']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxMethod, reason: role);
}
});
test('punctuation roles map to syntaxPunct', () {
for (final role in ['punctuation.bracket', 'punctuation.delimiter', 'punctuation.special']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxPunct, reason: role);
}
});
test('variable roles fall back to globalForeground', () {
for (final role in ['variable', 'variable.builtin', 'variable.parameter']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.globalForeground, reason: role);
}
});
test('property / field share syntaxMethod with functions', () {
for (final role in ['property', 'field']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxMethod, reason: role);
}
});
test('constant roles share syntaxNumber with numbers', () {
for (final role in ['constant', 'constant.builtin']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxNumber, reason: role);
}
});
test('tag / attribute share syntaxKeyword', () {
for (final role in ['tag', 'attribute']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxKeyword, reason: role);
}
});
test('namespace / module share syntaxType', () {
for (final role in ['namespace', 'module']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxType, reason: role);
}
});
test('markdown text.* roles distribute across keyword / string / type tokens', () {
expect(TreeSitterService.colorForRole('text.title', tokens), tokens.syntaxKeyword);
for (final role in ['text.literal', 'text.reference', 'text.uri']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxString, reason: role);
}
for (final role in ['text.emphasis', 'text.strong']) {
expect(TreeSitterService.colorForRole(role, tokens), tokens.syntaxType, reason: role);
}
});
test('unknown roles fall through to globalForeground', () {
expect(TreeSitterService.colorForRole('definitely-not-a-real-role', tokens), tokens.globalForeground);
expect(TreeSitterService.colorForRole('', tokens), tokens.globalForeground);
});
});
}
@@ -0,0 +1,80 @@
/// Real-library smoke test for `TreeSitterService`. Dlopen's the vendored
/// `native/linux-x64/libtree-sitter.so`, loads the bundled `dart` grammar
/// from `assets/grammars/dart.wasm` + `assets/queries/dart.scm` straight off
/// the filesystem (no `rootBundle`), and verifies highlight produces sane
/// spans over a tiny Dart program. Catches FFI-signature regressions that
/// the fake-driven branch tests cannot.
///
/// Skips cleanly on non-Linux hosts and on Linux hosts where the vendored
/// library hasn't been built yet.
library;
import 'dart:ffi';
import 'dart:io';
import 'package:clide/kernel/src/syntax/tree_sitter_ffi.dart';
import 'package:clide/kernel/src/syntax/tree_sitter_service.dart';
import 'package:flutter_test/flutter_test.dart';
const _libPath = 'native/linux-x64/libtree-sitter.so';
void main() {
if (!Platform.isLinux || !File(_libPath).existsSync()) return;
group('TreeSitterService — native smoke (real libtree-sitter.so + dart grammar)', () {
late TreeSitterService svc;
setUpAll(() {
final dylib = DynamicLibrary.open(_libPath);
svc = TreeSitterService(
lib: TreeSitterLib.fromDynamicLibrary(dylib),
grammarBytes: (lang) async => File('assets/grammars/$lang.wasm').readAsBytes(),
grammarQuery: (lang) async {
final f = File('assets/queries/$lang.scm');
return await f.exists() ? f.readAsString() : null;
},
);
});
// Intentionally do NOT call svc.dispose() at teardown — wasmtime's
// store-delete path collides with the Flutter test runner's process
// finalization (libc `double free` on exit). The OS reclaims everything
// when the runner process exits.
test('loads the dart grammar end-to-end', () async {
expect(await svc.hasGrammar('main.dart'), isTrue);
expect(await svc.languageFor('main.dart'), 'dart');
expect(svc.loadedLanguages, contains('dart'));
});
test('highlight returns at least one span for a tiny dart program', () async {
const source = 'void main() {\n print("hi");\n}\n';
final r = await svc.highlight('main.dart', source);
expect(r.spans, isNotEmpty);
// Every span must be in-range and well-ordered.
for (final s in r.spans) {
expect(s.start, lessThanOrEqualTo(s.end));
expect(s.end, lessThanOrEqualTo(source.length));
expect(s.role, isNotEmpty);
}
// The captures the upstream dart.scm emits cover at least one of these
// semantic roles for a `void main()` program — assert intersection so
// the test survives minor query reshufflings.
final roles = r.spans.map((s) => s.role).toSet();
expect(
roles.intersection({'keyword', 'type', 'function', 'string', 'punctuation.bracket', 'punctuation.delimiter'}),
isNotEmpty,
reason: 'expected at least one familiar dart role, got: $roles',
);
});
test('a second highlight reuses the cached grammar (no re-load)', () async {
// Re-issue a highlight; the grammar cache hit is internal but if the
// service is healthy this just completes without throwing and returns
// a sane result.
final r = await svc.highlight('main.dart', 'class X {}');
expect(r.spans, isNotEmpty);
});
});
}