provideTokenCount
abstract fun provideTokenCount(model: T, text: JsAny, token: CancellationToken): PromiseLike<JsInt>(source)
Returns the number of tokens for a given text using the model-specific tokenizer logic
Parameters
model
The language model to use
text
The text to count tokens for
token
A cancellation token