Important: This documentation covers Yarn 1 (Classic).
For Yarn 2+ docs and migration guide, see yarnpkg.com.

Package detail

@anush008/tokenizers

Anush008111.9kMIT0.0.0TypeScript support: included

Multi-arch builds of HuggingFace tokenizers

readme