pragmatic_tokenizer 3.2.0

A multilingual tokenizer to split a string into tokens.

Gemfile:
=

install:
=

Versions:

3.2.1 August 11, 2024 (71 KB)
3.2.0 November 29, 2020 (71 KB)
3.1.0 July 10, 2020 (70.5 KB)
3.0.7 March 18, 2018 (70.5 KB)
3.0.6 March 15, 2018 (72.5 KB)

Show all versions (55 total)

Runtime Dependencies (1):

unicode >= 0

Development Dependencies (5):

bundler ~> 1.9

rake >= 12.3.3

rspec >= 0

rubocop >= 0

stackprof >= 0

Show all transitive dependencies

Owners:

Pushed by:

Authors:

Kevin S. Dias

SHA 256 checksum:

=