| This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto). | |
| https://github.com/proycon/python-ucto | |
| python-ucto-0.6.7 | GPL-3 |
| download~amd64 ~x86 | pypi |