Camspiers\StatisticalClassifier\Tokenizer\Word::tokenize PHP Méthode

Word Class Documentation Afficher le fichier Open project: camspiers/statistical-classifier

tokenize() public méthode

public tokenize ( $document ) 
    public function tokenize($document)
    {
        return preg_split('/\\PL+/u', $document, null, PREG_SPLIT_NO_EMPTY);
    }
Word
tokenize