Camspiers\StatisticalClassifier\Tokenizer\Word::tokenize PHP Method

Word Class Documentation Datei anzeigen Open project: camspiers/statistical-classifier

tokenize() public method

public tokenize ( $document ) 
    public function tokenize($document)
    {
        return preg_split('/\\PL+/u', $document, null, PREG_SPLIT_NO_EMPTY);
    }
Word
tokenize