Tokenize a text

Source: R/generics.R, R/methods_document.R

Tokenize a text using the Cortical API

do_tokenize(object, ...)

# S4 method for Document
do_tokenize(object)

Arguments

object

An object of class Document

Value

vector of length n. Here, n is equal to the number of sentences in the text. Each sentence is tokenized using commas.