Get index from position
tokenIndexFromPos method returns the index, in the sequence of tokens and atoms identified by the disambiguation, of the token whose text contains a character occupying a given position.
For example considering the text:
Michael Jordan was one of the best basketball players of all time.
character at position 41 is the second b of basketball players:
Michael Jordan was one of the best basketball players of all time. ↑ 012345678901234567890123456789012345678901234567890123456789012345 0 1 2 3 4 5 6
so this statement:
var tokenIndex = DIS.tokenIndexFromPos(41);
tokenIndex variable to 8, which is the zero-based index of the basketball players token in the sequence of tokens and atoms identified by disambiguation.
When a token is composed of atoms, the same character is part of both the token and one of its atoms, but the
tokenIndexFromPos method always returns the index of the token.
The syntax is:
position is the zero-based position of a character in the document text.
Token index can be used to get the text of the token, get the token object or tag/untag the token.