Home
Search
Index
Definify.com
Definition
2024
tokenizer
tokenizer
English
Noun
tokenizer
(
plural
tokenizers
)
(
computing
)
A
system
that
parses
an
input
stream
into its component
tokens
.
Etymology
tokenize
+
-er
Similar Results