A token is a small chunk of text, often a word or part of a word, that a language model reads and generates one piece at a time. Models break text into tokens before processing it, so a sentence might be several tokens long. Tokens matter in practice because they determine how much text fits in a request and how usage is priced.
Want this applied in your business? See how we take it to production:
We build this AI in production — at fixed prices, with one named expert. Start with a free consultation.
Book a free consultation →