LLM token counter
- Tokens: 0
- Words: 0
- Characters: 0
Token counting is based on OpenAI's tiktoken project, you can find more info here: https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb .
Token counting is based on OpenAI's tiktoken project, you can find more info here: https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb .