Counting Tokens
Let's look at the prompt that's being sent to OpenAI from the Adventure Time script.
Inside of the handleLLMStart
inside of the new ChatOpenAI
configuration we pass in, there are a couple of parameters we can add.
let llm = new ChatOpenAI({ modelName: "gpt-4", openAIApiKey,
Transcript
00:00 If you want to see what the prompt looks like that's being sent to OpenAI, we can go into our handle.llm.start and the first thing passed into the callback is the LLM itself, the large language model, and the second are the prompts that are being sent. So let's just go ahead and log out these prompts and I'm going to start up our AdventureTime script. And with this open anytime a script is running if we go into the menu bar and look at the running script we can select ViewLog and it'll open the log associated with the script that's running. And if you compare the paths in your kenv, this is under scripts, the name of your script, and this is under logs, the name of your script but with log on the end. So if I go back to Adventure Time and I type I have 245 Chrome tabs open.
00:52 Hit enter. You'll see that our prompt shows up in our log right here. I'm gonna hit option Z to wrap this and you can read what our prompt looks like that's being sent in. If you want to better understand the tokens that are being passed in by your prompts we can use a library called GPT-3 encoder and we'll grab encode off of that. Now just as a side note here, if you open the main menu, hit tab to go to the kit tab, go to manage npm packages.
01:21 You can do things like uninstall an npm package, like if I search for gpt3-encoder, which I already installed, that will uninstall it. And if I go to install an npm package I can pick GPT-3 encoder, hit enter, and install and remove those manually. So now with that encode function let's just add to our logs we'll say tokens and then we'll encode our prompts dot length. This is actually an array so let's join this with an empty string and just the length of whatever encode gives us back should be the amount of tokens that we're sending to OpenAI. So now if we go back to our log here I'll open up AdventureTime.
02:05 I just closed the Most Important tab in the world. Hit enter and you can see all of this above here was 177 tokens which we sent.