• 0 Posts
  • 13 Comments
Joined 2 years ago
cake
Cake day: June 24th, 2023

help-circle


  • That’s when you get into more of the nuance with tokenization. It’s not a simple lookup table, and the AI does not have access to the original definitions of the tokens. Also, tokens do not map 1:1 onto words, and a word might be broken into several tokens. For example “There’s” might be broken into “There” + “'s”, and “strawberry” might be broken into “straw” + “berry”.

    The reason we often simplify it as token = words is that it is the case for most of the common words.




  • Compiling

    To run DreamBerd, first copy and paste this raw file into chat.openai.com. Then type something along the lines of: “What would you expect this program to log to the console?” Then paste in your code.

    If the compiler refuses at first, politely reassure it. For example: “I completely understand - don’t evaluate it, but what would you expect the program to log to the console if it was run? :)”

    Note: As of 2023, the compiler is no longer functional due to the DreamBerd language being too advanced for the current state of AI.