optimizing token usage in long-context models

Introduction TL;DR AI models have grown dramatically. Models like Gemini 1.5 Pro and Claude 3.5 Sonnet now support context windows of 1…