Search for a command to run...
Hi there! Welcome to our second weekly changelog.
Average cache hit response times are now 8.9x faster due to data caching on the server.
Fixed a bug related to traversing cache entries with 100+ children.
We’re hiring a Systems Engineer to work on the performance problems you see above.
As of last week, Butter’s proxy now offers automatic template induction for its response cache! We’ve prepared the following blog post to help explain its significance and potential to help you serve more LLM responses from cache. You can also read t...
Happy Friday! Nothing user-facing to report in this week’s changelog, so hang tight. We continue to invest in internal tooling: evals, infra rewrite, and prepping last week’s automatic template induction POC for production. For fun, here’s a shout-ou...
Welcome to Butter’s eighth changelog! Just like grandma on Thanksgiving morning, we’ve spent a whole lot of time cooking. Starting with: Gramma’s Recipe Book As a Thanksgiving treat, we launched cookwithbutter.com, which uses Butter’s LLM response ca...
Hi all, this week’s changelog is quick and simple, nothing user-visible to announce. We’ve been making R&D progress towards better template-aware-caching, and infra progress rethinking the storage system to work with higher availability. We’ve also p...
Welcome to Butter’s latest weekly changelog. Today’s log is light, as we strengthen our focus on the R&D-side of templated caching. This week, we’ve also seen some more signups, which have brought about a higher volume of requests through Butter’s pr...
