Skip to main content

Software development insights compiled for you by the people of bitcrowd

· 14 min read
Christoph Beck portrait
Joel Koch portrait
With logits processing, you can force open source LLM to comply with a schema you provide. Grammar Constrained Decoding saves you the process of resubmitting malformed LLM responses. Here we show the process with Bumblebee, and explain the background. Want a specific JSON format reliably? Read on!

· 10 min read
Christoph Beck portrait
Joel Koch portrait
Ever tried to get a poem from an LLM doesn't contain the letter `e`? Large Language Models are kind of amazing and surprisingly unreliable at the same time. Using Elixir's Bumblebee and open source LLMs, you can get much better control over the generation. In this article, we introduce you to logits processing, and how it can be used to achieve what ChatGPT can not accomplish. Find out how!

· 8 min read
Christoph Beck portrait
Your search works… but is it actually good? In our previous post, we showed why full-text search struggles with the long, natural-language queries common in MCP interaction. In this follow-up, we dive into dataset generation, evaluation, and search improvement. How do you know if a new embedding model is really better?"