Exmeralda.chat - A Chatbot for Hex Packages
In this blog, we have a ongoing series of posts about the uses and the technicalities of RAG, especially in the context of coding and Elixir. However, we discovered that many alchemists did not find the time to experiment with their own RAG system. To make this more accessible, we launched exmeralda.chat. It's a chatbot for Hex packages that we host for the community.
The Motivation
"AI in 2025"™ is much less about the large language models (LLMs) themselves, but much more about the system around the LLMs. Even the models of big commercial players like OpenAI or Anthropic are not a single model anymore. They are comprised of a system of models and tools. Elixir is great for building stable, production-ready systems. If AI should contribute to the adoption of Elixir, we need to make it easier to build AI systems with it, and we need to make sure that the techniques are well understood in the Elixir community.
One of the biggest challenges in building AI systems is to make the LLMs operate in the context of the desired domain. The most basic, yet still powerful, way to ground an LLM in a specific domain is to use RAG.
To make it easier to build RAG systems, we created rag, a library to structure RAG systems in Elixir. See this blog post about it.
Now, we want to take this a step further and make it easier to get in touch with RAG: Meet Exmeralda.chat.
Exmeralda.chat
Exmeralda.chat is a chatbot for Hex packages. You can ask it questions about packages on Hex.pm, how to use them, and how to solve problems with them. It is thought to be a bridge between a web search that often does not understand the problem, and using ChatGPT, which often does not understand the context.
It is powered by a RAG system that uses the rag library. We wanted to make the code easy to understand. So the mechanism is very basic. You can have a look at the source code on GitHub. You should find it very accessible.
Currently, it does the basic steps of:
- Ingestion of the package documentation and code into chunks in a vector database
- Retrieval of the most relevant chunks
- Augmenting the prompt with the retrieved information
- Answering questions through augmented generation
Never the less, it is already quiet powerful.
Behind the scenes
We pretty much follow the steps outlined in our blog post about RAG. We added a bit more beauty to the chatbot interface, and you can now select packages and versions you would like to have ingested.
How to use it
Sign up
Getting started is easy. In order to control bandwith, we need to put the system behind a login. We decided to use github for authentication. You can login with your github account, and you will need to agree to the terms of service.
Once you are logged in, you can start chatting with Exmeralda. First, you select the package and the version you are interested in:
Then, you can start chatting.
First question
This starts a new conversation. You can have multiple conversations at the same time. You can experiment with this: How are results different if you mix topics vs. keeping on topic per conversation?
Missing a package?
If you find you are missing a package, you can ask Exmeralda to add it:
There is a queue for package ingestion. Some packages take longer to ingest, as they have a lot of documentation and code. We are working on a way to visualize queue and the ingestion progress. For now, please allow some time to pass. Ingestion is the "compiling" of the 2020ies, so stretch, workout or walk what pets you might have in the meantime.
Sometimes, the ingestion gets derailed by files with an unexpected format or encoding. We are working on that as well. If the package is not there after a day, please feel free to post an issue on GitHub.
(Community) Work in Progress
We are very curious what you think of it. If you like it, please tell us and other people about it. If you find something odd or missing, please give us feedback via GitHub issues. We will implement an in-app feedback to the chat answers soon.
The strongest approval, though, is action: The system is open source, and we would love contributions. It would be great if we could run this thing together :)
Benefits for package creators
Writing documentation is a lot of work. Often, creators are experts in their field, while adopters are not. The two different perspectives make it hard to write documentation that bridges the gap.
Exmeralda is a chatbot that can answer questions about a package. It draws its knowledge from the package's documentation, but also from the code of the package itself. Therefore, it can answer questions that are not covered by the documentation. In a later step, we want to give package creators access to an anonymized version of the chat data, so they can better understand what questions people have.
Benefits for package adopters
Often, package adopters do not know what they can do with a package. Am I at the right place? What can I do with this package? Does this solve my problem? Exmeralda can help with this. It may also be able to answer questions that can not be found in web search, and are to specific for general chatbots.
Wrap up
We've covered a lot of ground in the previous blog posts. Now, we want to get people to interact with the technology. At the same time we want to make it easier for package creators to write documentation. Through the exchange of data, creators will understand better what questions adopters have, and what challenges they face.
This is an ongoing project. We want to see if people find it useful, how many people interact with it, and how much collaboration it will spark.
As of now, Exmeralda is a simple system, designed as showcase for RAG and AI with Elixir, and as a tool for the community. So far, people found it "surprisingly useful"
We hope you will too!
Elixir is an excellent choice for applications due to its scalability, fault tolerance, and concurrency model. Its lightweight processes and message-passing architecture make it ideal for orchestrating complex AI workflows efficiently. bitcrowd's first Elixir ML project dates back to 2020, and we have since then enabled various clients to build and scale their AI projects.
bitcrowd is an excellent choice if you need a scalable RAG system or a fully integrated AI pipeline. We help you build, optimize, and maintain it with a focus on reliability and performance.
Drop us a line via email if you want to build your next AI project with Elixir. Or book a call with us to discuss your project.