Skip to main content

Behind the Scenes: Replacing a Black-Box Billing Service

A behind-the-scenes dive into a project at bitcrowd: how we broke down the planning, implementation and release of Charger, the replacement billing service of our client Steady Media.

· 5 min read
Agathe Lenclen portrait

This could be your product’s story. We bring together strategy, design, and development to launch products that perform. Do you have a similar idea? Wondering how this would work for your application? Let’s talk!

In a previous blog post, we looked at the process of migrating 600k users to a new billing service for Steady Media. This was the final step of an internal project called “Charger”. Its aim was to replace the off-the-shelf payment platform they had started with, Chargebee, which had become a roadblock. In this post, we will reveal the behind-the-scenes insights into the process that made that final step a success.

The Challenge

Steady Media needed to migrate 600,000 users from Chargebee to a new in-house billing system. The complexity was brutal:

  • Chargebeeʼs billing logic is opaque - we had to reverse-engineer behavior through observation
  • Historical data had to work in the new system (canʼt rewrite the past, especially not payment transactions!)
  • Multiple payment gateways (Braintree, GoCardless, PayPal) with different quirks
  • Complex billing workflows: trial subscriptions, gift subscriptions, mid-term subscription upgrades, subscription cancellations (with or without refunds)
  • Complex support workflows: issuing refunds, invoice & credit notes document versioning
  • Legal compliance (invoicing and taxes)
  • Migration had to happen without downtime or data loss

Steadyʼs development team asked us to lead the Charger project, so that they could focus on their core application. The tech stack had to be compatible with their team skills: Elixir & Phoenix!

The Journey

bitcrowd built Charger across three iterations:

2021: Core System

The first iteration aimed at building the data model and implementing the external communication layer of Charger:

  • API layer with authentication between Charger and the Media Makers app
  • Webhook notifications between Charger and the Media Makers app
  • Payment providers: integrated Braintree and GoCardless services (API & webhooks)
  • Multiplexed calls from the Media Makers app to hit Chargebee OR Charger depending on where the user lived

Once these various components were finalized, we could start the implementation of the core flows: create new subscriptions, bill users and trigger payments, upgrade subscription plans with prorated billing, settle payments, and so on.

2023: Feature Parity

After the first development cycle ended, new features were added to the Media Makers app that needed to be back-ported to Charger. We added new models and new flows to support trial and gift subscriptions. We took the opportunity to add Paypal as a third payment provider. Finally, we also tackled the generation and design of legal billing PDF documents.

2025: Admin UI, Support Tools, and Migration

A missing piece in Charger was a simple, lightweight, workable admin UI, that could be useful for Steadyʼs developers, QA engineers, support staff and financial experts. The goal was to minimize the implementation effort, while preserving a clean UX. We opted for DaisyUI since we knew the admin UI would use standard components like index tables, description lists, navigation elements etc.

One strong requirement was to make sure that the index pages had a powerful and performant search as well as sorting and filtering mechanisms, as one of the pain points of Chargebee had been searches that time out once enough data was in the system. We solved this issue by paying attention to the queries' efficiency. Additionally, for cross-table filtering, we implemented a search-typeahead dropdown in order to avoid expensive joins.

Activity logs

For such a billing product, it was crucial to allow admin users to see a clear changeset history on any resource. Charger comes with an auditing layer, that tracks changes and versions each relevant resource (like payments, invoices...). We built an abstraction on top of the auditing, that allows to plug any resource in it and view its version history with a UI resembling a Git diff.

Support staff tooling

At that stage, we had a working product for developers, QA engineers and financial experts. But Steadyʼs support team still needed tailored features to enable them to solve specific situations. For example, when a userʼs credit card expired, and the system could not withdraw money from their account, the user might send the missing amount via a bank transfer. On Steadyʼs bank account, the balance is correct, but Charger does not know about this transaction, and the accounting balance is affected. Similarly, the support team might refund a user via a manual bank transfer. To keep the accounting in check, we implemented a solution to record offline refund & payment, as well as various manual actions & flows to fix any situation that would not auto-heal.

Outcome

We migrated all 600,000 users to Charger. The lights stayed green. Steady now controls their billing infrastructure and its support team can handle edge cases and requests through the admin UI. The developer team can also leverage the activity logs and powerful search capabilities to debug or resolve errors.

bitcrowd led on self-contained packages: we took topics Steadyʼs team didnʼt have capacity for and delivered them finished. We were able to coordinate with stakeholders (support team, finance, legal) to turn “we need X” into actionable tickets. We applied our standards of excellent documentation and test coverage, high code quality and review, which ensured a smooth handover with Steadyʼs team.

Tags:

This could be your product’s story. We bring together strategy, design, and development to launch products that perform. Do you have a similar idea? Wondering how this would work for your application? Let’s talk!

Agathe Lenclen portrait

Agathe Lenclen

Pattern Matching Sandwich Artist

We’re hiring

Work with our great team, apply for one of the open positions at bitcrowd