In an organization that heavily adopts Apache Kafka and event-driven architecture, Kafka events become a critical asset — enabling the development of various applications that react to real-time data.

However, not every developer wants or needs to interact with Kafka directly. To make Kafka more accessible and unlock its full potential across the organization, we built a platform called Firehose.

Firehose simplifies Kafka consumption by exposing events through a familiar interface: HTTP webhooks. Developers can use a self-serve UI to select the Kafka events they want to subscribe to, and Firehose will automatically deliver those events to their configured webhook endpoint. It also handles scaling, monitoring, and lifecycle management out of the box — all self-serve from the UI.

While Firehose empowers teams to consume events using their own logic, we also built Dagger — a no-code stream processing platform. With Dagger, developers can transform Kafka events using SQL, without writing or deploying custom application code. Dagger powers many critical systems, including supply-demand aggregation for surge pricing, fraud detection, and ad impression pipelines — all through declarative SQL, without the overhead of managing services.

Clickstream data is essential for marketing analytics and product development. However, collecting and managing this data presents several challenges. The ingestion system must be scalable to handle the high volume of user-generated events. It must also support flexible schemas to accommodate variations across event types. Additionally, the high frequency of incoming data requires careful selection of the communication protocol to ensure efficiency.

To address these challenges, we developed a clickstream ingestion platform called Raccoon.

Raccoon is a horizontally scalable, Go-based service that ingests clickstream data over the WebSocket protocol. It receives events from clients and routes them to Kafka topics for downstream processing. Built for scale and flexibility, Raccoon became the foundation for Gojek’s internal clickstream pipeline — enabling the company to replace a third-party SaaS solution that previously cost over $5 million USD annually.

Being able to define customer segments based on user behavior and attributes is essential for effective marketing. Traditionally, creating these segments requires marketers to identify data sources, write SQL queries, and manually execute them — a process that is tedious, error-prone, and time-consuming.

To remove this burden, we developed the Customer Segmentation Engine — a system that enables marketers to define segments declaratively based on customer properties and past behaviors, without writing SQL. Once defined, a segment can be executed to generate a list of customers who match the criteria. Segment definitions are reusable and can be saved for future campaigns, allowing marketers to focus on evaluation and optimization rather than data wrangling.

The engine accepts a segment definition in JSON format. When executed, it compiles the definition into SQL, runs the query, and prepares the result as downloadable CSV files. The engine supports complex logic including nested conditions, logical operators, time-based user behavior filters, and inclusion/exclusion rules.

This system has become a foundational tool for segmentation across the organization, saving hundreds of hours in manual effort and significantly accelerating campaign execution.

An internal platform was developed to support analytics on top of clickstream data, primarily for product analytics. It allowed users to visualize trends, build dashboards, browse a catalog of clickstream events, and trace individual user activity.

While the platform had some adoption, it suffered from critical issues:

• Trend data was often inaccurate and stale.
• The event catalog was frequently outdated.
• User-level event tracing was extremely slow and frustrating to use.
• The support channel was overwhelmed with complaints and bug reports. To address these challenges, I led a full system redesign. After the redesign:

Data accuracy was significantly improved, restoring trust in reported metrics.

• System latency was reduced from minutes to seconds.
• Data freshness is now near real-time.
• The event catalog is automatically and reliably updated.
• Support requests dropped to nearly zero. The result was a much more stable, performant, and trustworthy platform — enabling the product to grow further.

A/B testing allows product teams to optimize experiences based on data rather than assumptions. To enable this, an internal A/B testing platform was developed.

The platform provides:

• Remote configuration, allowing teams to control feature behavior without deploying new code.
• Traffic splitting, enabling controlled experimentation across user segments.
• Data collection, where all traffic assignments are logged for downstream analytics and experiment evaluation.

This platform empowers product managers to run experiments at scale, make data-driven decisions, and iterate quickly based on measurable impact.