Blog

Recently we faced an interesting issue in our data pipeline where we observed Kafka consumer lag suddenly dropping whenever it reached a certain threshold. This behavior was unexpected and potentially concerning, as consumer lag—the difference between the latest message produced and the message currently being processed by consumers—should typically decrease gradually as consumers process messages. The Problem Our application was relatively simple, with two instances of pods running in Kubernetes. Since we didn’t need high...

Recently we faced an issue, where there were some hidden characters inside the hostname of a configuration store of our backend services. Because of this, we were getting continously getting DNS resolution errors. One quick way to reveal such hidden characters is to run the command echo "string" | cat -v For example try copying the belwo string into dig or a whois website. madhur.​co.​in The above string contains hidden characters which can be reveled...

Quick update on my personal homelab server. I set up a dashboard using homepage to monitor and manage all my services in one place. Though it was a bit tedious to set up, it turned out well. I also setup a Debian VM on my proxmox to host some of the applications on docker. The applications I am hosting through docker are: FreshRSS - RSS Aggregator Memos - Self-hosted note-taking application for capturing and organizing...

I was playing around with Qtile when it suddenly stopped working. I tried everything to troubleshoot but wasn’t getting any clues. There was nothing in ~/.local/share/qtile/qtile.log I found this entry in systemd journal through journalctl | grep qtile Here’s what the logs showed: Dec 24 09:14:45 madhur-b550mds3h systemd-coredump[26096]: Process 26057 (qtile) of user 1000 terminated abnormally with signal 11/SEGV, processing... Dec 24 09:14:45 madhur-b550mds3h systemd-coredump[26097]: Process 26057 (qtile) of user 1000 dumped core. Dec 24...

I needed to resize a disk in my Proxmox VM. Specifically, I was working with a Debian 12 (Bookworm) VM created through Proxmox VE Helper Scripts. These VMs come with a default size of 2 GB, even when using the advanced installation option. Unfortunately, there’s no way to select the disk size during the initial installation process. Pre-Requisites First, install the required tools: apt install parted fdisk Steps to Resize the disk Resize the disk...

If you are looking to copy huge amount of data to/from AWS S3 Look no further than s5cmd Download the entire s3 bucket to local s5cmd cp s3://bucket/ . I was able to download almost 10 GB folder in couple of minutes, which is around 83MBps ~ 10 MB/s Whereas aws s3 cp --recursive s3://bucket/ . was giving me around 500 KB/s

Spring data MongoDB adds a _class attribute to every document written to MongoDB If we want to prvent adding Spring Data MongoDB from adding _class attribute, we can configure this behaviour through: @Configuration public class MongoConfig extends AbstractMongoClientConfiguration { @Override protected boolean autoIndexCreation() { return true; } @Bean public MappingMongoConverter mappingMongoConverter(MongoDatabaseFactory databaseFactory, MongoMappingContext mappingContext) { DbRefResolver dbRefResolver = new DefaultDbRefResolver(databaseFactory); MappingMongoConverter converter = new MappingMongoConverter(dbRefResolver, mappingContext); converter.setTypeMapper(new DefaultMongoTypeMapper(null)); return converter; } }

If you want to quickly spin up a local development environment of kubernets cluster, Kubernetes in docker aka Kind is the way to go Kind runs the kubernetes inside a docker container as opposed to Minikube which used to run kubernetes inside a VM Running Kind on linux requires to create a cluster configuration file cluster.yaml Here is the sample configuration which would work for most of the cases: kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: -...

The entire code of this article is present on this github repository Load testing WebSocket servers is crucial for ensuring their performance and scalability. This article will guide you through using Locust, a popular open-source load testing tool, to test your Java WebSocket server. We will use Jetty WebSocket in Java to run our websocket server. Jetty WebSocket is a Java library and part of the larger Jetty project, which is a popular open-source web...

Monitoring Redis Streams lag in Prometheus and Grafana In this post, we will look at how we can measure redis streams lag from the application perspective. We would want to measure the lag and plot this in visual format using Prometheus in Grafana Assumptions We are going to assume that we have set of streams and set of consumer groups which are consuming from the stream. We are using consumer groups because we want high...