Introduction

Prometheus is a popular monitoring system, especially in the world of Kubernetes, with a pretty mature developer ecosystem. It has API support in a lot of programming languages (https://prometheus.io/docs/instrumenting/clientlibs) that enables applications to instrument code in Prometheus language. A common architecture is for applications to expose a “scraping” HTTP endpoint to Prometheus, which can then scrape at a configured time interval all the metrics that the application exposes, and put these metrics in it’s time-series database. AWS recently announced support for scraping these Prometheus metrics into AWS CloudWatch instead (https://aws.amazon.com/blogs/containers/using-prometheus-metrics-in-amazon-cloudwatch/) via the AWS CloudWatch Agent (currently in beta). …


Fluentbit (https://fluentbit.io/) is becoming increasingly popular as a light-weight alternative to Fluentd for log collection, processing and forwarding in Kubernetes environments. Fluentbit is typically deployed as a DaemonSet(https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) in production environments, where a Fluentbit instance runs on every node in the cluster. In a typical EFK (Elastic Search + Fluentbit + Kibana) stack, this instance tails docker (usually the container runtime) container logs (written to stdout/stderr of containers) on the node (/var/log/containers/*.log). Fluentbit parses these JSON formatted logs using a pre-configured docker json parser, enriches the log message with Kubernetes metadata, using a Kubernetes filter, and outputs that to an…

Sudhindra Tirupati Nagaraj

Passionate about building products/technologies that delight customers and solving business problems. Focus on excellence of delivered solutions.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store