Setup Elasticseach, Kibana and Fluent Bit for Your Kubernetes Cluster Logging
This walk-through guides you to setup an in-cluster Elasticsearch and Kibana suite, with cluster-level logging data gathered by Fluent Bit. You may access and search logs from every pod in cluster, as long as the workload in pod writes log to
- Kubernetes 1.11 or higher (minikube not working)
- Predefined storage class called
hdd-ssd(you may change it in
Clone this repo to get necessary yaml files.
Elasticsearch and Kibana
Elastic Cloud on Kubernetes(ECK) is now generally available. ECK makes it easy to deploy Elasticsearch and Kibana on k8s with various topology.
- Deploy ECK
- Create Namespace
- Deploy Elasticsearch and Kibana
FluentBit runs as DaemonSet on every node in cluster, gathering logs from every workload. FluentBit attach metadata like pod name and label to logs delivered to Elasticsearch.
Well-structured log(in JSON) can be searched/filtered by term in Elasticsearch.
And off you go.