# Open Source Log Monitoring: The Concise Guide to Grafana Loki
URL:: https://grafana.com/blog/2023/12/11/open-source-log-monitoring-the-concise-guide-to-grafana-loki/?pg=blog&plcmt=body-txt
Author:: Ed Welch
## AI-Generated Summary
Grafana Loki is an open-source log monitoring tool that has gained popularity over the years. It was designed to be highly scalable and Kubernetes-friendly, leveraging Prometheus labels and Grafana's UI. The tool has evolved to support better log ingestion, improved log indexing using Bloom filters, and continued use of object storage for log storage. The query language, LogQL, has also evolved to support more complex queries, but there are plans to develop a query experience that doesn't require knowledge of the query language. The flexibility and scalability of Loki have contributed to its success in the logging world.
## Highlights
> December 11, 2023 • 11 min
> * * *
> Five years ago today, [Grafana Loki](https://grafana.com/oss/loki?pg=blog&plcmt=body-txt) was introduced to the world on the [KubeconNA 2018](https://www.forbes.com/sites/oracle/2018/12/27/at-kubecon-2018-kubernetes-lifts-cloud-native-coders-to-new-heights/) stage when David Kaltschmidt, now a Senior Director of Engineering at Grafana Labs, clicked the button to make the [Loki repo public](https://github.com/grafana/loki) live in front of the sold-out crowd. At the time, Loki was a prototype: We bolted together [Grafana](https://grafana.com/oss/grafana?pg=blog&plcmt=body-txt) as a UI, Cortex internals, and Prometheus labels to find out if there was a need for a new open source tool to manage logs. ([View Highlight](https://read.readwise.io/read/01hnzc3qy336x4qdje4a64v7ky))
> The concept was fairly simple: leverage the categorization system of Prometheus labels, the single pane of glass provided by Grafana, and the cloud native distributed system design of Cortex to build a logging system that is highly scalable and very Kubernetes-friendly. ([View Highlight](https://read.readwise.io/read/01hnzc5y819t272bq279dxfwce))
> There was one additional piece here which I think really made Loki the obvious choice in Kubernetes clusters around the world: Promtail, a simple agent that leveraged the same service discovery used by Prometheus. This made it really easy to automatically find and categorize your logs with the same labels you were using to organize your metrics. ([View Highlight](https://read.readwise.io/read/01hnzc6dk6ktps1q0fjec05cpq))
> As such, Loki was built with a very simple data model: It stores everything as a string. ([View Highlight](https://read.readwise.io/read/01hnzctnb16qywjr1w89w3dd26))
> Having an entirely schema-less, structureless data model makes ingestion as easy as possible—no agreement on log format, schemas, or structure is necessary. Instead, Loki adopts the approach of parsing and adding structure to logs at query time, often called “schema at query.” ([View Highlight](https://read.readwise.io/read/01hnzctxgnza7x18gx3knmjrge))
> Using the default settings in [Grafana Agent](https://grafana.com/oss/agent/), logs have a typical creation-to-read latency of less than 2s. ([View Highlight](https://read.readwise.io/read/01hnzcvk14ky9jhqmvt22gnjqz))
> We also wanted Loki to scale easily, quickly, and without limits. This is why it’s built as a distributed system, which allows for easy insertion of more hardware to quickly handle more load. ([View Highlight](https://read.readwise.io/read/01hnzcxmgp5zfw4mpadgmbfa4y))
> One traditional operational complexity of databases is managing a large index. Often they are hard to partition horizontally leaving you with requirements for very expensive, very “tall” machines with hundreds of GB of RAM and/or extremely fast local disks. ([View Highlight](https://read.readwise.io/read/01hnzcy4ad315151kkahf61svj))
> Loki was designed to build a very small index. We only index the Prometheus-style labels that define what we call a log stream. ([View Highlight](https://read.readwise.io/read/01hnzcyj1j0cy918c12j20ejhg))