# [[Enlightning with Whitney Lee - Logging with Grafana Loki]]
[On Whitney's personal channel](https://www.youtube.com/live/zx7sRNpL9Ac)
![[Enlightning with Whitney Lee - Logging with Grafana Loki]]
Related:: ""
## Timestamps
00:00:00 Start of show
00:02:00 Introductions
00:04:33 Why did we build Loki?
00:13:00 What is Loki?
00:17:11 Schema at query
00:21:30 Row-oriented vs. column-oriented databases
00:29:24 What you need to know about ingesting data in Loki
00:35:45 Loki at query time
00:43:56 Monolithic deployment mode for Loki
00:55:30 Components of Loki
01:16:20 Simple scalable and microservices deployment modes for Loki
01:22:01 Storage options in Loki
01:31:39 Visualizing logs in Loki
01:46:26 Summary - Loki in 10 minutes
## Talking points
- What are [[Logs]]?
- Chronological records of events and other output produced by a component or service
- [[Logs#Issues with collecting logs]]
- Databases
- Row-oriented
- Data stored in rows with all the columns together
- Best for operations where you're reading or writing complete rows rather than just one of those things. (logs)
- Column-oriented
- Data stored in columns
- Best for reading down the column over a large amount of rows (metrics)-- anything where you're getting counts, sums, averages, etc.
- Background: Why was [[Grafana Loki|Loki]] created?
- At the time (in 2019), it was difficult to get logs out of Kubernetes because the pods are ephemeral.
- Object storage instead of block storage (saving to disk)-- more scalable, decentralized
- Simplicity in ingestion, "schema at query"
- Microservices-based, cloud-native
- Cost-effective and performant
- Databases usually have a large index, requiring "tall" (scaled up) machines
- Loki doesn't do full-text indexation; instead it indexes the timestamp and labels (key-value pairs) and doesn't index the content log line
- What is Loki?
- Loki is a log aggregation system that is built to be horizontally scalable, highly available, and multi-tenant.
- It's a "system" because it's not a single component.
- Essentially a database, but there's more to it than that.
- What do you need to visualise and query logs?
- Your service or component that you want to get logs from
- [[Ingesting logs into Loki|Log ingester]]
- [[How to deploy Grafana Loki|Deployment of Loki]]
- A way to visualise logs-- like [[Grafana]]
- Storage
- Ingesting logs into Loki
- Promtail
- OpenTelemetry collector
- Alloy
- fluentd, Fluent Bit, Logstash
- [[How to deploy Grafana Loki|Deploying Loki]]
- Different modes
- Monolithic
- Simple scalable
- Microservices
- Architecture of Loki?
- Installation
- Helm
- Docker/Docker Compose
- Grafana Tanka
- Binary
- Build from source
- Querying Loki
- What's an index?
- Label is a type of index
- Log streams
- separate files on disk
- unique combination of key-value pairs
- like "series" for Prometheus
- Chunks: multiple log streams
- How does Loki find data?
- Sorting algorithms: bubble sort, insertion sort, merge sort, bucket sort
- What Loki uses
- minimal indexing
- chunking
- sharding
- replication/parallelization
- compression and storage optimisation
- Loki's place in the ecosystem
- Logs vs traces
- Logs vs metrics
- > I'm also curious about if/when queueing systems get added
- For ingesting: hash ring
- For querying: parallelization and sharding
- You Choose - show with Viktor
%%
# Excalidraw Data
## Text Elements
## Drawing
```json
{
"type": "excalidraw",
"version": 2,
"source": "https://github.com/zsviczian/obsidian-excalidraw-plugin/releases/tag/2.3.0",
"elements": [],
"appState": {
"theme": "dark",
"viewBackgroundColor": "#ffffff",
"currentItemStrokeColor": "#1e1e1e",
"currentItemBackgroundColor": "transparent",
"currentItemFillStyle": "solid",
"currentItemStrokeWidth": 2,
"currentItemStrokeStyle": "solid",
"currentItemRoughness": 1,
"currentItemOpacity": 100,
"currentItemFontFamily": 5,
"currentItemFontSize": 20,
"currentItemTextAlign": "left",
"currentItemStartArrowhead": null,
"currentItemEndArrowhead": "arrow",
"scrollX": 532.75390625,
"scrollY": 405.88214111328125,
"zoom": {
"value": 1
},
"currentItemRoundness": "round",
"gridSize": null,
"gridColor": {
"Bold": "#C9C9C9",
"Regular": "#EDEDED"
},
"currentStrokeOptions": null,
"previousGridSize": null,
"frameRendering": {
"enabled": true,
"clip": true,
"name": true,
"outline": true
},
"objectsSnapModeEnabled": false
},
"files": {}
}
```
%%