%%
date:: [[2024-07-10]]
parent::
%%
# [[How Loki stores data]]
![[How Loki stores data.svg]]
Here's a look at how [[Grafana Loki]] stores the [[Logs|Logging]] information it receives from a [[Telemetry collector|Telemetry collector]] like [[Grafana Alloy|Alloy]].
## Log structure
Loki stores received information in indexes, chunks, and streams.
An [[Database index|index]] is a way to reduce a large amount of data into a smaller set so that it's easier to find what you're looking for. Indices allow Loki to retrieve information faster. In Loki, the index includes the *labels*, which are ways to categorize data.
A stream is a separate file on disk that contains multiple log entries having the same unique combination of key-value pairs. This is analogous to [[Time-series database|time-series]] or series in [[Prometheus]].
A chunk is a container for the log entries themselves. Chunks are made up of multiple log streams.
## Log format
A [[Grafana Loki|Loki]] log consists of:
- A timestamp
- Labels/selectors (key-value pairs)
- Content of the log line
Of these, the content is unindexed.
![[loki-how-it-works.png]] [^goh]
Instead of being indexed, the log line is grouped into streams and then indexed with Prometheus-style labels. So it's more like a table of contents than an index.
[[LogQL]] is the query language used to query logs in Loki. When you query logs, the raw logs are queried by the label selector you've specified, then by timeframe, and then only the last part is brute forced. This process makes Loki more performant than other log aggregators.
![[loki-querying.png]] [^goh]
## Object Storage
Loki needs a separate object storage. Here are some options for what to use:
- [[Minio]]
- [[AWS Simple Storage Service (S3)]]
- [[Google Cloud Storage]]
- [[Azure Blob Storage]]
- on disk when running locally
%%
# Text Elements
# Drawing
```json
{
"type": "excalidraw",
"version": 2,
"source": "https://github.com/zsviczian/obsidian-excalidraw-plugin/releases/tag/2.1.4",
"elements": [
{
"id": "4y8R7iOA",
"type": "text",
"x": 118.49495565891266,
"y": -333.44393157958984,
"width": 3.8599853515625,
"height": 24,
"angle": 0,
"strokeColor": "#1e1e1e",
"backgroundColor": "transparent",
"fillStyle": "solid",
"strokeWidth": 2,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"frameId": null,
"roundness": null,
"seed": 967149026,
"version": 2,
"versionNonce": 939059582,
"isDeleted": true,
"boundElements": null,
"updated": 1713723615080,
"link": null,
"locked": false,
"text": "",
"rawText": "",
"fontSize": 20,
"fontFamily": 4,
"textAlign": "left",
"verticalAlign": "top",
"containerId": null,
"originalText": "",
"lineHeight": 1.2
}
],
"appState": {
"theme": "dark",
"viewBackgroundColor": "#ffffff",
"currentItemStrokeColor": "#1e1e1e",
"currentItemBackgroundColor": "transparent",
"currentItemFillStyle": "solid",
"currentItemStrokeWidth": 2,
"currentItemStrokeStyle": "solid",
"currentItemRoughness": 1,
"currentItemOpacity": 100,
"currentItemFontFamily": 4,
"currentItemFontSize": 20,
"currentItemTextAlign": "left",
"currentItemStartArrowhead": null,
"currentItemEndArrowhead": "arrow",
"scrollX": 583.2388916015625,
"scrollY": 573.6323852539062,
"zoom": {
"value": 1
},
"currentItemRoundness": "round",
"gridSize": null,
"gridColor": {
"Bold": "#C9C9C9FF",
"Regular": "#EDEDEDFF"
},
"currentStrokeOptions": null,
"previousGridSize": null,
"frameRendering": {
"enabled": true,
"clip": true,
"name": true,
"outline": true
}
},
"files": {}
}
```
%%
[^goh]: Bekker, W. (2023). *Getting started with Grafana Loki (Grafana Office Hours #09*. Retrieved from: https://www.youtube.com/watch?v=OLebNPLIJMI