# [[Bloom filters in Loki]]
![[Bloom filters in Loki.svg]]
[[Grafana Loki]] uses [[Bloom filter|Bloom filters]] to quickly [[Querying Loki|retrieve results from queries]]. According to [[Owen Diehl]] [^grafanacon], you only need to query 10% of the data to find information in the log line. Theoretically, this leads to a 10x improvement on query speeds.
As of [[Loki 3.0]], bloom filters are included by default, although they're experimental.
<iframe width="560" height="315" src="https://www.youtube.com/embed/L_GHahMOWEY?start=2344" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
Since the announcement at GrafanaCON, however, the team has found that it's difficult to create and maintain line blooms at scale, at least in a performant way. So they've announced that they're pivoting towards creating blooms only on [[Structured metadata]] instead. Here's a Community Call where they discuss that pivot:
<iframe width="560" height="315" src="https://www.youtube.com/embed/keXrUf8jw1A" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
[[Loki Community Call 2024-09-05 - Bloom filter changes for Grafana Loki]]
This change drops bloom filters on full log lines, but keeps it for searching structured metadata such as those used by [[OpenTelemetry|OTel]].
%%
- [Initial Design doc for bloom filters in Loki](https://docs.google.com/document/d/1qtA-D4JmJJRDeAnBosD-7j1rW2NK51Az0JH0nqoirOE/edit?pli=1#heading=h.alq4ht7t2q43)
- Discussing problems with creating bloom filters for the log line: [Revisiting Bloom filters](https://docs.google.com/document/d/1Q8DuQV3ctBT7oC-dI9kQYc5F18xMSeumvZI4jhpzqa8/edit#heading=h.9gte82uqdzoc)
- Pivoting to structured metadata rather than log line: [Structured metadata blooms](https://docs.google.com/document/d/18DTYlWRAqLhkWhJblaSLdRtlADhBNk3kiPENCpcV0rI/edit#heading=h.8hutot9wgjf)
%%
%%
# Excalidraw Data
## Text Elements
## Drawing
```json
{
"type": "excalidraw",
"version": 2,
"source": "https://github.com/zsviczian/obsidian-excalidraw-plugin/releases/tag/2.1.4",
"elements": [
{
"id": "4y8R7iOA",
"type": "text",
"x": 118.49495565891266,
"y": -333.44393157958984,
"width": 3.8599853515625,
"height": 24,
"angle": 0,
"strokeColor": "#1e1e1e",
"backgroundColor": "transparent",
"fillStyle": "solid",
"strokeWidth": 2,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"frameId": null,
"roundness": null,
"seed": 967149026,
"version": 2,
"versionNonce": 939059582,
"isDeleted": true,
"boundElements": null,
"updated": 1713723615080,
"link": null,
"locked": false,
"text": "",
"rawText": "",
"fontSize": 20,
"fontFamily": 4,
"textAlign": "left",
"verticalAlign": "top",
"containerId": null,
"originalText": "",
"lineHeight": 1.2
}
],
"appState": {
"theme": "dark",
"viewBackgroundColor": "#ffffff",
"currentItemStrokeColor": "#1e1e1e",
"currentItemBackgroundColor": "transparent",
"currentItemFillStyle": "solid",
"currentItemStrokeWidth": 2,
"currentItemStrokeStyle": "solid",
"currentItemRoughness": 1,
"currentItemOpacity": 100,
"currentItemFontFamily": 4,
"currentItemFontSize": 20,
"currentItemTextAlign": "left",
"currentItemStartArrowhead": null,
"currentItemEndArrowhead": "arrow",
"scrollX": 583.2388916015625,
"scrollY": 573.6323852539062,
"zoom": {
"value": 1
},
"currentItemRoundness": "round",
"gridSize": null,
"gridColor": {
"Bold": "#C9C9C9FF",
"Regular": "#EDEDEDFF"
},
"currentStrokeOptions": null,
"previousGridSize": null,
"frameRendering": {
"enabled": true,
"clip": true,
"name": true,
"outline": true
}
},
"files": {}
}
```
%%
[^grafanacon]: Diehl, O. (2024). *Keynote - Loki turns 3.0* at GrafanaCON 2024 in Amsterdam. A recording of this presentation is on YouTube: https://www.youtube.com/watch?v=L_GHahMOWEY