# [[Deploying Loki on AWS]]
![[Deploying Loki on AWS.svg]]
[docs](https://grafana.com/docs/loki/latest/setup/install/helm/deployment-guides/aws/)
[[Grafana Loki|Loki]] can be deployed on a [[Kubernetes]] cluster on [[AWS]] using:
- [[AWS Simple Storage Service (S3)]] as a storage for Loki
- [[AWS Elastic Kubernetes Service|EKS]] running on [[Amazon EC2]] nodes ([[AWS Fargate]] is an alternative for a more [[Serverless computing|serverless]] experience)
- [[Helm for Kubernetes|Helm]] charts to handle the configuration
[^1]
## Create cluster
First, spin up the Kubernetes cluster within EKS and the worker nodes that will hold them by using this clsuter config file:
```yaml
# A simple example of ClusterConfig object:
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: <INSERT-CLUSTER-NAME>
region: <INSERT-REGION-FOR-CLUSTER>
version: "1.31"
iam:
withOIDC: true
addons:
- name: aws-ebs-csi-driver
managedNodeGroups:
- name: loki-workers
instanceType: m7i.2xlarge
desiredCapacity: 3
minSize: 2
maxSize: 3
amiFamily: AmazonLinux2023
iam:
withAddonPolicies:
ebs: true
volumeSize: 80
volumeType: gp3
ebsOptimized: true
```
The config file above does the following:
- Starts 3 instances of `m7i.2xlarge` nodes (8 cores, 32 GB RAM, 10 Gb/s EBS bandwidth)
- Installs a plugin, [[Amazon EBS CBI driver]] lets Kubernetes provision and use [[Amazon EBS|EBS]] volumes as persistent volumes for Loki.
> [!tip]- Other plugins required
> [[Core DNS]] (ensures services and pods can talk to each other via [[DNS]]) and [[Kube Proxy]] (enables communication between nodes and pods) are also required, but should be provided by default.
Second, create the cluster by running this command using [[EKS CTL]]:
```bash
eksctl create cluster -f cluster.yml
```
## Create S3 buckets for Loki storage
We need two S3 buckets: one for storing [[Loki Chunks|chunks]] and another for storing [[Loki ruler|rules]].
Run the following command using the [[AWS CLI]]:
```bash
aws s3api create-bucket --bucket <YOUR CHUNK BUCKET NAME e.g. `loki-aws-dev-chunks`> --region <S3 region your account is on, e.g. `eu-west-2`> --create-bucket-configuration LocationConstraint=<S3 region your account is on, e.g. `eu-west-2`> \
aws s3api create-bucket --bucket <YOUR RULER BUCKET NAME e.g. `loki-aws-dev-ruler`> --region <S3 REGION your account is on, e.g. `eu-west-2`> --create-bucket-configuration LocationConstraint=<S3 REGION your account is on, e.g. `eu-west-2`>
```
Make sure to replace `region` and `bucket` with the appropriate values.
## Set permissions for Loki to access S3
Using [[IAM]] roles to connect Loki to S3 is more secure than the alternative, which is to use access keys and secret keys stored directly in Loki config.
### Create an IAM policy JSON file
IAM policies are set using a [[JSON]] file like this:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "LokiStorage",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::< CHUNK BUCKET NAME >",
"arn:aws:s3:::< CHUNK BUCKET NAME >/*",
"arn:aws:s3:::< RULER BUCKET NAME >",
"arn:aws:s3:::< RULER BUCKET NAME >/*"
]
}
]
}
```
Make sure you update the bucket names in the `Resource` section.
### Create the IAM policy
Run this using AWS CLI:
```bash
aws iam create-policy --policy-name LokiS3AccessPolicy --policy-document file://loki-s3-policy.json
```
### Create a trust policy JSON file
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::< ACCOUNT ID >:oidc-provider/oidc.eks.<INSERT REGION>.amazonaws.com/id/< OIDC ID >"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.<INSERT REGION>.amazonaws.com/id/< OIDC ID >:sub": "system:serviceaccount:loki:loki",
"oidc.eks.<INSERT REGION>.amazonaws.com/id/< OIDC ID >:aud": "sts.amazonaws.com"
}
}
}
]
}
```
Make sure you use your AWS account ID, region, and OIDC ID (which you can find in the EKS cluster configuration) with the appropriate values.
### Create the IAM role
```bash
aws iam create-role --role-name LokiServiceAccountRole --assume-role-policy-document file://trust-policy.json
```
### Attach the IAM policy to the IAM role
```
aws iam attach-role-policy --role-name LokiServiceAccountRole --policy-arn arn:aws:iam::<Account ID>:policy/LokiS3AccessPolicy
```
## Add authentication for Loki (optional but recommended)
Deploying Loki on AWS will mean it is accessible to anyone on the internet, so it's recommended to at least use basic authentication for Loki.
### Create namespace for Loki
Create a new namespace for Loki to use:
```bash
kubectl create namespace loki
```
### Install `htpasswd`
```bash
sudo apt-get update && sudo apt-get install apache2-utils
```
### Create `auth` file
Create an `auth` file by running this command:
```bash
htpasswd -c .htpasswd <username>
```
When prompted, enter a password.
### Create a Kubernetes secret
```bash
kubectl create secret generic loki-basic-auth --from-file=.htpasswd -n loki
```
### Create a `canary-basic-auth` secret
```bash
kubectl create secret generic canary-basic-auth \
--from-literal=username=<USERNAME> \
--from-literal=password=<PASSWORD> \
-n loki
```
## Deploy Loki via Helm
### Add Grafana chart repository to Helm
Make sure you've installed [[Helm for Kubernetes|Helm]]. Then run the following:
```bash
helm repo add grafana https://grafana.github.io/helm-charts
```
Update the repo:
```bash
helm repo add grafana https://grafana.github.io/helm-charts
```
### Create the Helm chart config file
```yaml
loki:
schemaConfig:
configs:
- from: "2024-04-01"
store: tsdb
object_store: s3
schema: v13
index:
prefix: loki_index_
period: 24h
storage_config:
aws:
region: <S3 BUCKET REGION> # for example, eu-west-2
bucketnames: <CHUNK BUCKET NAME> # Your actual S3 bucket name, for example, loki-aws-dev-chunks
s3forcepathstyle: false
ingester:
chunk_encoding: snappy
pattern_ingester:
enabled: true
limits_config:
allow_structured_metadata: true
volume_enabled: true
retention_period: 672h # 28 days retention
compactor:
retention_enabled: true
delete_request_store: s3
ruler:
enable_api: true
storage:
type: s3
s3:
region: <S3 BUCKET REGION> # for example, eu-west-2
bucketnames: <RULER BUCKET NAME> # Your actual S3 bucket name, for example, loki-aws-dev-ruler
s3forcepathstyle: false
alertmanager_url: http://prom:9093 # The URL of the Alertmanager to send alerts (Prometheus, Mimir, etc.)
querier:
max_concurrent: 4
storage:
type: s3
bucketNames:
chunks: "<CHUNK BUCKET NAME>" # Your actual S3 bucket name (loki-aws-dev-chunks)
ruler: "<RULER BUCKET NAME>" # Your actual S3 bucket name (loki-aws-dev-ruler)
# admin: "<Insert s3 bucket name>" # Your actual S3 bucket name (loki-aws-dev-admin) - GEL customers only
s3:
region: <S3 BUCKET REGION> # eu-west-2
#insecure: false
# s3forcepathstyle: false
serviceAccount:
create: true
annotations:
"eks.amazonaws.com/role-arn": "arn:aws:iam::<Account ID>:role/LokiServiceAccountRole" # The service role you created
deploymentMode: Distributed
ingester:
replicas: 3
zoneAwareReplication:
enabled: false
querier:
replicas: 3
maxUnavailable: 2
queryFrontend:
replicas: 2
maxUnavailable: 1
queryScheduler:
replicas: 2
distributor:
replicas: 3
maxUnavailable: 2
compactor:
replicas: 1
indexGateway:
replicas: 2
maxUnavailable: 1
ruler:
replicas: 1
maxUnavailable: 1
# This exposes the Loki gateway so it can be written to and queried externaly
gateway:
service:
type: LoadBalancer
basicAuth:
enabled: true
existingSecret: loki-basic-auth
# Since we are using basic auth, we need to pass the username and password to the canary
lokiCanary:
extraArgs:
- -pass=$(LOKI_PASS)
- -user=$(LOKI_USER)
extraEnv:
- name: LOKI_PASS
valueFrom:
secretKeyRef:
name: canary-basic-auth
key: password
- name: LOKI_USER
valueFrom:
secretKeyRef:
name: canary-basic-auth
key: username
# Enable minio for storage
minio:
enabled: false
backend:
replicas: 0
read:
replicas: 0
write:
replicas: 0
singleBinary:
replicas: 0
```
### Deploy Loki
```bash
helm install --values values.yaml loki grafana/loki -n loki --create-namespace
```
%%
# Excalidraw Data
## Text Elements
## Drawing
```json
{
"type": "excalidraw",
"version": 2,
"source": "https://github.com/zsviczian/obsidian-excalidraw-plugin/releases/tag/2.1.4",
"elements": [
{
"id": "4y8R7iOA",
"type": "text",
"x": 118.49495565891266,
"y": -333.44393157958984,
"width": 3.8599853515625,
"height": 24,
"angle": 0,
"strokeColor": "#1e1e1e",
"backgroundColor": "transparent",
"fillStyle": "solid",
"strokeWidth": 2,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"frameId": null,
"roundness": null,
"seed": 967149026,
"version": 2,
"versionNonce": 939059582,
"isDeleted": true,
"boundElements": null,
"updated": 1713723615080,
"link": null,
"locked": false,
"text": "",
"rawText": "",
"fontSize": 20,
"fontFamily": 4,
"textAlign": "left",
"verticalAlign": "top",
"containerId": null,
"originalText": "",
"lineHeight": 1.2
}
],
"appState": {
"theme": "dark",
"viewBackgroundColor": "#ffffff",
"currentItemStrokeColor": "#1e1e1e",
"currentItemBackgroundColor": "transparent",
"currentItemFillStyle": "solid",
"currentItemStrokeWidth": 2,
"currentItemStrokeStyle": "solid",
"currentItemRoughness": 1,
"currentItemOpacity": 100,
"currentItemFontFamily": 4,
"currentItemFontSize": 20,
"currentItemTextAlign": "left",
"currentItemStartArrowhead": null,
"currentItemEndArrowhead": "arrow",
"scrollX": 583.2388916015625,
"scrollY": 573.6323852539062,
"zoom": {
"value": 1
},
"currentItemRoundness": "round",
"gridSize": null,
"gridColor": {
"Bold": "#C9C9C9FF",
"Regular": "#EDEDEDFF"
},
"currentStrokeOptions": null,
"previousGridSize": null,
"frameRendering": {
"enabled": true,
"clip": true,
"name": true,
"outline": true
}
},
"files": {}
}
```
%%
[^1]: Clifford, J. (2025). *Deploying the Loki Helm on AWS | Grafana*. https://www.youtube.com/watch?v=5lXmWmofqwM