%% date:: [[2024-07-10]], [[2024-09-19]] parent:: %% # [[Architecture of Loki]] ![[Architecture of Loki.svg]] The illustration below shows the architecture of [[Grafana Loki|Loki]] for a single instance on the left, and in a [[Multi-tenant]] setup on the right. ![[loki-scaling.png]] [^goh] Loki employs a [[Microservices|microservices-based]] architecture: it consists of smaller components that can be individually deployed, configured, and tuned. Loki can also be employed in [[How to deploy Grafana Loki|monolithic and simple scalable modes]], but the components we'll go over here assume *microservices mode*, where every component can be independently scaled. <iframe width="560" height="315" src="https://www.youtube.com/embed/1uk8LtQqsZQ?start=222" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> *How to get started with Loki | Zero to Hero: Loki.*[^whatisloki] The microservices that make up Loki fall into three main buckets: - the read path - the write path - administrative (`backend`) ## Components of Loki <iframe width="560" height="315" src="https://www.youtube.com/embed/_hv4i84Z68s" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> ### Write Path The write path refers to the group of microservices that work together to write data to Loki. It's recommended to have 3 replicas of these components to ensure 100% consistent object storage and availability. - [[Loki distributor|Distributor]]: handles and validates incoming data pushed to Loki, then distributes data to ingesters - [[Loki ingester|Ingester]]: processes data and then sends it to long-term object storage ### Read path These microservices are responsible for ensuring that data can be retrieved *from* Loki as [[LogQL]] queries. - [[Loki query frontend|Query frontend]] (optional): [[Sharding|shards]] queries and stitches them back together, maintains a queue of work - [[Loki query scheduler|Query scheduler]] (optional): assigns jobs to queriers - [[Loki querier|Querier]]: executes [[LogQL]] queries, looks for results first in the ingesters' in-memory data and then the object storage ### Administrative - [[Loki index gateway|Index gateway]]: handles and serves metadata queries from the index and gives the query frontend and querier information about how many logs there are and where to find which logs - [[Loki compactor|Compactor]]: merges index files and handles log retention rules - [[Loki ruler|Ruler]]: allows the creation of alerting and recording rules for pre-computing query results > [!tip] Experimental components > Not included here are experimental components of Loki, including the [[Loki bloom planner|Planner]], [[Loki bloom builder|Builder]], and [[Loki bloom gateway|Bloom gateway]], which all have to do with the experimental [[Bloom filters in Loki]]. %% # Excalidraw Data ## Text Elements ## Element Links uVdx9TC3: [[Loki query frontend]] guUYa0VP: [[Loki querier]] VjgtyRME: [[Loki distributor]] T0XUqo5T: [[Loki ingester]] ROoXSpWK: [[Loki ruler]] uqn5pf5T: [[Loki compactor]] SSaMHzXt: [[Object storage]] ## Embedded Files fdde2785608b0da386c3897d7d5a132db0302812: [[loki-microservices-architecture.png]] ## Drawing ```compressed-json N4KAkARALgngDgUwgLgAQQQDwMYEMA2AlgCYBOuA7hADTgQBuCpAzoQPYB2KqATLZMzYBXUtiRoIACyhQ4zZAHoFAc0JRJQgEYA6bGwC2CgF7N6hbEcK4OCtptbErHALRY8RMpWdx8Q1TdIEfARcZgRmBShcZQUebQBGADYEmjoghH0EDihmbgBtcDBQMBKIEm4IAEUAMQAlBAB9AGYAGWdK/QApADNJSVwAIQA1AFEGgEFx1JLIWEQKwn1opH5S zG4ADnjtAE4mxI2Npp2AdgAGeJ4AVnj41cgYbmd4poAWbVfEnYv4s6uvjZnDaJe4QCgkdTcfbbHaJK5nOHwnjxI6vK6gyQIQjKaTcHhnOLowqQazKYLcM6g5hQUhsADWCAAwmx8GxSBUadZmHBcIFstNSppcNg6cpaUIOMRmaz2RJORxubyslABZBuoR8PgAMqwckSQQeVUQam0hkAdQhkjxVJp9IQOpgevQBvKoPFOI44VyaDuxIgbB52DUjx9Z 0pfrFwjgAEliN7UHkALqg7rkTKx7gcISa0GESVYCq4eJG8WSz3MeNZnN+sIIYjcHY8E4nU7A30zBhMVicbhEjuMFjsDgAOU4YihiTDk+bO1zzAAIukoHXuN0CGFQZphJKRsFMtl40nQUI4MRcMv6z6Tlcdjsrq99q8dq9XqCiBw6Zns/g32wRSu0BpIQEBtc8hHjCBEElPNlCNdVggzCRumIYgECbDZ/iBTQzjPJpgWwPDTmIE5iCuIsmh4YhsKa Akth4I1mHccQE2JMB2xmeJiWTP1sFpOAv01QoAF9VmKUpygkIQhmITAdgAFUZJojTmZiIECbAog4MkVj9dYfUnbREkSN5nyaG9Xh4PC+D9ENUGcL5tAuIELOvDZr0SE5rI7cFiEhNBEjiE5gSaPCzhC5zTlfP1MWxXE0B4ZIeE+DZ8WcjCzmvKKO1JJ1ww7E07WlNkKgAYniBByvKo0hRFSMJSlFlirlcgFR5PkVRTDVtV1VSXXrG1TQQC1fKtNA zm0PtSgKhkHSdY0WVdP13Ukct43YyAA2FYNuF+PLSjqmM43ybiO1TXB00vVAqx/P081QvT0Fwei3W3YhVu4MTShUqFiREmsEAA1AkhOJImiCszQQHbsuB9eJZz9KGh1HDhxzQEGEqCngNjnRdggvVd1xAv0t3q3cMmVQ8TtKE8z3xq8bzvV5zjw15sb9d9PwkPI8haelCFQABHYDSBgVBU04ZdJUTRMjVZf9LrXfAwmE0TbsuiBlCEABVABNXAzi GAAFZT4FU9TNO0o0HqScbLj+A4mxCp8gVBWznDwhJXjh84n1hJt4kmyAfL81BEneQErhMoFLNeDKTgxLEcRVeK4mMgKw/vK4tiSf5QRy5i9oEW0GSK2V0DKirK+q4VRVLBqZQ5FrFXauCutm3qFv6mti6Gy0KQmga7Xbio+pLYQPS9HbQU2oNYB2sNQQO2NKZTNMEEQq7v1zfMHogXAlJe+r3rQT7ZlNn6Zj+/KAcuhL4gfL4vNKRGezQJ9Ia7JG x2Yycb0BSy2YdkIAuJcgNFYbmJq9Mm+4cgCRuh2Gm55AbxGvLeTOnkjhNDfHmTm6Bua8zpPzIWTBCBMGlrLP8DIFaExVoUT6ZR1ZDAAFbKFgLUAAsiME28wJDm3zlbPEYNtCHEBEzB8GEs7Nldk8ByPB/YbD2CDFBZxnygmDqNUOVxHLwm+E2HYGEvb6ITrFZOvAtEnE+McFBcIFEHBQXnLSuVB4l0amXCAFdKo6Q7DVWur1S6Ny5G1ZUrdNTD31 J3BiPdhoh3GoHY0PcwnOgiYfCeFYp5+hnttUMhcNbikOivP0Z0LpwO3vdQsrwx5H0nifVi0Bz5oCaL9KkN9uBhyxiDeEOSX4w1DlghGn9ODI1RrwMGBImaqNuiAvGYDCabigXuCmJS/SILpkDVBjNzIIifpADmFR8F81QI4E0hBNBCCgGychv55YEyVggWhJR6ESXQHJM4AANLWAs2BXDktws2CANL8NBA9SyJxhHnGbGie8YYXhZVKG7Y4jlLJg x4M+K4GNDhqL7v5d4KKHz23vIooyxik54jiNnVmciWxwgyiCP0/CxrOKZK40qniqqbhrnVSU/jmqBKVPyTqoSeoj2Sd3Qa0SNGxMZYk+ahoUkrWqUDaegYslAwXhGPJy9jqr3OuvS611SkFgkLgK4lSywKtPnUnhqBGmX2acglRD4WwKO2Z2Qcr9Q4uu6UM5iWctgYTwutMoUyECrPAUTbx8zyYHiWQg08SDLooIZveRIMKETYI/HsnmByYLhGXK QS57NKEzNufcooasKi1AAPJsFeVqOAZoADSvyKh8McV4tYUJ8TCLaUzDKkdryBvhaCvYrNnxoNvDsexfp1GtPeJHP4rkUGTpeIGmKJL4owkUXeMGZk3KTrifS1AOTppMobhIDxVd2W1Trty9A8pm7BIFd1R0HdZWirtOK/ucST3StHnK4+iqMnKrntkxeGqjpoCPIUteG99W3R3oWRIpq3rmtqd9BpTT/rIKuFcZmf8ukDJ6Z5D+bqRzfzxNnYE7 kcagOobcuZpMFnRrQHB2NtNkHrOTU2L4cTdlcyzYQ1ApBsxkJllcqhNzlYlCvmWoB6shACw4FcOA3RvnNt4f8i25IgV4hOE0QyFjGw4evGi2EsKHiCN2PsYKIMgSuRw5ikarStGfHhEFL4yUETmakInOKvANiGQCucIzBw/gB0AaUQ9x6e63vcay9tkAfGcvrk1O9Tcgn8sKW3IV4S335SiVio9A930zRy0kvLpRloAcDZkkDqqclLwgwmKmaoYN 6q3vBspRqTjIYAxa9D1rMPX0BiFJKZwXxYxI9DXsEXIBevIw0sGjY5FTqAcG0NszIGMajbAljHW2Pxp2pxz4T4Qq0o7HxvBAn+Z6H0IGc5+axOFuuWgMNpbHnqy1FqXA7CAASRhXkdT9ANtSmnAW6R2pOiaFwWxbICviYjNlLOIl7WGfYAVbjx2nYVgKhkR1os8jRF8lxiV+ZCgkGiembFAjhklc7kW20UkZbFi9bLiYcpvcynlrU+VA9Otll9wq KtFzFYVyVJX7RlZlYtDsVWFU1eA7ZXaYGoyasgy1iARTdUxvEgho1GxeuoZmGfK1NrpN2oTbopoLxfVTaHL2QN82UY/yxviPYuGaPTLoxAiN22YGVn29TONqzE1oM+MZY4OTLsQG5pWzQTDNOoGpGyZYBaLtFu93c83dDy0SHGKQOk+gKBa3nOwgA4gLV585DY8DkpIV485KjYErep9ArbLY6Z9FD+ES64dyIyi62yllDI4dR2FIyyJVulBnf5VO +OmyWXG17F1a6yf6ZeBlfYWcaeNk+A4y2DKJcs/i9Xa9fiudpd5S3J9v6RX5dF05saxX79Dyl3+pa495VpJ9EqradXlfqqq5NZQanRtY66QB3SGqPSHCG7f6oD9b1KDa2pYaW4EjW67qzaurTZvzwz9iEbercDuRbA2KYHAK4whrFo+6CiRr+7gEQArIcZJrh56YWLpq4KsalBywSavY0LZ4PK57oDzjRiVrEANpGCMjOBGBGDYBmgcCEBahDCaA NA7ANrsKt6g4AqM6d68B6YGZPjXC4Y4YophzSLxT6Z7AHDGS2ZuRZwObY6P6aIfDUrubPihYmHRS+amJYyBZNgqLXChY3BZz75OJH4X5xaVxs7eIc7n5nqX487X5ZaCqC65Yy5TQFYOHi4v6lbJHlapGQBy5wEK5/5K5qodiNYFKgE6qwaB4QF67QFl6wHxgIGm5DZTQtKLZjYTaYHdK9j05zb4ELaoCTrcYXCxye4UGZ4MY7hMa7abzVgHYh7HZ GQEgohsF0FcGUFZ5gAyYfYVAcByR0iKbODjb4DdDsJmgbACydANrYCYDuwHzA6IEaFaYJYQAPQUTjTAgWS9rXhgw3CmF2RCIjoKK+wMyTpY7eQ45zo0SGFbAti3DW6k5eGbpLb9q7rwkHqM6H5ZGnqpbhGeKn6+L1Sxb3oZZ86lDwTPpzTv44mfpP7foJJv536Vaf7Va/6zwlENbgYVEUlgF7bzG65dbQFDCNEfRoaIFm7bEW47TGZhT4Z27uqI5 4GkYEHxQZQXB3i4YupkG0aSbhrUF+6LL8nwJB7sYJpLGTjIiYGXYcE7IZ56nvYCEQCMhwAtC/aMg4j4BNB0hajdDdBGCaACyUQUCYCdDqHt7aYQ5mGgpuRxyQqxwXBvAAnuw7CIohSeSoroqYEz6hw4qmT4qnYnBEoeEmKkrCK3AUp6LOF9F7xYlHrM5hGs6vFJac6xHQDpa84hJUmvp5HxIP4xLP5pGDS37C5qSsny7skqoAFlHclarQZVHtYCm 1FCl7w7Cik1LG6WrMSSkyYCDtGqqmROqNgKk9IBQnmqlAzHCAh6aiLjEbb0ZbbTE7YB5Ln0HB6MFh5hwpqsxrHGniabGOlyYVBGANDxCEB0hGD4D6BXDMAjD0DKA4aEBDAETzjVDhlg5aFRkjIxmHDjYWJ4Q4ZuQQlwoyKpmUooiKKY4OqOYhxwjaIqIEhOr3hwyYGr5eHmKWLgk2KwhwnBEFwNltlNmEnJYkkdkJH85JHUnMki4fpi6DkyXZFSW jkFFrSTn/6lH7Sznq7arFJ/mdZQF7xTCHxmpwHNHbmtF7mAxtJuQBwaX9GkazrnmDEoKfCMyszeY6le56lTHEDQJGlzEmmQAMHmlMFfn3zWk4LrH2k8Elp8GybiTqzdA8DYDxCMhDAcD4BQDdAgxagQVwAnCvJNDdBcCggg4RmvHWwGR2z/ApR6ZRyYHwoBY0Rwh6aXA97nBxI5lhzCJ/D1UxxxxIlllpwJRohojZxwg1lRYCV4lCVXpElcphGkm dk35Mmjknp0lFYMnDmrW9kqXpIdi1ackq4nhq7NY6Xa56VAJ1F7wDDrnwHiktHIHDa3zfn7DHn9IOU4FOXO6Q63hPgVmkHrabE+V+XMYBXHjvkhWfmfDhW/ng3PbcFiy8HbHgAnR7xwBwA6hIJilfSYiZAVBEAkqrAMCkIUADDRHEmNl+nU3dACgQDYAiDtTRjLj6A6iDTH4RHtr02M3KjM0ZDk1n6U1tlLXiWQAM2kBM0s3VAC5KWpHc0S280s1 s2yUZGDny2S0ZDK2KU9ldylDi0a36C1DjmFHE362K0ZCVqK7zx7Tq3m36DVASzVDnQai2Swq23ZB8320SxaiEBGDMT4im080e0s1yRYBQDjCE09IQDBDdDknu1QCe1Y2kDh0S1sAUCYi4CLkmnx2e0jCSj560jp0hDqy8iF101m3B0ZAF1p1yRPF1zl1B0J1S1rxG1OgcHGi8Qsj4CvJ4jJDjIeQopnYvC7rE2MS0iag6x4gIiIpLo3j/A3iImFA QBGBsAGA43PwEDAR4gfBorNWlo50s1G1VJwFjn1R01igkA+1+1T3E0X3EA6gID8RoCTS5IkDsJsCoR524CaDBCTFL1323r0IDAsjqykDKBCgAAUyI8cvA9isDMDsSAAlEaPUJrPgLyC2uA7gFAzRNQCMpSPg3g0gxAPvRXVAFrQgJbVAEOC+dnVrvUPmKQDBOvZAFkN/b/dwEBPqWLUQE/agFw6CBwDqpw8Jtw/6Gcu+MxAI36EsAXkwMOMI4BKI 6CLIwyKQF/T/YDFw/vXYAnhpMwFqEI3AO/Z/UI5o3/ZFhpIQIwHJKvfgCw1uSPOkFYz2KCAzcnvoLXVaradHdFUjQ+aAQYFqM49Q+6mGm+KEOHVYzY3Y3AsJOAFfJrhqOEB9EJCAEJEAA=== ``` %%