# [[Context Window]] ![[Context Window.svg]] A [[Context]] window is a limit of tokens that an [[Large Language Model|LLM]] can contain within a prompt. While context windows are growing as new models are developed, context windows are finite. Being within the context window of an LLM isn't a guarantee that the LLM will remember all that information. In fact, there is a known [[Lost in the middle]] problem where somewhere in the middle of the context window limit, LLMs often begin to hallucinate. This occurs because context is a resource with diminishing marginal returns. [^1] In general, even if the context window is not reaching half utilization, [[Context rot]] makes it so that the more information the LLM has to recall, the less efficient it gets at doing so. ## Context window as an attention budget One way of thinking of the context window is as a finite budget from which [[Attention]] is spent. As a developer, you choose what to spend that budget on, depending on your requirements. %% # Excalidraw Data ## Text Elements ## Drawing ```compressed-json N4KAkARALgngDgUwgLgAQQQDwMYEMA2AlgCYBOuA7hADTgQBuCpAzoQPYB2KqATLZMzYBXUtiRoIACyhQ4zZAHoFAc0JRJQgEYA6bGwC2CgF7N6hbEcK4OCtptbErHALRY8RMpWdx8Q1TdIEfARcZgRmBShcZQUebR44gAYaOiCEfQQOKGZuAG0AXX4IXDg4AGUoqHFUUDBIdXTqiCJlaRS6hkIECgAhXGwAa2VSYQ5iAGE2fDZSbggAYgAzZZX2 yGwRQKyASSr9CpGBhEnp2Yl5gEYEK6u1iA3SLahd9L7B4dGJqZm5qHIOZhwXBPO4PJ4vfQAMUI+HwFRgwTmgg8oM2mWeewObCOAHUSOpuHxwOs0TtMX9sQh4YiJMiSKjHuiIQAlYStDjhHJoC78EmMsnpADyQOwahg3AuiUSvPupIx6UhnCgkNw+hh4rQAFYZWCmXtFVkyoQjNUeNLibL+fL9AAVLBQACCLS4EmCiygDPB5OBjsebAokhCxG4HCE sJ1cohAFExg6/QGg3NgSMqBGrRC4ymbfAmiMhGNPXqFeQMqyaahQ+GLcxsCNYQANbgAZgALHEeC3EgBOLsAdibFxbLc1A5lNbr+AAmtwAGxdxLaAAcTZnbc1vcXFx42otRjYBm4tQ69AIQmqF2JAF80170qz88QOcwueg8wWZcMSEaTYTzR1P8QFQIHA3A7v+pAkAAsmwxAIDGuCaMEwZoIsBBhB+EFfKch7EpAPRTMhr7KJouAABQ8BcvbULwlH URRVGoAumoAJRrJAzIIMoYbAnMpDEWRPBNtKvBCXRomMdoLEQFeN7oliRzClA7AAiGYb4DKiwlggHFjEwhAcMoOF1JAmQIUh3B/GevLrEQIFoJZCDWRAHCqtUDlOcIUBEByFmkFZuEQPowJHKQAByrm+f5xmBcFTDwYhCCEQ5MkWnYABWCDYNkZQuXA0GwfF5koWhjkWv0SmMDa+74EZHSwIgSJpFlylsbKzBQAY2YNWglbqQF0yDIl3Cofg6EBZ pBhlE1SmcMNJVOfgoSOs1lXVapsIpR0jjMGZxxKvakGZEIc2jaVHSaPpjgGQACoEixMJkYhzKZCXBjKhDMD0l36cohVDfZflnZAH2QSQcBsPpUA5SUcA/X9SWAzKmhsJgU3BM1nD5U0ehZLg+nSWA150IsMLhIel4gJeQA== ``` %% [^1]: Anthropic (2005). [[Effective Context Engineering for AI Agents]]