# A Guide to LLM Observability With OpenTelemetry  URL:: https://techstrong.ai/building-with-ai/a-guide-to-llm-observability-with-opentelemetry/ Author:: Ishan Jain  ## AI-Generated Summary Large Language Models (LLMs) are crucial in today's AI applications, making their reliability and performance monitoring essential. OpenTelemetry provides tools for observing LLMs, helping developers track key metrics and improve application efficiency. Using both manual and automated methods, developers can better understand and optimize their LLM applications for better user experiences. ## Highlights > Large Language Models (LLMs) are the backbone of today’s rapidly evolving AI landscape, powering everything from intelligent chat assistants to advanced data analyzers. As these models become increasingly complex and prevalent in the applications we use daily (some [estimate](https://erp.today/llm-use-by-data/) there will be around 750 million LLM apps built by 2025), ensuring their reliability and performance becomes more important. ([View Highlight](https://read.readwise.io/read/01j7zmxgza9sz8x8ecjkseabwd)) > Here are the key signals to monitor: > Traces > 1. Request Metadata: > 1. Temperature: Measures how creative or random the output should be. > 2. top_p: Controls how selective the model is with its output text choices. > 3. Model Name or Version: Helps track performance changes with updates. > 4. Prompt Details: The inputs sent to the LLM, which can vary greatly. > 2. Response Metadata: > 1. Tokens: Directly impacts cost and measures response length. > 2. Cost: Important for budgeting and managing expenses. > 3. Response Details: Characteristics of model outputs and potential inefficiencies. > Metrics > 1. Request Volume: Total number of requests made to the LLM service to understand usage patterns. > 2. Request Duration: Time taken to process each request, including network latency and response generation time, to understand service performance. > 3. Costs and Tokens Counters: Tracking total costs and tokens over time for budgeting and cost optimization. ([View Highlight](https://read.readwise.io/read/01j7zn7zte36bstdhya4e74ngt))