Skip to technology filters Skip to main content
Dynatrace Hub

Extend the platform,
empower your team.

Popular searches:
Home hero bg
MilvusMilvus
Milvus

Milvus

Gain insights about vector database resource utilization and cache behavior

Technology
Free trialDocumentation
  • Product information

Overview

Vector databases, exemplified by Milvus, play a crucial role as semantic caches within contemporary Large Language Model (LLM) service frameworks.

Semantic caches are instrumental in mitigating latency for familiar and frequently accessed user prompts, concurrently optimizing the overall expenditure associated with cloud-based pre-trained model services.

Vigilant monitoring of cache efficiency and memory utilization is imperative for optimal resource allocation, while the cache's adaptability to dynamic contexts serves as a metric for its ability to accurately respond to evolving conversation dynamics. Furthermore, considerations of cache warm-up times contribute to expediting the availability of cached information. In the realm of vector databases, the performance of queries and indexing speed emerges as pivotal indicators directly influencing the system's efficacy in handling similarity searches.

Key factors such as scalability, accuracy of vector representations, and storage efficiency assume critical roles in proficiently managing expanding datasets. Additionally, the performance metrics related to updates, deletions, and query throughput further impact the overall effectiveness of these systems in delivering real-time and accurate responses in natural language processing and similarity search applications.

Striking an optimal balance across these Key Performance Indicators (KPIs) ensures that both semantic LLM caches and vector databases, like Milvus, achieve peak performance across diverse use cases.

To summarize, the overarching goal of vector databases, exemplified by Milvus, is to address performance-related challenges, enhance operational efficiency, and contribute to a more seamless and responsive experience in various natural language processing applications.

Get started

Setting up full stack observability for your GenAI applications is possible with Traceloop's OpenLLMetry, with OpenTelemetry under the hood, which can seamlessly provide comprehensive end-to-end insights into your production environments.

To set up OpenLLMetry with Dynatrace, see Dynatrace Documentation.

Details

Milvus exposes Prometheus-compatible metrics for monitoring at port 9091 under the path /metrics. A standard Prometheus setup can be used to visualize metrics on various dashboards in your Dynatrace environment.

Milvus metrics are then used to measure request latencies, import speed, time spent on vector vs object storage, memory usage, application usage, and more.

Beside other measurements, Milvus exposes following metrics that allow users to observe the health and performance of their vectorized index.

You can use Dynatrace to display, analyze and alert on many different Milvus telemetry metrics that can be categorized into the following three main areas:

• Milvus Performance Metrics • System Performance Metrics: Metrics relating to CPU/GPU usage, network traffic, and disk read speed. • Hardware Storage Metrics: Metrics relating to data size, data files, and storage capacity.

Milvus Performance Metrics

  • Insert per Second: Number of vectors that are inserted in a second. (Real-time display)
  • Queries per Minute: "Queries Per Minute" (QPM) is a performance metric employed in technical documentation to gauge a system's efficiency in processing search queries within a designated timeframe, typically a minute. This metric is derived by dividing the total number of executed queries by the duration of the observed time period. For example, if a system completes 100 search queries in 5 minutes, the QPM would be calculated as 20 QPM. QPM is instrumental in assessing the responsiveness and effectiveness of systems, particularly in real-time applications where timely query processing is critical. It serves as a key indicator for developers to optimize algorithms and resources, identifying potential bottlenecks and improving overall system performance. Interpretation of QPM is often complemented by considering other metrics like query elapsed time and resource utilization, providing a comprehensive view of a system's efficiency in handling search queries.
  • Query Time per Vector: Average time to query one vector. Divide the query elapsed time by the number of queried vectors.
  • Query Service Level: Query service level = n_queries_completed_within_threshold1 / n_queries. Generally, it is recommended to set 3 time periods - threshold1, threshold2, and threshold3, to track the query service level.
  • Uptime

Milvus System Performance Metrics

-GPU Utilization: GPU utilization ratio (%).

  • GPU Memory Usage: GPU memory (GB) currently consumed by Milvus.
  • CPU Utilization: CPU utilization ratio (%). Divide the time that the server is busy by the total elapsed time.
  • Memory Usage: Memory (GB) currently consumed by Milvus.
  • Cache Utilization: Cache utilization ratio (%).

-Network IO: Network IO read/write speed (GB/s).

  • Disk Read Speed: Disk read speed (GB/s).
  • Disk Write Speed: Disk write speed (GB/s).

Milvus Hardware storage metrics

-Data Size: Total amount (GB) of data stored in Milvus. Total File: Number of data files currently stored in Milvus.

Dynatrace
Documentation
By Dynatrace
Dynatrace support center
Copy to clipboard
Dynatrace Hub
Get data into DynatraceBuild your own app
All (776)Log Management and AnalyticsKubernetesAI and LLM ObservabilityInfrastructure ObservabilitySoftware DeliveryApplication ObservabilityApplication SecurityDigital ExperienceBusiness Analytics
Filter
Type
Built and maintained by
Deployment model
SaaS
  • SaaS
  • Managed
Partner FinderBecome a partnerDynatrace Developer

Discover recent additions to Dynatrace

Problems logo

Problems

Analyze abnormal system behavior and performance problems detected by Davis AI.

Logs logo

Logs

Explore all your logs without writing a single query.

Security Investigator logo

Security Investigator

Fast and precise forensics for security and logs on Grail data with DQL queries.

Business Flow logo

Business Flow

Track, analyze, and optimize your critical business processes.

Cost & Carbon Optimization logo

Cost & Carbon Optimization

Track, analyze, and optimize your IT carbon footprint and public cloud costs.

Davis Anomaly Detection logo

Davis Anomaly Detection

Detect anomalies in timeseries using the Davis AI

Analyze your data

Understand your data better with deep insights and clear visualizations.

Notebooks logo

Notebooks

Create powerful, data-driven documents for custom analytics and collaboration.

Dashboards logo

Dashboards

Transform complex data into clear visualizations with custom dashboards.

Automate your processes

Turn data and answers into actions, securely, and at scale.

Workflows logo

Workflows

Automate tasks in your IT landscape, remediate problems, and visualize processes

Jira logo

Jira

Create, query, comment, transition, and resolve Jira tickets within workflows.

Slack logo

Slack

Automate Slack messaging for security incidents, attacks, remediation, and more.

Secure your cloud application

See vulnerabilities and attacks in your environment.

Security Overview logo

Security Overview

Get a comprehensive overview of the security of your applications.

Code-Level Vulnerabilities logo

Code-Level Vulnerabilities

Detect vulnerabilities in your code in real time.

Threats & Exploits logo

Threats & Exploits

Understand, triage, and investigate application security findings and alerts.

Are you looking for something different?

We have hundreds of apps, extensions, and other technologies to customize your environment

Leverage our newest innovations of Dynatrace Saas

Kick-start your app creation

Kick-start your app creation

Whether you’re a beginner or a pro, Dynatrace Developer has the tools and support you need to create incredible apps with minimal effort.
Go to Dynatrace Developer
Upgrading from Dynatrace Managed to SaaS

Upgrading from Dynatrace Managed to SaaS

Drive innovation, speed, and agility in your organization by seamlessly and securely upgrading.
Learn More
Log Management and Analytics

Log Management and Analytics

Innovate faster and more efficiently with unified log management and log analytics for actionable insights and automation.
Learn more