Batch OTLP requests
The following configuration example shows how you configure a Collector instance and its native batch processor to queue and batch OTLP requests and improve throughput performance.
Demo configuration
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
send_batch_max_size: 1000
timeout: 30s
send_batch_size : 800
exporters:
otlphttp:
endpoint: $DT_ENDPOINT/api/v2/otlp
headers:
Authorization: "Api-Token $DT_API_TOKEN"
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
logs:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
Prerequisites
- At least Core distribution with the batch processor
- The API URL of your Dynatrace environment
- An API token with the relevant access scope
Components
For our configuration, we configure the following components.
Receiver
Under receivers
, we specify the standard otlp
receiver as active receiver component for our Collector instance.
This is for demonstration purposes. You can specify any other valid receiver here (for example, zipkin
).
Processor
Under processors
, we specify the batch
processor with the following parameters:
send_batch_max_size
configured for a maximum of 1,000 entries per batchtimeout
configured to always send data after 30 seconds, regardless of any other batch limitssend_batch_size
configured to always send data after 800 entries, regardless of any other batch limits
With this configuration, the Collector queues telemetry entries in batches and sends a batch either after 30 seconds have passed or at least 800 entries are queued.
Exporter
Under exporters
, we specify the default otlphttp
exporter and configure it with our Dynatrace API URL and the required authentication token.
For this purpose, we set the following two environment variables and reference them in the configuration values for endpoint
and Authorization
.
DT_ENDPOINT
contains the base URL of your ActiveGate (for example,https://{your-environment-id}.live.dynatrace.com
)DT_API_TOKEN
contains the API token
Service pipeline
Under service
, we assemble our receiver and exporter objects into pipelines for traces, metrics, and logs and enable our batch processor by referencing it under processors
for each respective pipeline.