OpenTelemetry Metrics

I'm trying to configure an ASP.NET Core application to send metrics that I can then view via Grafana. I'm pretty confused about the whole observability architecture.

Key points:
- Everything is running within Docker Compose
- I've managed to configure ASP -> OTLP -> Loki -> Grafana
- I've managed to configure Prometheus to scrape OTLP metrics

Potential issue: all examples I've seen magically connect to the OTLP collector without specifying its endpoint. Whenever I've tried that, didnt work, so I use

HttpProtobuf

HttpProtobuf

mode with the HTTP endpoint injected from env.

My confusion: I'm lost in the architecture.
- Do I need OTLP? Why is it useful?
- How does Prometheus come into play? Do I send metrics ASP -> OTLP and then configure Prom. to scrape that? Or expose my ASP metrics for Prom. to scrape and expose to Grafana?

  prometheus:
    container_name: prometheus
    hostname: "prometheus"
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./configs/prometheus.yml:/etc/prometheus/prometheus.yml
  collector:
    container_name: "collector"
    hostname: "collector"
    image: otel/opentelemetry-collector-contrib:latest
    command: '--config=/etc/otelcol/otel-collector-config.yml'
    volumes:
      - ./configs:/etc/otelcol
    ports:
      - "4317:4317"
      - "4318:4318"
      - "8888:8888"
      - "13133:13133"
      - "55679:55679"
      - "1777:1777"
      - "3500:3500"
      - "3600:3600"

  prometheus:
    container_name: prometheus
    hostname: "prometheus"
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./configs/prometheus.yml:/etc/prometheus/prometheus.yml
  collector:
    container_name: "collector"
    hostname: "collector"
    image: otel/opentelemetry-collector-contrib:latest
    command: '--config=/etc/otelcol/otel-collector-config.yml'
    volumes:
      - ./configs:/etc/otelcol
    ports:
      - "4317:4317"
      - "4318:4318"
      - "8888:8888"
      - "13133:13133"
      - "55679:55679"
      - "1777:1777"
      - "3500:3500"
      - "3600:3600"

Excerpt from

docker-compose.yml

docker-compose.yml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  prometheus:
    config:
      scrape_configs:
        - job_name: 'collector'
          scrape_interval: 10s
          static_configs:
            - targets: ['0.0.0.0:8888']
exporters:
  otlphttp/metrics:
    endpoint: "http://prometheus:9090/api/v1/otlp"
service:
  pipelines:
    metrics:
      receivers: [otlp,prometheus]
      exporters: [debug,otlphttp/metrics]

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  prometheus:
    config:
      scrape_configs:
        - job_name: 'collector'
          scrape_interval: 10s
          static_configs:
            - targets: ['0.0.0.0:8888']
exporters:
  otlphttp/metrics:
    endpoint: "http://prometheus:9090/api/v1/otlp"
service:
  pipelines:
    metrics:
      receivers: [otlp,prometheus]
      exporters: [debug,otlphttp/metrics]

OpenTelemetry Metrics

HttpProtobuf

HttpProtobuf

  prometheus:
    container_name: prometheus
    hostname: "prometheus"
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./configs/prometheus.yml:/etc/prometheus/prometheus.yml
  collector:
    container_name: "collector"
    hostname: "collector"
    image: otel/opentelemetry-collector-contrib:latest
    command: '--config=/etc/otelcol/otel-collector-config.yml'
    volumes:
      - ./configs:/etc/otelcol
    ports:
      - "4317:4317"
      - "4318:4318"
      - "8888:8888"
      - "13133:13133"
      - "55679:55679"
      - "1777:1777"
      - "3500:3500"
      - "3600:3600"

  prometheus:
    container_name: prometheus
    hostname: "prometheus"
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./configs/prometheus.yml:/etc/prometheus/prometheus.yml
  collector:
    container_name: "collector"
    hostname: "collector"
    image: otel/opentelemetry-collector-contrib:latest
    command: '--config=/etc/otelcol/otel-collector-config.yml'
    volumes:
      - ./configs:/etc/otelcol
    ports:
      - "4317:4317"
      - "4318:4318"
      - "8888:8888"
      - "13133:13133"
      - "55679:55679"
      - "1777:1777"
      - "3500:3500"
      - "3600:3600"

Excerpt from

docker-compose.yml

docker-compose.yml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  prometheus:
    config:
      scrape_configs:
        - job_name: 'collector'
          scrape_interval: 10s
          static_configs:
            - targets: ['0.0.0.0:8888']
exporters:
  otlphttp/metrics:
    endpoint: "http://prometheus:9090/api/v1/otlp"
service:
  pipelines:
    metrics:
      receivers: [otlp,prometheus]
      exporters: [debug,otlphttp/metrics]

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  prometheus:
    config:
      scrape_configs:
        - job_name: 'collector'
          scrape_interval: 10s
          static_configs:
            - targets: ['0.0.0.0:8888']
exporters:
  otlphttp/metrics:
    endpoint: "http://prometheus:9090/api/v1/otlp"
service:
  pipelines:
    metrics:
      receivers: [otlp,prometheus]
      exporters: [debug,otlphttp/metrics]

OpenTelemetry Metrics

OpenTelemetry Metrics

Similar Threads

Similar Threads

Similar Threads