标签搜索

目 录CONTENT

文章目录

自定义Prometheus微服务监控

陈铭
2023-04-04 / 0 评论 / 0 点赞 / 230 阅读 / 1,314 字 / 正在检测是否收录...

Docker部署

# 安装prometheus
docker run -d -p 9090:9090 --name prometheus --restart=always -v /opt/prometheus/data/:/data  -v /opt/prometheus/prometheus.yml:/opt/prometheus/prometheus.yml -v /opt/prometheus/rule.yml:/opt/prometheus/rule.yml prom/prometheus --config.file=/opt/prometheus/prometheus.yml --web.enable-lifecycle --storage.tsdb.retention=90d
# 安装grafana
docker run -d --name=grafana -p 3000:3000 grafana/grafana 

其中prometheus挂载了一些配置文件进去

prometheus.yml

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - "/opt/prometheus/rule.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["192.168.217.134:9090"]
    
  - job_name: 'node-exporter'
    static_configs:
      - targets: ['192.168.217.134:9100']
  - job_name: 'casesys'
    metrics_path: '/rycem/api/casesys/actuator/prometheus'
    static_configs:
      - targets: ['192.168.217.134:9102']

rule.yml

groups:
- name: Hosts.rules
  rules:
  - alert: HostDown           
    expr: up{job=~"node-exporter|prometheus|grafana|alertmanager|casesys"} == 0
    for: 0m
    labels:
      severity: ERROR
    annotations:
      title: 'Instance down'
      summary: "{{$labels.instance}}"
      description: "主机: 【{{ $labels.instance }}】has been down for more than 1 minute"

微服务改造

自定义Prometheus注解模块

Pom.xml

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>

注解类

package com.rycem.common.prometheus.annotation;


import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import static java.lang.annotation.ElementType.METHOD;

/**
 * @ClassName PrometheusMetrics
 * @Description 普罗米修斯自定义注释
 * @Author 陈铭
 * @Date 21:42 2023/4/3
 * @Version 1.0
 **/
@Target(METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface PrometheusMetrics {
    String value() default "";
}

切面类

切面直接切至上述的注解,那么所有被注解的接口都会被切面到

package com.rycem.common.prometheus.aop;


import com.rycem.common.prometheus.exception.MetricException;
import com.rycem.common.prometheus.monitor.PrometheusMonitor;
import io.prometheus.client.Histogram;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;

import javax.servlet.http.HttpServletRequest;
import java.util.Objects;

/**
 * @ClassName PrometheusAspect
 * @Description 普罗米修斯切面
 * @Author 陈铭
 * @Date 21:42 2023/4/3
 * @Version 1.0
 **/
@Aspect
@Component
public class PrometheusAspect {

    @Autowired
    private PrometheusMonitor prometheusMonitor;

    @Pointcut("@annotation(com.rycem.common.prometheus.annotation.PrometheusMetrics)")
    public void point() {
    }

    @Around("point()")
    public Object around(ProceedingJoinPoint joinPoint) throws MetricException {
        Histogram.Timer requestTimer = null;
        String[] labels = null;
        Object result;

        try {
            // 获取request信息
            HttpServletRequest request = ((ServletRequestAttributes) Objects.requireNonNull(
                    RequestContextHolder.getRequestAttributes())
            ).getRequest();
            labels = prometheusMonitor.buildLabels(request, joinPoint);
            // 请求开始
            requestTimer = prometheusMonitor.getRequestConsuming().labels(labels).startTimer();
            // 请求总次数加1
            prometheusMonitor.getTotalCount().labels(labels).inc();
            result = joinPoint.proceed();
            // 请求成功次数加1
            prometheusMonitor.getSuccessCount().labels(labels).inc();
        } catch (Throwable e) {
            // 请求失败次数加1
            prometheusMonitor.getFailureCount().labels(labels).inc();
            throw new MetricException("【普罗米修斯监控AOP异常】==>"
                    .concat("接口URL: ").concat(Objects.requireNonNull(labels)[0])); // 抛出自定义异常
        } finally {
            // 请求结束
            assert requestTimer != null;
            requestTimer.observeDuration();
        }
        return result;
    }
}

监控类

监控类封装了Prometheus的各类计数器

package com.rycem.common.prometheus.monitor;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import io.micrometer.prometheus.PrometheusMeterRegistry;
import io.prometheus.client.Counter;
import io.prometheus.client.Histogram;
import lombok.Getter;
import lombok.Setter;
import org.aspectj.lang.ProceedingJoinPoint;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import javax.servlet.http.HttpServletRequest;

/**
 * @ClassName PrometheusMonitor
 * @Description 普罗米修斯监控类,包装了各类计数器
 * @Author 陈铭
 * @Date 22:15 2023/4/3
 * @Version 1.0
 **/
@Component
@Getter
@Setter
public class PrometheusMonitor {

    public static final String SUCCESS_COUNTER_NAME="success_count";
    public static final String FAILURE_COUNTER_NAME="failure_count";
    public static final String TOTAL_COUNTER_NAME="total_count";
    public static final String HISTOGRAM_NAME="request_consuming";
    public static final String[] LABEL_NAMES={"uri","method","params","body"};

    @Autowired
    private ObjectMapper objectMapper;

    @Autowired
    private PrometheusMeterRegistry registry;

    private Counter successCount;
    private Counter failureCount;
    private Counter totalCount;
    private Histogram requestConsuming;

    @PostConstruct
    public void init() {
        successCount = Counter.build().name(SUCCESS_COUNTER_NAME)
                .labelNames(LABEL_NAMES)
                .help("request error counter of api")
                .register(registry.getPrometheusRegistry());
        failureCount = Counter.build().name(FAILURE_COUNTER_NAME)
                .labelNames(LABEL_NAMES)
                .help("request error counter of api")
                .register(registry.getPrometheusRegistry());
        totalCount = Counter.build().name(TOTAL_COUNTER_NAME)
                .labelNames(LABEL_NAMES)
                .help("total request counter of api")
                .register(registry.getPrometheusRegistry());
        requestConsuming = Histogram.build().name(HISTOGRAM_NAME)
                .labelNames(LABEL_NAMES)
                .help("request consuming of api")
                .register(registry.getPrometheusRegistry());
    }

    public String[] buildLabels(HttpServletRequest request, ProceedingJoinPoint joinPoint) throws JsonProcessingException {
        String uri = request.getServletPath();
        String restMethod = request.getMethod();
        String params = "";
        String body = "";
        if ("GET".equals(restMethod)) {
            params = objectMapper.writeValueAsString(request.getParameterMap());
        } else {
            body = objectMapper.writeValueAsString(joinPoint.getArgs());
        }
        return new String[]{uri, restMethod, params, body};
    }

}

使用注解模块

  • pom中引入该模块的依赖
  • 对接口方法上方注解@PrometheusMetrics

测试

本地启动微服务,从Swagger执行一下接口,访问 http://127.0.0.1:9102/rycem/api/casesys/actuator/prometheus
image-1680621529647

Prometheus查看数据

将微服务打包成docker镜像并运行,之前启动的grafana和prometheus都在的
image-1680621535926
上述的prometheus.yml配置中已经就配好了监听该微服务,随便访问几次微服务接口,查询Prometheus控制台
image-1680621543193

Grafana查看数据

以case/list接口为例,查询成功和失败的数量
image-1680621554303

image-1680621556991

0

评论区