大数据监控体系构建

基于开源xxx_exporter+promethues+grafana的构建监控系统

1.概况

其中，exporter一般是使用来采集各种组件运行时的指标数据；

promethues构建指标时序数据库；

grafana构建指标显示面板；

目前已有各种docker容器方便的构建各种监控体系；

2.详细构建过程

其中我的promethues的配置如下：

#全局配置信息：global:scrape_interval:15s #默认抓取间隔1m, 15秒向目标抓取一次数据evaluation_interval: 15s #个性化的抓取配置信息scrape_configs:- job_name: prometheusstatic_configs:- targets: ['192.168.32.70:9090']labels:instance: 192.168.32.70#监控物理主机：cpu、memory、disk等 - job_name: node_exporterstatic_configs:- targets:- '192.168.32.62:9100'- '192.168.32.63:9100'- '192.168.32.64:9100'- '192.168.32.65:9100'- '192.168.32.66:9100'- '192.168.32.67:9100'- '192.168.32.68:9100'- '192.168.32.69:9100'- '192.168.32.70:9100'- '192.168.32.71:9100'- '192.168.32.72:9100'- '192.168.32.73:9100'- '192.168.32.101:9100'- '192.168.32.106:9100'- '192.168.32.111:9100'- '192.168.32.116:9100'- '192.168.32.121:9100'- '192.168.32.126:9100'- '192.168.32.141:9100'- '192.168.32.146:9100'# - job_name: 'redis'# static_configs:#- targets: ['192.168.32.70:9121']# labels:# instance: redis-poc- job_name: 'mongo'static_configs:- targets: ['192.168.32.70:9104']labels:instance: mongo3- job_name: 'postgres'static_configs:- targets: ['192.168.32.70:9187']labels:instance: 'node70'platform: 'dcos-node70'- job_name: 'kafka'static_configs:- targets: ['192.168.32.70:9308']labels:instance: kafka-67- job_name: 'redis_exporter_poc'static_configs:- targets:- redis://192.168.32.141:16055- redis://192.168.32.68:16052- redis://192.168.32.68:16053metrics_path: /scraperelabel_configs:- source_labels: [__address__]target_label: __param_target- source_labels: [__param_target]target_label: instance- target_label: __address__replacement: 192.168.32.70:9121- job_name: 'elasticsearch'scrape_interval: 15smetrics_path: "/_prometheus/metrics"static_configs:- targets: ['192.168.32.62:9200']

3.构建之后的效果

grafana指标展示界面

2.在线下载已知的控制面板

监控虚拟机物理节点资源

kafka相关监控体系

其他的类似，还有redis、postgresql、mysql、MongoDB等相关的监控体系；

甚至还有微服务相关的监控体系

时间关系，大概展示一些图片，其实dashboard面板还是挺美观的