prometheus+grafanaonkubernetes部署(代码片段)

关键词：

一键部署链接 点我跳转
使用k8s方式部署Prometheus + grafana 监控pod 和 node 节点信息

环境基础：
k8s集群
创建命名空间方便管理

kubectl create namespace monitoring

3个组件

1.数据源pod（任选其一,推荐kube-state-metrics方式 更详细）

node节点基础数据，cpu，io，内存，磁盘，网络等
偏向使用DaemonSet部署的node-exporter

- node-exporter.yaml
一键部署默认是这种pod

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitoring
  labels:
    k8s-app: node-exporter
spec:
  selector:
    matchLabels:
      k8s-app: node-exporter
  template:
    metadata:
      labels:
        k8s-app: node-exporter
    spec:
      containers:
      - image: prom/node-exporter:v1.0.1
        name: node-exporter
        ports:
        - containerPort: 9100
          protocol: TCP
          name: http
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: node-exporter
  name: node-exporter
  namespace: monitoring
spec:
  ports:
 - name: http
    port: 9100
    nodePort: 31672
    protocol: TCP
  type: NodePort
  selector:
    k8s-app: node-exporter

        更为详细的k8s内部信息，pod，svc，namespace，等集群信息
推荐deployment部署kube-state-metrics方式
项目地址 点我跳转

        本次实验使用这种模式

        - kube-state-metrics.yaml

        包含ClusterRoleBinding-ClusterRole-sa-svc-dep 一系列资源创建作用是收集集群数据，所以不用在同一个namespace。

kubectl create ns ops-monit

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: v1.9.7
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: ops-monit
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: v1.9.7
  name: kube-state-metrics
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - replicasets
  - ingresses
  verbs:
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - list
  - watch
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - list
  - watch
- apiGroups:
  - certificates.k8s.io
  resources:
  - certificatesigningrequests
  verbs:
  - list
  - watch
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  - volumeattachments
  verbs:
  - list
  - watch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  verbs:
  - list
  - watch
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: v1.9.7
  name: kube-state-metrics
  namespace: ops-monit
---
apiVersion: v1
kind: Service
metadata:
#  annotations:
#    prometheus.io/scrape: 'true'
  labels:
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: v1.9.7
  name: kube-state-metrics
  namespace: ops-monit
spec:
  clusterIP: None
  ports:
  - name: http-metrics
    port: 8080
    targetPort: http-metrics
  - name: telemetry
    port: 8081
    targetPort: telemetry
  selector:
    app.kubernetes.io/name: kube-state-metrics
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: v1.9.7
  name: kube-state-metrics
  namespace: ops-monit
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kube-state-metrics
  template:
    metadata:
      labels:
        app.kubernetes.io/name: kube-state-metrics
        app.kubernetes.io/version: v1.9.7
    spec:
      containers:
      - image: quay.mirrors.ustc.edu.cn/coreos/kube-state-metrics:v1.9.7
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 5
          timeoutSeconds: 5
        name: kube-state-metrics
        ports:
        - containerPort: 8080
          name: http-metrics
        - containerPort: 8081
          name: telemetry
        readinessProbe:
          httpGet:
            path: /
            port: 8081
          initialDelaySeconds: 5
          timeoutSeconds: 5
      nodeSelector:
        beta.kubernetes.io/os: linux
      serviceAccountName: kube-state-metrics

2.Prometheus相关

• rbac-setup.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
- apiGroups: [""]
  resources:
  - nodes
  - nodes/proxy
  - services
  - endpoints
  - pods
  verbs: ["get", "list", "watch"]
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
  verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus
  namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
subjects:
- kind: ServiceAccount
  name: prometheus
  namespace: monitoring

• configmap.yaml
  用与重新定义收集到的数据源提供给grafan展现

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
  namespace: monitoring
data:
  prometheus.yml: |
    global:
      scrape_interval:     15s
      evaluation_interval: 15s
    scrape_configs:
    - job_name: 'k8s-cadvisor'
      metrics_path: /metrics/cadvisor
      kubernetes_sd_configs:
      - role: node
      relabel_configs:
      - source_labels: [__address__]
        regex: '(.*):10250'
        replacement: '$1:10255'
        target_label: __address__
        action: replace
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)

      metric_relabel_configs:
      - source_labels: [instance]
        separator: ;
        regex: (.+)
        target_label: node
        replacement: $1
        action: replace

      - source_labels: [pod_name]
        separator: ;
        regex: (.+)
        target_label: pod
        replacement: $1
        action: replace
      - source_labels: [container_name]
        separator: ;
        regex: (.+)
        target_label: container
        replacement: $1
        action: replace

    - job_name: kube-state-metrics
      kubernetes_sd_configs:
      - role: endpoints
        namespaces:
          names:
          - ops-monit
      relabel_configs:
      - source_labels: [__meta_kubernetes_service_label_app_kubernetes_io_name]
        regex: kube-state-metrics
        replacement: $1
        action: keep
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: k8s_namespace
      - source_labels: [__meta_kubernetes_service_name]
        action: replace
        target_label: k8s_sname

• prometheus-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: prometheus-deployment
  name: prometheus
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      containers:
      - image: prom/prometheus:v2.7.1
        name: prometheus
        command:
        - "/bin/prometheus"
        args:
        - "--config.file=/etc/prometheus/prometheus.yml"
        - "--storage.tsdb.path=/prometheus"
        - "--storage.tsdb.retention=24h"
        ports:
        - containerPort: 9090
          protocol: TCP
        volumeMounts:
        - mountPath: "/prometheus"
          name: data
        - mountPath: "/etc/prometheus"
          name: config-volume
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 500m
            memory: 2500Mi
      serviceAccountName: prometheus    
      volumes:
      - name: data
        emptyDir: 
      - name: config-volume
        configMap:
          name: prometheus-config
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: prometheus
  name: prometheus
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - port: 9090
    targetPort: 9090
    nodePort: 30003
  selector:
    app: prometheus

        3. Grafana 部署 （两种方式,任选其一 k8s-dep部署 or docker-compose部署）

        docker-compose部署

version: '3'
services:
  grafana:
    image: grafana/grafana                                       # 原镜像`grafana/grafana`
    container_name: grafana                                      # 容器名为'grafana'
    restart: always                                              # 指定容器退出后的重启策略为始终重启
    volumes:                                                     # 数据卷挂载路径设置,将本机目录映射到容器目录
      - "./data:/var/lib/grafana"
      - "./log:/var/log/grafana"
    ports:                              # 映射端口
      - "3000:3000"

        k8s-dep方式

• grafana.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana-dep
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana-dep
  template:
    metadata:
      labels:
        app: grafana-dep
    spec:
      containers:
      - image: grafana/grafana
        imagePullPolicy: Always
        #command:
        #  - "tail"
        #  - "-f"
        #  - "/dev/null"
        securityContext:
          allowPrivilegeEscalation: false
          runAsUser: 0
        name: grafana
        ports:
        - containerPort: 3000
          protocol: TCP
        volumeMounts:
        - mountPath: "/var/lib/grafana"
          name: data
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 500m
            memory: 2500Mi
      volumes:
      - name: data
        emptyDir: 
---
apiVersion: v1
kind: Service
metadata:
  name: grafana
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 3000
    nodePort: 30006
  selector:
    app: grafana-dep

        执行yaml文件

kubectl create -f xxx.yaml

        查看相关资源状态

kubectl get all -n ops-monit ##查看kube-state-metrics服务

kubectl get all -n monitoring ##查看监控相关

         相关地址：http://:30003 ##Prometheus地址第一步查看是否有数据源

        http://192.168.17.130:30006/ ##grafana地址 默认密码admin/addmin

开始配置grafana

添加数据源

 添加上面的ip:30003端口

导入grafan模板
地址 点我跳转
13105 官网 Dashboard-id

 最后效果展示

prometheus

官方网址：https://prometheus.io/GitHub网址：https://github.com/prometheus/prometheus软件下载地址：https://prometheus.io/download/第三方中文介绍：https://github.com/1046102779/prometheus  查看详情

prometheus视频教程

1.prometheus计算cpu的使用率算法-12.prometheus计算cpu的使用率算法-23.prometheus的计算函数4.prometheus公式拆分详细讲解5.prometheus的公式得到最终解控指标结果6.prometheus的公式取单个指标的值7.prometheus的gauge类型数据bash脚本pushgateway到server... 查看详情

prometheus在kubernetes集群的搭建教程

Prometheus在kubernetes集群的搭建一、Prometheus介绍1.Prometheus简介2.Prometheus介绍3.Prometheus与市面的监控系统区别4.Prometheus特点5.Prometheus的网站6.Prometheus原理图7.Prometheus监控示意图二、集群环境检查1.检查kubernetes的集群状态2.Prometheus使用... 查看详情

prometheus快照

prometheus快照：启动prometheus服务时加--web.enable-admin-api/space/prometheus/prometheus--config.file=/space/prometheus/prometheus.yml--storage.tsdb.path=/space/prometheus/data--web.enable-admin-api制作快照：curl-X 查看详情

prometheus：prometheus如何处理数据类型溢出？

】prometheus：prometheus如何处理数据类型溢出？【英文标题】：prometheus:howprometheushandlesthedatatypeoverflow?【发布时间】：2018-11-1600:14:18【问题描述】：prometheus客户端不断更新指标，似乎不考虑溢出。比如计数器Add()：func(c*counter)Add(vfl... 查看详情

prometheus监控??简介

文档目录：?prometheus是什么？?prometheus能为我们带来些什么?prometheus对于运维的要求?prometheus多图效果展示1)Prometheus是什么prometheus是?个开源系统监控和报警的?具集合，由SoundCloud创建（http://soundcloud.com/），?从2012诞?之后，?今已经... 查看详情

prometheus+grafana，开源监控神器！(代码片段)

1Prometheus介绍 Prometheus是一个开源系统监控和警报工具包，用于收集和聚合指标作为时间序列数据，即将指标信息、记录的时间戳以及称为标签的可选键值对一起存储，详细介绍请查看官方文档 2Prometheus及其组件安装因Prom... 查看详情

prometheus

Visit prometheus.io forthefulldocumentation,examplesandguides.Prometheus,a CloudNativeComputingFoundation project,isasystemsandservicemonitoringsystem.Itcollectsmetricsfromconfiguredtargetsatgiveninte 查看详情

prometheus 未启动（未加载 prometheus.yml 文件）

】prometheus未启动（未加载prometheus.yml文件）【英文标题】：prometheusnotstarting(notloadingprometheus.ymlfile)【发布时间】：2021-08-0310:19:06【问题描述】：在启动Prometheus时出现以下错误：level=errorts=2021-05-13T04:42:39.652Zcaller=main.go:355msg="Errorlo... 查看详情

prometheus+grafana

prometheus集中管理服务搭建#搭建在监控服务主机上，用于收集节点服务器信息下载：https://prometheus.io/download/解压运行：nohup./prometheus--config.file=./prometheus.yml&>>./prometheus.log&访问http://192.168.1.24:9090node-exporter 查看详情

无法使用 docker (prom/prometheus) 加载 prometheus.yml 配置文件

】无法使用docker(prom/prometheus)加载prometheus.yml配置文件【英文标题】：Can\'tloadprometheus.ymlconfigfilewithdocker(prom/prometheus)【发布时间】：2019-03-0213:53:39【问题描述】：我正在尝试使用以下自定义conf文件通过docker加载prometheus：danilo@mach... 查看详情

监控利器prometheus——prometheus简介与安装(代码片段)

Prometheus简介与安装1、Prometheus简介2、安装PrometheusServer3、安装Grafana1、Prometheus简介Prometheus是一个根据应用的metrics来进行监控的开源工具，所有采集的监控数据均以指标(metric)的形式保存在内置的时间序列数据库当中(TSDB)。时... 查看详情

prometheus告警处理(代码片段)

在PrometheusServer中定义告警规则以及产生告警，Alertmanager组件则用于处理这些由Prometheus产生的告警。Alertmanager即Prometheus体系中告警的统一处理中心。Prometheus告警简介告警能力在Prometheus的架构中被划分成两个独立的部分。如下所... 查看详情

prometheus详解——prometheus原理与架构

...继续给大家介绍Linux运维相关知识，本文主要内容是Prometheus的原理与架构。一、Prometheus架构Prometheus的监控实现，是基于HTTP周期性的抓取被监控组件的状态，任意组件只要提供符合Prometheus定义的数据格式的HTTP接口... 查看详情

将 prometheus 就绪应用程序添加到 Prometheus

】将prometheus就绪应用程序添加到Prometheus【英文标题】：AddprometheusreadyapplicationtothePrometheus【发布时间】：2021-01-1515:05:49【问题描述】：我在Kubernetes服务的default命名空间上安装了一个python应用程序。它使用starlette_exporter并将/metri... 查看详情

prometheus基础知识及部署！(代码片段)

PrometheusPrometheus基础知识prometheus监控目标Prometheus特性Prometheus监控体系Prometheus监控四大指标Prometheus部署安装Prometheus两server节点安装node_exporter安装grafanaPrometheus基础知识prometheus采集的主要是时间序列数据，采集随着时间变化... 查看详情

prometheus基础知识及部署！(代码片段)

PrometheusPrometheus基础知识prometheus监控目标Prometheus特性Prometheus监控体系Prometheus监控四大指标Prometheus部署安装Prometheus两server节点安装node_exporter安装grafanaPrometheus基础知识prometheus采集的主要是时间序列数据，采集随着时间变化... 查看详情

prometheus安装

1、prometheus下载https:/prometheus.io/download/2、查看文件类型[root@hadoop3home]#fileprometheus-2.37.1.linux-amd64.tar.gz指定解压路径解压文件[root@hadoop3home]#tarxfprometheus-2.37.1.linux-amd64.tar.gz-C/opt/[root@hadoop3prometheus]#./prometheus--version3、后台启动prometheus... 查看详情