• 欢迎访问露水湾网站,WordPress信息,WordPress教程,推荐使用最新版火狐浏览器和Chrome浏览器访问本网站
  • Git主题现已支持滚动公告栏功能,兼容其他浏览器,看到的就是咯,在后台最新消息那里用li标签添加即可。
  • 最新版Git主题已支持说说碎语功能,可像添加文章一样直接添加说说,新建说说页面即可,最后重新保存固定连接,

kubernetes api servers showing down in prometheus (kube-state-metrics) –“forbidden: User \”system:anonymous\“ cannot get path \”/metrics\“”,

docker dewbay 2个月前 (05-12) 92次浏览 已收录 0个评论 扫描二维码

I’m new to k8s, prometheus. I’m trying to collect the metrics of each pods with prometheus but unable to so because of the error: API ERROR.

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/metrics\"",
  "reason": "Forbidden",
  "details": {

  },
  "code": 403
}

system:anonymous means that an unauthenticated user is trying to get a resource from your cluster, which is forbidden. You will need to create a service account, then give that service account some permissions through RBAC, then make that service account to get the metrics. All that is documented.

As a workaround, you can do this:

kubectl create clusterrolebinding prometheus-admin --clusterrole cluster-admin --user system:anonymous

Now, note that this is a terrible idea, unless you are playing with kubernetes. With this permission you are giving any unauthenticated user total permissions into your cluster.

Create the following manifests:

ServiceAccount.yaml:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
        labels:
            app.kubernetes.io/name: kube-state-metrics
        name: kube-state-metrics
        namespace: grafana

ClusterRole.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
    labels:
        app.kubernetes.io/name: kube-state-metrics
    name: kube-state-metrics
rules:
    - apiGroups:
        - ""
      resources:
          - configmaps
          - secrets
          - nodes
          - pods
          - services
          - resourcequotas
          - replicationcontrollers
          - limitranges
          - persistentvolumeclaims
          - persistentvolumes
          - namespaces
          - endpoints
      verbs:
          - list
          - watch
    - apiGroups:
        - extensions
      resources:
          - daemonsets
          - deployments
          - replicasets
          - ingresses
      verbs:
          - list
          - watch
    - apiGroups:
        - apps
      resources:
          - statefulsets
          - daemonsets
          - deployments
          - replicasets
      verbs:
          - list
          - watch
    - apiGroups:
        - batch
      resources:
          - cronjobs
          - jobs
      verbs:
          - list
          - watch
    - apiGroups:
        - autoscaling
      resources:
          - horizontalpodautoscalers
      verbs:
          - list
          - watch
    - apiGroups:
        - authentication.k8s.io
      resources:
          - tokenreviews
      verbs:
        - create
    - apiGroups:
        - authorization.k8s.io
      resources:
          - subjectaccessreviews
      verbs:
          - create
    - apiGroups:
        - policy
      resources:
          - poddisruptionbudgets
      verbs:
          - list
          - watch
    - apiGroups:
        - certificates.k8s.io
      resources:
          - certificatesigningrequests
      verbs:
          - list
          - watch
    - apiGroups:
        - storage.k8s.io
      resources:
          - storageclasses
      verbs:
          - list
          - watch
    - nonResourceURLs:
          - "/metrics"
      verbs:
          - get

ClusterRoleBinding.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
    labels:
        app.kubernetes.io/name: kube-state-metrics
    name: kube-state-metrics
roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: kube-state-metrics
subjects:
    - kind: ServiceAccount
      name: kube-state-metrics
      namespace: grafana

And inform your Kube-State-Metrics deployment to use the new ServiceAccount with the following addition to your Template Spec: serviceAccountName: kube-state-metrics.

TJ Zimmerman

1,1048 silver badges24 bronze badges
  • 1
    does namespace: “{{ namespace }}” value take the desired namespace to scrape? – SOF Nov 7 ’19 at 14:45
  • 1
    Sorry, that was jinja templating left over from my personal code. I use Ansible to deploy all of this and inject variables in with Jinja. I have updated my response and switched the namespace declaration to Grafana which is the variable set for my namespaces. The core idea here is that you deploy a ServiceAccount to the namespace in which you want to use it. Then you provision a ClusterRole which is namespace-agnostic where you delegate the permissions you would associate with the ServiceAccount. Then you create a ClusterRoleBinding to bind the CR to the SA. Wherein you specify the NS. – TJ Zimmerman Nov 8 ’19 at

露水湾 , 版权所有丨如未注明 , 均为原创丨本网站采用BY-NC-SA协议进行授权
转载请注明原文链接:kubernetes api servers showing down in prometheus (kube-state-metrics) –“forbidden: User \”system:anonymous\“ cannot get path \”/metrics\“”,
喜欢 (0)
[]
分享 (0)
关于作者:
发表我的评论
取消评论

表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址