Feature Request - toggle for Prometheus scraping as ClusterRole

[dplusic]

Expected Behaviour

Helm Chart should not create ClusterRole when clusterRole=false.

Current Behaviour

It creates ClusterRole even if clusterRole=false.

Possible Solution

It should create Role instead of ClusterRole.

Steps to Reproduce (for bugs)

1. Prepare a service account without permission creating ClusterRole
2. Run helm install openfaas openfaas/openfaas --set clusterRole=false --set createCRDs=false
3. Error message: Error: rendered manifests contain a resource that already exists. Unable to continue with install: could not get information about the resource: clusterroles.rbac.authorization.k8s.io "openfaas-prometheus" is forbidden: User "system:serviceaccount:default:default" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

Context

I want namespaced installation of openfaas.

Your Environment

• FaaS-CLI version ( Full output from: faas-cli version ): -

• Docker version docker version (e.g. Docker 17.0.05 ): -

• What version and distriubtion of Kubernetes are you using? kubectl version

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.4", GitCommit:"d360454c9bcd1634cf4cc52d1867af5491dc9c5f", GitTreeState:"clean", BuildDate:"2020-11-11T13:17:17Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

• Operating System and version (e.g. Linux, Windows, MacOS): Ubuntu 18.04 on WSL

• Link to your project or a code example to reproduce issue: -

• What network driver are you using and what CIDR? i.e. Weave net / Flannel: -

[alexellis]

This is expected, Prometheus needs to scrape in more than one namespace. The cluster role flag that you mention is only for the OpenFaaS Operator and does not configure the RBAC for Prometheus.

Happy for you to send a PR, otherwise you'll need to wait for this to be prioritized. Premium Subscription customers get priority.

[alexellis]

/set title: Feature Request - toggle for Prometheus scraping as ClusterRole

[alexellis]

I took a look into this today to refresh my memory, you should read the notes on this PR:

71edafa#diff-48d9710e441cfca34edd16570dabeda39dc6ecfae662e42f4e9bab7813290aa7

And if converted to two Roles and RoleBindings to the Prometheus SA, the following error is given by Prometheus which fails to work:

level=error ts=2020-11-29T16:33:17.946Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:283: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:openfaas:openfaas-prometheus\" cannot list resource \"pods\" in API group \"\" in the namespace \"openfaas-fn\": RBAC: role.rbac.authorization.k8s.io \"openfaas-prometheus\" not found"

With:

alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get sa -n openfaas
NAME                  SECRETS   AGE
default               1         4m55s
openfaas-prometheus   1         4m48s
openfaas-controller   1         4m48s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get sa -n openfaas-fn
NAME      SECRETS   AGE
default   1         4m56s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get role -n openfaas
NAME                  CREATED AT
openfaas-prometheus   2020-11-29T16:29:47Z
openfaas-profiles     2020-11-29T16:29:47Z
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get role -n openfaas-fn
NAME                  CREATED AT
openfaas-controller   2020-11-29T16:29:47Z
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get rolebinding -n openfaas
NAME                  ROLE                       AGE
openfaas-prometheus   Role/openfaas-prometheus   5m3s
openfaas-profiles     Role/openfaas-profiles     5m3s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get rolebinding -n openfaas-fn
NAME                  ROLE                       AGE
openfaas-prometheus   Role/openfaas-prometheus   5m4s
openfaas-controller   Role/openfaas-controller   5m4s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ 

We would welcome your suggestions @dplusic