console 403: no RBAC policy matched for system:anonymous

[michaelgugino]

https://github.com/openshift/installer/blob/master/docs/dev/libvirt-howto.md#connect-to-the-cluster-console

https://${OPENSHIFT_INSTALL_CLUSTER_NAME}-api.${OPENSHIFT_INSTALL_BASE_DOMAIN}:6443/console/

does not pull up a console, just json output.

bf36c90

[wking]

does not pull up a console, just json output.

What is the JSON output? Things have been exciting since #330 landed, there are many things that could be going wrong ;).

Also, we have a nice issue template... ;). Even if you push your issues with hub or whatever instead of going through GitHub's web UI, those are the sort of things we're looking for in bug reports.

[michaelgugino]

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/console\": no RBAC policy matched",
  "reason": "Forbidden",
  "details": {
    
  },
  "code": 403
}

Yeah, it wasn't working last week either, not sure if the output was the same, but never got a console.

[wking]

"message": "forbidden: User \"system:anonymous\" cannot get path \"/console\": no RBAC policy matched",

We think this is openshift/origin#20983.

[mtnbikenc]

Ran into the same problem.

Platform: libvirt
Accessing: https://test1-api.tt.testing:6443/console
Response:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/console\": no RBAC policy matched",
  "reason": "Forbidden",
  "details": {
    
  },
  "code": 403
}

$ bin/openshift-install version
bin/openshift-install v0.1.0-65-gc0331204949479b41010ff7553297075a777bc2a
Terraform v0.11.8

$ oc get pods --all-namespaces                                                                                                                                                                                                                                                [±master ●]
NAMESPACE                              NAME                                                      READY     STATUS                 RESTARTS   AGE
kube-system                            kube-apiserver-f2lzx                                      1/1       Running                0          33m
kube-system                            kube-controller-manager-b6985978d-z8tnw                   1/1       Running                0          33m
kube-system                            kube-core-operator-7f4d6b8dcf-6kzgl                       1/1       Running                0          26m
kube-system                            kube-dns-787c975867-p8dtz                                 3/3       Running                0          33m
kube-system                            kube-flannel-5rn9f                                        2/2       Running                0          27m
kube-system                            kube-flannel-fdv56                                        2/2       Running                0          27m
kube-system                            kube-flannel-z7vcb                                        2/2       Running                4          19m
kube-system                            kube-proxy-kbs8t                                          1/1       Running                0          33m
kube-system                            kube-proxy-klfqx                                          1/1       Running                0          19m
kube-system                            kube-proxy-lkwf8                                          1/1       Running                0          33m
kube-system                            kube-scheduler-78d86f9754-gdkdd                           1/1       Running                0          33m
kube-system                            metrics-server-5767bfc576-tzp4p                           2/2       Running                0          24m
kube-system                            pod-checkpointer-5847n                                    1/1       Running                0          33m
kube-system                            pod-checkpointer-5847n-test1-master-0                     1/1       Running                0          32m
kube-system                            tectonic-network-operator-qbgn7                           1/1       Running                0          33m
openshift-cluster-api                  clusterapi-apiserver-6b855f7bc5-bxqmp                     2/2       Running                0          26m
openshift-cluster-api                  clusterapi-controllers-85f6bfd9d5-96cd6                   2/2       Running                0          25m
openshift-cluster-api                  machine-api-operator-5d85454676-8tgmf                     1/1       Running                0          33m
openshift-cluster-version              bootstrap-cluster-version-operator-test1-bootstrap        1/1       Running                1          33m
openshift-cluster-version              cluster-version-operator-8bb6cff75-nkwrl                  1/1       Running                0          33m
openshift-core-operators               openshift-service-cert-signer-operator-6dbfb84555-wsm54   1/1       Running                0          6m
openshift-ingress                      default-http-backend-6985d557bb-r5w29                     1/1       Running                0          25m
openshift-ingress                      router-67b6b747c5-jp7vk                                   1/1       Running                0          25m
openshift-ingress                      tectonic-ingress-controller-operator-fcb9c6f4b-jtdcn      1/1       Running                0          26m
openshift-machine-config-operator      machine-config-controller-5b74549d47-w6q65                1/1       Running                0          26m
openshift-machine-config-operator      machine-config-daemon-jfjrz                               1/1       Running                3          19m
openshift-machine-config-operator      machine-config-daemon-mfjvb                               1/1       Running                0          24m
openshift-machine-config-operator      machine-config-operator-68fcdcb4f9-7bxqd                  1/1       Running                0          33m
openshift-machine-config-operator      machine-config-server-8rprn                               1/1       Running                0          25m
openshift-operator-lifecycle-manager   catalog-operator-7db987f97b-xdrcj                         0/1       CreateContainerError   0          33m
openshift-operator-lifecycle-manager   olm-operator-748f55cdd-m254c                              0/1       CreateContainerError   0          33m
openshift-operator-lifecycle-manager   olm-operator-867dbbb8f9-sj9dk                             0/1       CreateContainerError   0          5m
openshift-operator-lifecycle-manager   package-server-7fb4c799bc-v4h4c                           0/1       CreateContainerError   0          33m
openshift-service-cert-signer          apiservice-cabundle-injector-7bc57bccf4-hrz9z             1/1       Running                0          5m
openshift-service-cert-signer          configmap-cabundle-injector-5889f997bc-4cxmn              1/1       Running                0          5m
openshift-service-cert-signer          service-serving-cert-signer-5d867cb55d-44gxm              1/1       Running                0          5m
tectonic-system                        kube-addon-operator-784b4b6c7-d8sjh                       1/1       Running                0          26m

Was the web-console replaced by something?

[imranrazakhan]

I was getting same json issue and just run below command

oc login -u system:admin

Now i am able to access it

[praveenkumar]

Same error I am getting today.

# kubectl get --all-namespaces pods
NAMESPACE                              NAME                                                              READY     STATUS             RESTARTS   AGE
default                                registry-6fcb8b7789-4x72w                                         1/1       Running            0          20m
kube-system                            kube-apiserver-krqjc                                              1/1       Running            0          41m
kube-system                            kube-controller-manager-b6985978d-zxhm2                           1/1       Running            0          41m
kube-system                            kube-core-operator-7f4d6b8dcf-hdfgw                               1/1       Running            0          32m
kube-system                            kube-dns-787c975867-kz8np                                         3/3       Running            0          41m
kube-system                            kube-flannel-h2kqm                                                2/2       Running            5          29m
kube-system                            kube-flannel-lt5dd                                                2/2       Running            0          36m
kube-system                            kube-flannel-t4l49                                                2/2       Running            0          36m
kube-system                            kube-proxy-5xz66                                                  1/1       Running            0          29m
kube-system                            kube-proxy-kvgxr                                                  1/1       Running            0          41m
kube-system                            kube-proxy-nbqvd                                                  1/1       Running            0          41m
kube-system                            kube-scheduler-78d86f9754-j4pr8                                   1/1       Running            0          41m
kube-system                            metrics-server-5767bfc576-67znk                                   2/2       Running            0          30m
kube-system                            pod-checkpointer-7v2xn                                            1/1       Running            0          41m
kube-system                            pod-checkpointer-7v2xn-test1-master-0                             1/1       Running            0          39m
kube-system                            tectonic-network-operator-76qt7                                   1/1       Running            0          41m
openshift-apiserver                    apiserver-kgcgc                                                   1/1       Running            0          31m
openshift-cluster-api                  clusterapi-apiserver-6b855f7bc5-s2flm                             2/2       Running            0          33m
openshift-cluster-api                  clusterapi-controllers-85f6bfd9d5-vrw2s                           2/2       Running            0          31m
openshift-cluster-api                  machine-api-operator-5d85454676-m9z9d                             1/1       Running            0          39m
openshift-cluster-version              cluster-version-operator-fqkl6                                    1/1       Running            0          41m
openshift-controller-manager           controller-manager-w9kdb                                          1/1       Running            0          30m
openshift-core-operators               openshift-cluster-openshift-apiserver-operator-5fbd49d8f7-vlthq   1/1       Running            0          39m
openshift-core-operators               openshift-cluster-openshift-controller-manager-operator-7cw26h6   1/1       Running            0          39m
openshift-core-operators               openshift-service-cert-signer-operator-6d6c6f55db-jspwh           1/1       Running            0          39m
openshift-image-registry               cluster-image-registry-operator-58c7c9bfd6-r7l92                  1/1       Running            0          29m
openshift-ingress                      tectonic-ingress-controller-operator-fcb9c6f4b-tlhzq              0/1       CrashLoopBackOff   9          31m
openshift-machine-config-operator      machine-config-controller-6948b45dd9-n8cxd                        1/1       Running            0          32m
openshift-machine-config-operator      machine-config-daemon-drsmd                                       1/1       Running            4          29m
openshift-machine-config-operator      machine-config-daemon-fsxft                                       1/1       Running            0          31m
openshift-machine-config-operator      machine-config-operator-545fcb447d-h9pfj                          1/1       Running            0          39m
openshift-machine-config-operator      machine-config-server-5dwbh                                       1/1       Running            0          31m
openshift-monitoring                   cluster-monitoring-operator-c64f5b475-tlplc                       1/1       Running            0          29m
openshift-monitoring                   prometheus-operator-5bf8644c75-xmrcm                              1/1       Running            0          20m
openshift-operator-lifecycle-manager   catalog-operator-5d5d8c7689-wf2ql                                 1/1       Running            0          39m
openshift-operator-lifecycle-manager   olm-operator-76b7f57649-8zj5v                                     1/1       Running            0          39m
openshift-operator-lifecycle-manager   package-server-f994b8699-f7hsc                                    0/1       CrashLoopBackOff   8          39m
openshift-service-cert-signer          apiservice-cabundle-injector-d4c746869-z66ks                      1/1       Running            0          32m
openshift-service-cert-signer          configmap-cabundle-injector-77bd46b-bqnd8                         1/1       Running            0          32m
openshift-service-cert-signer          service-serving-cert-signer-55fb7cc589-l6rzs                      1/1       Running            0          32m
openshift-web-console                  webconsole-86f4f55644-cl9r5                                       1/1       Running            0          20m
tectonic-system                        kube-addon-operator-784b4b6c7-fdkqj                               1/1       Running            0          32m

== web console output

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/console/\": no RBAC policy matched",
  "reason": "Forbidden",
  "details": {
    
  },
  "code": 403
}

@imranrazakhan where did you run that oc command to login?

[imranrazakhan]

@praveenkumar i just run this command on shell of VM where i installed 3.11

[root@c1-ocp ~]# oc login -u system:admin

[spadgett]

This is the old console that is being removed. The new console is behind a route. You're seeing this because the master console proxy is removed (since it's no longer needed).

I believe @deads2k was going to add a redirect to the new console from the old URL, but I'm not sure if that's in place.

[spadgett]

We think this is openshift/origin#20983.

It's a different cause.

[wking]

I believe @deads2k was going to add a redirect to the new console from the old URL, but I'm not sure if that's in place.

Do you know the new URL? A redirect would be nice, but we may be able to help a bunch of people by updating our docs.

[spadgett]

For new 4.0 installs, it is https://console-openshift-console.<default-routing-suffix>

cc @benjaminapetersen

[stlaz]

@spadgett I'm not entirely sure what you mean by default-routing-suffix. There does not seem to be anything at the following location:

# curl https://console.openshift-console.${OPENSHIFT_INSTALL_BASE_DOMAIN}
curl: (6) Could not resolve host: console.openshift-console.tt.testing

edit: this is calling it from the KVM host, not KVM guest

[spadgett]

@stlaz Look at the route in the openshift-console namespace. You can see the hostname there. By default, console uses a generated hostname.

[stlaz]

Thanks. Here's what I did:

# oc get route -n openshift-console
NAME      HOST/PORT                                                    PATH      SERVICES   PORT      TERMINATION          WILDCARD
console   console-openshift-console.router.default.svc.cluster.local             console    https     reencrypt/Redirect   None

Curling this just gives me Could not resolve hots:. I wonder which service is used to add this to the KVM host machine DNS? I am using the dnsmasq with NetworkManager solution from the guide. For the record, the following works just fine with it:

# host $OPENSHIFT_INSTALL_CLUSTER_NAME-bootstrap.$OPENSHIFT_INSTALL_BASE_DOMAIN
test1-bootstrap.tt.testing has address 192.168.126.10