When working with Kubernetes (K8s) using a kubectl proxy, or even in Azure with the az aks browse command using Azure Kubernetes Service (AKS). There are times when mysterious errors can begin to show when attempting to access and use the Kubernetes Dashboard (kubernetes-dashboard) web application. Depending on where you see the errors they could be a 503 Service Temporarily Unavailable error, a 401 Unauthorized error, or some other “cyptic” error message. This can be a bit confusing and lead you down many different paths of troubleshooting that will lead to nowhere. The error is likely an authentication issue, related to the auth cookies getting messed up.

Fix Kubernetes Dashboard Strange 401 Unauthorized, 503 Service Unavailable Errors 1 — Screenshot: Kubernetes Dashboard cryptic content display error

Is it an HTTP 503 or 401 Error?

In actuality, the error is more 401 Unauthorized than it is 503 Service Temporarily Unavailable. The reason a 503 error is returned is due to failed authorization of the request that is related to a missing or corrupt auth cookie of some kind.

Inspecting the message shown in the browser will let you see the following full message:

<html> <head><title>503 Service Temporarily Unavailable</title></head> <body bgcolor="white"> <center><h1>503 Service Temporarily Unavailable</h1></center> <hr><center>nginx/1.13.5</center> </body> </html> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page -->

The oddest part of these errors is that the main interface for the Kubernetes Dashboard will display, but the panes of content will display an error, or even a blank page. If you dig into the web browser console logging, you’ll see there are 503 Service Temporarily Unavailable error messages being returned from the server. Although, if you copy/paste the URL for the requests that are erroring into a new browser tab they will load just fine. This just adds to the potential confusion.

Fix Kubernetes Dashboard Strange 401 Unauthorized, 503 Service Unavailable Errors 2 — Screenshot: Browser Console log of 503 Errors

Alternatively, when attempting to access these URLs in a new browser tab, you may get a JSON response showing an error message as well:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "no endpoints available for service \"kubernetes-dashboard\"",
  "reason": "ServiceUnavailable",
  "code": 503
}

Fix it by Signing Out and Then Back in Again

Since the errors are a result of a missing or corrupt auth cookie on the request some how, you just need to rest that to fix the errors. Don’t worry, this is the most likely cause of the error, and there probably isn’t anything wrong with your Kubernetes cluster. Don’t panic just yet.

In the Kubernetes Dashboard UI, select the “profile” icon in the upper-right of the page, then select Sign out. Once signed out of the Kubernetes Dashboard, then sign in again and the errors should go away.

Fix Kubernetes Dashboard Strange 401 Unauthorized, 503 Service Unavailable Errors 3 — Screenshot: K8s Dashboard Sign Out menu

Fix: Sign out of the Kubernetes (K8s) Dashboard, then Sign in again. This will reset the auth cookies in the browser sessions and the errors should resolve themselves.

Over time, there may be occasions where you’ll need to sign out and back in again to make these errors go away. This is generally when you close the browser tab and reopen another and it auto signs you in again due to cookies persisting in the browser. At least it’s an easy fix when it does happen and you can keep working without troubleshooting or even redeploying the Kubernetes Dashboard, or even worse, on the Kubernetes cluster.

I hope this helps you save some time in troubleshooting!

About the Author

Chris Pietschmann

Chris Pietschmann is a Microsoft MVP, HashiCorp Ambassador, and Microsoft Certified Trainer (MCT) with 20+ years of experience designing and building Cloud & Enterprise systems. He has worked with companies of all sizes from startups to large enterprises. He has a passion for technology and sharing what he learns with others to help enable them to learn faster and be more productive.