Omni Documentation
Try OmniTalos Linux
  • Omni Documentation
  • Omni Support Matrix
  • Tutorials
    • Getting Started with Omni
    • Upgrading Omni Clusters
    • Installing Airgapped Omni
    • Using SAML and ACLs for fine-grained access control
    • Setting Up the Bare-Metal Infrastructure Provider
  • How-to guides
    • Using SAML with Omni
      • Add a User to Omni with SAML Enabled
      • Auto-assign roles to SAML users
      • Configure Workspace ONE Access for Omni
      • Configure Okta for Omni
      • Configure Entra ID AD for Omni
      • Configure Unifi Identity Enterprise for Omni
    • Register machines with Omni
      • Register a Bare Metal Machine (ISO)
      • Register a Bare Metal Machine (PXE/iPXE)
      • Register an AWS EC2 Instance
      • Register an Azure Instance
      • Register a GCP Instance
      • Register a Hetzner Server
    • Create a Cluster
    • Install talosctl
    • Install and Configure Omnictl
    • Use Kubectl With Omni
    • Create a Kubeconfig for a Kubernetes Service Account
    • Create a Patch For Cluster Machines
    • Manage Access Policies (ACLs)
    • Create a Hybrid Cluster
    • Run Omni on your own infrastructure
      • Deploy Omni On-prem
      • Configure Keycloak for Omni
      • Back Up On-prem Omni Database
      • How to expose Omni with Nginx (HTTPS)
    • Install Talos Linux Extensions
    • Scale a Cluster Up or Down
    • Etcd backups
    • Restore Etcd of a Cluster Managed by Cluster Templates
    • Create an Omni Service Account
    • Create a Machine Class
    • Expose an HTTP Service from a Cluster
    • Export a Cluster Template from a Cluster Created in the UI
    • Audit logs
    • Set Initial Machine Labels Using Omnictl or Image Factory
  • Explanation
    • Machine Registration
    • Authentication and Authorization
    • Omni KMS Disk Encryption
    • Infrastructure Providers
  • Reference
    • omnictl CLI
    • Access Policies (ACLs)
    • Generating omnictl CLI reference
    • Cluster Templates
Powered by GitBook
On this page
  • Switching between users when authenticating to the same cluster
  • OIDC authentication over SSH
Edit on GitHub
Export as PDF
  1. How-to guides

Use Kubectl With Omni

PreviousInstall and Configure OmnictlNextCreate a Kubeconfig for a Kubernetes Service Account

Last updated 2 months ago

With an Omni managed cluster, you use kubectl as with any other Kubernetes cluster, with the caveat that you must use the kubeconfig file that you download from Omni, and have the OIDC plug-in installed for your kubectl.

All Kubernetes kubectl commands are routed through the API endpoint created by Omni, and Omni validates access through the configured OIDC provider or other user authorization mechanism. This ensures your Kubernetes cluster is safe - unlike other systems, mere possession of a kubeconfig grants no access - the user also has be valid in the configured authentication system of Omni.

Download the KubeConfig file

Navigate to the clusters page by clicking on the “Clusters” button in the sidebar.

Click on the cluster and then click "Download kubeconfig" from the cluster dashboard on the right. The downloaded file will reflect the name of the cluster.

Install the OIDC plug in

Install the oidc-login plugin per the official documentation:

Access the cluster with kubectl

kubectl --kubeconfig ./talos-default-kubeconfig.yaml get nodes

Be sure you use the name of the downloaded kubeconfig file, which will vary with the name of the cluster.

The first time you use the kubectl command to query a cluster, a browser window will open requiring you to authenticate with your identity provider.

If you get a message error: unknown command "oidc-login" for "kubectl" Unable to connect to the server then you need to install the oidc-login plugin as noted above, and ensure it is in your $PATH.

Authentication for omnictl, talosctl, and kubectlwill last for 8 hours. After 8 hours you will need to re-authenticate each tool for your environment.

Switching between users when authenticating to the same cluster

If you have multiple contexts in your kubeconfig(s) authenticating to the same cluster (in the same Omni instance), switching between these contexts does not switch the authenticated user. This is a of the OIDC-based login we use - it uses the existing logged-in user, as their auth token is the already cached for that cluster.

To work around that, when you want to switch to another user, you need to clear the authentication cache first. To do this, run one of the following:

kubectl oidc-login clean # OR
rm -rf "${KUBECACHEDIR:-$HOME/.kube/cache}/oidc-login"

After doing this, the next kubectl command you run should trigger the OIDC login flow again, where you can authenticate as the user you need via Switch User option.

OIDC authentication over SSH

If you need to use kubectl, talosctl, or omnictl on a remote host over SSH you may need a way to forward your local client traffic to the remote host where kubectl-oidc_login is installed.

To do that you can tunnel the ports over SSH when you connect to the host. This command will open a tunnel using the default ports oidc-login attempts to use.

ssh -L 8000:localhost:8000 -L 18000:localhost:18000 $HOST

You can run this in a separate terminal temporarily while you authenticate your CLI tools.

If you want to have the port forwarding happen automatically every time you connect to the host you should update your ~/.ssh/config file to contain the following lines for the host.

Host myhost
  LocalForward 8000 127.0.0.1:8000
  LocalForward 18000 127.0.0.1:18000

You will also need to disable automatic browser opening because it will likely try to open a browser on the SSH host or fail to open if one is not installed. Do that by adding --skip-open-browser in your $KUBECONFIG file.

      args:
        - oidc-login
        - get-token
        - --oidc-issuer-url=https://$YOUR_ENDPOINT.omni.siderolabs.io/oidc
        - --oidc-client-id=native
        - --oidc-extra-scope=cluster:not-eks
        - --skip-open-browser
      command: kubectl
      env: null
https://github.com/int128/kubelogin#getting-started
known limitation