Open in app

Sign in

Write

Sign in

EKS Anywhere, OIDC integration with Azure-AD/Microsoft-Entra

Ambar Hassani
9 min readMay 13, 2024

This article is part of the series EKS Anywhere, extending the Hybrid cloud momentum | by Ambar Hassani | Apr, 2022 | Medium

Many customers use Microsoft Azure Active Directory (Azure AD) as their Federated Identity Provider. One of the common scenarios in Kubernetes access is to integrate the cluster access via their Azure AD, or what is now re-branded to Microsoft Entra. Combined with Kubernetes RBAC, this approach enables use of existing authentication for corporate users to manage the EKS Anywhere clusters. In this blog, we will see a practical implementation of the same.

Creating OIDC setup in Azure-AD along with Users and Groups.

Access the Microsoft Entra ID service from the Azure portal

Register and configure a new application with Azure AD tenant

At the end of this step, you will note down the Application (client) ID and the Directory (tenant) ID. Both these values are important for the OIDC authentication flow.

Toggle the client flow to public in the authentication item within navbar. A public client application doesn’t require a certificate or secret as part of the authentication process. Hence users don’t need to store any secrets or certificates in their local environment to access the EKS Anywhere cluster

Make sure that Read permission for Microsoft Graph API is in place. If not, setup permission for Microsoft Graph by choosing the Add a permission button. Select Grant admin consent for Default Directory. This allows the application to grant administrative consent to read the user profile and perform authentication.

By default, a user’s group information isn’t included as part of the OIDC token returned by Azure AD. So we will include groups claim as well as a part of the OIDC token flow.

Next, we will create three groups that will align with the Kubernetes RBAC model that we will implement later. The group names will be:

  • level1
  • level2
  • level3

Take note of the Object IDs created for the groups. These will be used while creating the RBAC statements within the EKS-Anywhere cluster

Next we will create three users and assign them to the respective groups such as

  • user eksa-kube-view > level1 group
  • user eksa-kube-view > level2 group
  • user eksa-kube-view > level3 group

Repeat the above steps to create the other two users, i.e. eksa-kube-edit (level2 group) and eksa-kube-admin (level3 group)

If you have not., then make a note of the application ID and the Tenant ID

Create the EKS-Anywhere cluster or update an existing cluster with OIDC configurations. Below is a snip of the important statements required in the Cluster configuration YAML. Note the usage of Application ID and Tenant ID retrieved in the previous step.

apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: testcluster-1
  namespace: default
spec:
<snip>
  identityProviderRefs:
  - kind: OIDCConfig
    name: azure-ad
<snip>
---
<snip>
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: OIDCConfig
metadata:
  name: azure-ad
  namespace: default
spec:
  clientId: <application-id-from-azure-portal>
  groupsClaim: groups
  issuerUrl: https://sts.windows.net/<azure-tenant-id/
  usernameClaim: upn

---
<snip>

Next, we will configure the RBAC within our EKS-Anywhere cluster. Note the use of the group Object ID as a value for each group that we created in the previous steps

cat <<EOF | kubectl create -f -
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: level3
subjects:
- kind: Group
  name: eb83d25e-6d06-43ae-acb2-f41d2f2250ed
  apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: level2
subjects:
- kind: Group
  name: 9215db65-d22f-4f9c-ae89-e40474bfb63d
  apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: edit
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: level1
subjects:
- kind: Group
  name: a7d185f0-39a5-4e47-bb0f-4ca75a467bff
  apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
EOF

Let’s now verify the access from a Windows host. We will be using azure-kubelogin extension for kubectl

To do so, let’s install Chocolatey if it’s not already provisioned in your windows environment.

cd C:\Users\Administrator

@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

Next, with Chocolatey, let’s install azure-kubelogin

choco install kubernetes-cli azure-kubelogin

The logs for the same are provided below

Chocolatey v2.2.2
Installing the following packages:
kubernetes-cli;azure-kubelogin
By installing, you accept licenses for the packages.
Progress: Downloading kubernetes-cli 1.30.0... 100%

kubernetes-cli v1.30.0 [Approved]
kubernetes-cli package files install completed. Performing other installation steps.
The package kubernetes-cli wants to run 'chocolateyInstall.ps1'.
Note: If you don't run this script, the installation will fail.
Note: To confirm automatically next time, use '-y' or consider:
choco feature enable -n allowGlobalConfirmation
Do you want to run the script?([Y]es/[A]ll - yes to all/[N]o/[P]rint): A

Extracting 64-bit C:\ProgramData\chocolatey\lib\kubernetes-cli\tools\kubernetes-client-windows-amd64.tar.gz to C:\ProgramData\chocolatey\lib\kubernetes-cli\tools...
C:\ProgramData\chocolatey\lib\kubernetes-cli\tools
Extracting 64-bit C:\ProgramData\chocolatey\lib\kubernetes-cli\tools\kubernetes-client-windows-amd64.tar to C:\ProgramData\chocolatey\lib\kubernetes-cli\tools...
C:\ProgramData\chocolatey\lib\kubernetes-cli\tools
 ShimGen has successfully created a shim for kubectl-convert.exe
 ShimGen has successfully created a shim for kubectl.exe
 The install of kubernetes-cli was successful.
  Software installed to 'C:\ProgramData\chocolatey\lib\kubernetes-cli\tools'
Progress: Downloading azure-kubelogin 0.1.3... 100%

azure-kubelogin v0.1.3 [Approved]
azure-kubelogin package files install completed. Performing other installation steps.
Downloading azure-kubelogin 64 bit
  from 'https://github.com/Azure/kubelogin/releases/download/v0.1.3/kubelogin-win-amd64.zip'
Progress: 100% - Completed download of C:\Users\Administrator\AppData\Local\Temp\1\chocolatey\azure-kubelogin\0.1.3\kubelogin-win-amd64.zip (22.67 MB).
Download of kubelogin-win-amd64.zip (22.67 MB) completed.
Hashes match.
Extracting C:\Users\Administrator\AppData\Local\Temp\1\chocolatey\azure-kubelogin\0.1.3\kubelogin-win-amd64.zip to C:\ProgramData\chocolatey\lib\azure-kubelogin\tools...
C:\ProgramData\chocolatey\lib\azure-kubelogin\tools
 ShimGen has successfully created a shim for kubelogin.exe
 The install of azure-kubelogin was successful.
  Software installed to 'C:\ProgramData\chocolatey\lib\azure-kubelogin\tools'

Chocolatey installed 2/2 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Next, we will create a default kubeconfig with our EKS Anywhere cluster endpoint and generate a context for the same named as “azure-user”. We will also configure the credentials for “azure-user” via kubelogin extension

cd C:\Users\Administrator or cd %userprofile%
mkdir .kube

SET EKSA_CLUSTER_NAME=testcluster-1
SET EKSA_CLUSTER_API_ENDPOINT=https://10.204.111.55:6443
SET AZURE_AD_APP_CLIENT_ID=<application-id-from-azure-portal>
SET AZURE_AD_TENANT_ID=<azure-tenant-id>

kubectl config --kubeconfig=%userprofile%\.kube\config set-cluster %EKSA_CLUSTER_NAME% --server=%EKSA_CLUSTER_API_ENDPOINT% --insecure-skip-tls-verify

kubectl config set-credentials "azure-user" --exec-api-version=client.authentication.k8s.io/v1beta1 --exec-command=kubelogin --exec-arg=get-token  --exec-arg=--environment --exec-arg=AzurePublicCloud --exec-arg=--server-id --exec-arg=%AZURE_AD_APP_CLIENT_ID% --exec-arg=--client-id --exec-arg=%AZURE_AD_APP_CLIENT_ID% --exec-arg=--tenant-id --exec-arg=%AZURE_AD_TENANT_ID%

kubectl config set-context azure-user --user=azure-user --cluster=%EKSA_CLUSTER_NAME%
kubectl config use-context azure-user

Time to verify

kubectl --user=azure-user get nodes
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code H46GJ6XDN to authenticate

We are redirected to the Microsoft portal for authentication. Provide the code from the above command and login with the user (principal name). In my case, I am testing the admin user created as eksa-kube-admin@ambaru2ubegmail.onmicrosoft.com

Upon first login, you will asked to change the password. Follow the instructions, which should finally result into a successful OIDC auth in case everything has been configured correctly.

Upon successful authentication, the terminal output automatically results in the output as seen below. Once authenticated we are able to view the cluster nodes and also create Kubernetes resources as the permissions model applied to this group within the RBAC is a cluster-admin

kubectl --user=azure-user get nodes
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code H46GJ6XDN to authenticate.
NAME                                    STATUS   ROLES           AGE     VERSION
testcluster-1-6k2r5                        Ready    control-plane   7h45m   v1.25.11-eks-984f31e
testcluster-1-bwbc7                        Ready    control-plane   7h48m   v1.25.11-eks-984f31e
testcluster-1-md-0-cd84b894cxdwx8h-bq56b   Ready    <none>          7h49m   v1.25.11-eks-984f31e
testcluster-1-md-0-cd84b894cxdwx8h-k86qk   Ready    <none>          7h49m   v1.25.11-eks-984f31e
testcluster-1-mvbbz                        Ready    control-plane   7h50m   v1.25.11-eks-984f31e


kubectl --user=azure-user create ns oidc-test-1
namespace/oidc-test-1 created

To test another user

kubelogin remove-tokens

Repeat the procedure to re-login

kubectl --user=azure-user get nodes
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code IZGUF7KKG to authenticate.

Provide eksa-kube-edit@ambaru2ubegmail.onmicrosoft.com as the username along with the password flow (after initial change).

One can observe that although the OIDC authentication is successful, the user eksa-kube-edit@ambaru2ubegmail.onmicrosoft.comR being a part of level2 group is mapped via RBAC model to default cluster-edit role which does not allow it to view nodes. At the same token, the role bears the permission to view namespaces.

kubectl --user=azure-user get nodes
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code IZGUF7KKG to authenticate.
Error from server (Forbidden): nodes is forbidden: User "https://sts.windows.net/f63c9c1c-df22-4d17-b9a8-079c88523049/#eksa-kube-edit@ambaru2ubegmail.onmicrosoft.com" cannot list resource "nodes" in API group "" at the cluster scope
kubectl --user=azure-user get ns
NAME                                STATUS   AGE
capi-kubeadm-bootstrap-system       Active   7h51m
capi-kubeadm-control-plane-system   Active   7h51m
capi-system                         Active   7h51m
capv-system                         Active   7h51m
cert-manager                        Active   7h51m
default                             Active   7h57m
eksa-packages                       Active   7h49m
eksa-packages-testcluster-1         Active   7h49m
eksa-system                         Active   7h51m
etcdadm-bootstrap-provider-system   Active   7h51m
etcdadm-controller-system           Active   7h51m
kube-node-lease                     Active   7h57m
kube-public                         Active   7h57m
kube-system                         Active   7h57m
oidc-test-1                         Active   0h10m
kubectl --user=azure-user create ns oidc-test-2
Error from server (Forbidden): namespaces is forbidden: User "https://sts.windows.net/f63c9c1c-df22-4d17-b9a8-079c88523049/#eksa-kube-edit@ambaru2ubegmail.onmicrosoft.com" cannot create resource "namespaces" in API group "" at the cluster scope

Similarly, one can test out the third user with view-only privileges.

Hopefully, this demonstration provides insights into how Azure-AD/Microsoft Entra can be used as a Identity Provider for EKS Anywhere clusters.

cheers,

Ambar@thecloudgarage

#iwork4dell

AWS
Eks Anywhere
Openid
Microsoft Entra Id
Azure Ad

--

--

Ambar Hassani

Written by Ambar Hassani

77 Followers

24+ years of blended experience of technology & people leadership, startup management and disruptive acceleration/adoption of next-gen technologies

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams