EC2 Ubuntu 20.04 Software Defined Client for Dell APEX Block Storage for AWS
PowerFlex 4.5 is a versatile Software Defined Storage and supports a broad array of use cases ranging from containerized applications, virtual machines and physical bare-metal workloads as shown in the below visual.
Specifically, within AWS, PowerFlex 4.5 SDS is delivered as an APEX subscription offering called as Dell APEX Block Storage for AWS. The beast mode of this offering is designed to deliver relatively much higher performance and better TCO as compared to native block offering by AWS. Read more on the given URL. APEX Block Storage for Public Cloud | Dell USA
To weigh it in, the extreme nature of PowerFlex lies in its ability to deal with performance at scale. Talk about:
- 10 Million Transactions per second for an Oracle RAC cluster
- 7 Million Transactions per second for a SQL server in a 8U rack space
- 1 Billion Indexing events in 3 hours for Elastic stack
- 2.2 Million operations per second for MongoDB on Kubernetes
- 325K operations per second for Cassandra on Kubernetes
- 16 Petabytes on a single storage cluster
In this blog we will observe how to configure an EC2 instance running ubuntu 20.04 as a Software Defined client communicating with APEX Block over AWS (PowerFlex 4.5 running as SDS in AWS).
The net result will be that a user can consume APEX Block Storage for AWS volumes for high performance workloads running on EC2 ubuntu 20.04 instances.
The entire procedure has been automated with a Terraform configuration to ease deployment given a certain change in variables.
Values assumed for this exhibit:
- PowerFlex 4.5 with 2.8.7 version
- 5.4.0–167-generic Linux kernel used for ubuntu 20.04 OS in an EC2 instance.
Creating a base AMI for the ubuntu SDC
Step-1 Download and self-host the PowerFlex packages.
Login to Dell.com and locate the powerflex software packages.
Download the package zip named as Software_Only_Complete_4.5.0_287
Once download unzip the same and one can find another zip file named PowerFlex_4.5.0.287_SDCs_for_manual_install.zip under the extracted file set.
Upload this zip file to a self-hosted web server that you can perform a wget. The terraform template uses wget to fetch this file. Alternatively, if you are well-versed with Terraform, then use the remote file provisioner method and edit the user-data part to skip the wget.
I generally download the file and place them in a temporary site that can be retrieved via wget.
Step-2 Download and modify the Terraform template.
wget https://raw.githubusercontent.com/thecloudgarage/powerflex/main/create-ami-for-ec2-ubuntu-20-04.tfChange the following values in the below terraform template.
- AWS region name
- AMI ID for the ubuntu 20.04 LTS x86 for a specific region
- Security group and Subnet ID values as per your VPC
- MDM (Metadata manager) IP addresses for your APEX Block AWS (PowerFlex4.5) cluster
- ubuntu and root user passwords
Next, execute the terraform template.
terraform init && terraform plan && terraform apply -auto-approveThe terraform script will run for almost 15 minutes. Observed logs from the terraform creation are shown below.
At the end of the terraform apply, AMI ID is shown as an output. We will use this AMI ID to spin up a test instance and verify SDC operations.
terraform init && terraform plan && terraform apply -auto-approve
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/null...
- Finding latest version of hashicorp/time...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)
- Installing hashicorp/time v0.10.0...
- Installed hashicorp/time v0.10.0 (signed by HashiCorp)
- Installing hashicorp/aws v5.33.0...
- Installed hashicorp/aws v5.33.0 (signed by HashiCorp)
<SNIP>
aws_instance.sdc[0]: Creating...
aws_instance.sdc[0]: Still creating... [10s elapsed]
aws_instance.sdc[0]: Still creating... [20s elapsed]
aws_instance.sdc[0]: Still creating... [30s elapsed]
aws_instance.sdc[0]: Creation complete after 31s [id=i-0fa361bdc12aeb5d0]
time_sleep.wait_for_ec2_creation: Creating...
time_sleep.wait_for_ec2_creation: Still creating... [10s elapsed]
time_sleep.wait_for_ec2_creation: Still creating... [20s elapsed]
<SNIP>
time_sleep.wait_for_ec2_creation: Still creating... [9m40s elapsed]
time_sleep.wait_for_ec2_creation: Still creating... [9m50s elapsed]
time_sleep.wait_for_ec2_creation: Still creating... [10m0s elapsed]
time_sleep.wait_for_ec2_creation: Creation complete after 10m0s [id=2024-01-22T15:52:06Z]
null_resource.checkconnection: Creating...
null_resource.checkconnection: Provisioning with 'remote-exec'...
null_resource.checkconnection (remote-exec): Connecting to remote host via SSH...
null_resource.checkconnection (remote-exec): Host: 172.26.2.45
<SNIP>
null_resource.checkconnection (remote-exec): Private key: false
null_resource.checkconnection (remote-exec): Certificate: false
null_resource.checkconnection (remote-exec): SSH Agent: false
null_resource.checkconnection (remote-exec): Checking Host Key: false
null_resource.checkconnection (remote-exec): Target Platform: unix
null_resource.checkconnection (remote-exec): Connected!
null_resource.checkconnection: Creation complete after 2s [id=2772965549447851475]
time_sleep.wait_for_ec2_root_connection: Creating...
time_sleep.wait_for_ec2_root_connection: Still creating... [10s elapsed]
time_sleep.wait_for_ec2_root_connection: Still creating... [20s elapsed]
time_sleep.wait_for_ec2_root_connection: Creation complete after 20s [id=2024-01-22T15:52:28Z]
aws_ami_from_instance.create_sdc_ami: Creating...
aws_ami_from_instance.create_sdc_ami: Still creating... [10s elapsed]
aws_ami_from_instance.create_sdc_ami: Still creating... [20s elapsed]
aws_ami_from_instance.create_sdc_ami: Still creating... [30s elapsed]
aws_ami_from_instance.create_sdc_ami: Still creating... [40s elapsed]
aws_ami_from_instance.create_sdc_ami: Still creating... [50s elapsed]
<SNIP>
aws_ami_from_instance.create_sdc_ami: Still creating... [5m30s elapsed]
aws_ami_from_instance.create_sdc_ami: Creation complete after 5m36s [id=ami-0f1946d7d660272e2]
Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
Outputs:
ami-id = "ami-0f1946d7d660272e2"
instances = [
"172.26.2.45",
]We can observe the corresponding EC2 instance that was created in the AWS console.
and the AMI is also created.
Once the AMI is created, we can destroy the EC2 instance used to create it through console or CLI. Next, we will test this AMI by creating a new EC2 instance and validate the SDC operation with APEX Block Storage for AWS cluster.
Testing the AMI for SDC operations
Herein, we will launch a new instance from the AMI created. The terraform template can be downloaded from the below URL. Please change the AMI ID as per the above value.
wget https://raw.githubusercontent.com/thecloudgarage/powerflex/main/create-ec2-ubuntu-20-04-sdc-from-ami.tfCreate another terraform template, e.g. my-test-sdc-ubuntu-20–04.tf
terraform init && terraform plan && terraform apply -auto-approveOutput logs from the terraform creation.
terraform init && terraform plan && terraform apply -auto-approve
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v5.33.0...
- Installed hashicorp/aws v5.33.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
<SNIP>
+ vpc_security_group_ids = [
+ "sg-080b7c006220a6283",
]
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ instances = [
+ (known after apply),
]
aws_instance.sdc[0]: Creating...
aws_instance.sdc[0]: Still creating... [10s elapsed]
aws_instance.sdc[0]: Still creating... [20s elapsed]
aws_instance.sdc[0]: Still creating... [30s elapsed]
aws_instance.sdc[0]: Still creating... [40s elapsed]
aws_instance.sdc[0]: Creation complete after 42s [id=i-067202c5898f4adb2]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
instances = [
"172.26.2.6",
]The new EC2 instance has been created with the specified AMI.
Once the instance is created, SSH into as “root” user and run the below command to verify the SDC connectivity to the APEX Block Storage for AWS cluster.
/opt/emc/scaleio/sdc/bin/drv_cfg --query_mdmsOutput
ssh root@172.26.2.6
The authenticity of host '172.26.2.6 (172.26.2.6)' can't be established.
ED25519 key fingerprint is SHA256:Wfw0Ikc3N2pMrXfo0Lur39WvrapIBhOMTwcIMhV97o4.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '172.26.2.6' (ED25519) to the list of known hosts.
root@172.26.2.6's password:
Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-167-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Mon Jan 22 16:30:34 UTC 2024
System load: 0.07 Processes: 141
Usage of /: 64.3% of 7.57GB Users logged in: 0
Memory usage: 25% IPv4 address for eth0: 172.26.2.6
Swap usage: 0%
Expanded Security Maintenance for Applications is not enabled.
71 updates can be applied immediately.
53 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
root@ip-172-26-2-6:~# /opt/emc/scaleio/sdc/bin/drv_cfg --query_mdms
Retrieved 1 mdm(s)
MDM-ID 2998b596ca90370f SDC ID 21f3595700000004 INSTALLATION ID 673742d842485111 IPs [0]-10.204.111.85 [1]-10.204.111.86 [2]-10.204.111.87
root@ip-172-26-2-6:~#We can observe that the command has retrieved the list of MetaData Manager IP addresses, which means the SDC connectivity has been established with the APEX Block Storage for AWS (PowerFlex 4.5) cluster.
We can also verify this in the PowerFlex 4.5 manager web console.
At this stage there are no volumes associated with this SDC client. Let’s create a test volume.
We will then map the created volume to the SDC client using the mapping action as shown below
Now we are all set to consume block storage volumes on our EC2 instance using Dell APEX Block on AWS.
We can observe the volume map in the EC2 SDC instance
root@ip-172-26-2-6:~# /opt/emc/scaleio/sdc/bin/drv_cfg --query_vols
Retrieved 1 volume(s)
VOL-ID 64f9770200000006 MDM-ID 2998b596ca90370f
root@ip-172-26-2-6:~#The 8G volume is named as “scinia” as seen in the below output
root@ip-172-26-2-6:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 24.9M 1 loop /snap/amazon-ssm-agent/7628
loop1 7:1 0 55.7M 1 loop /snap/core18/2790
loop2 7:2 0 63.5M 1 loop /snap/core20/2015
loop3 7:3 0 91.9M 1 loop /snap/lxd/24061
loop4 7:4 0 40.9M 1 loop /snap/snapd/20290
xvda 202:0 0 8G 0 disk
├─xvda1 202:1 0 7.9G 0 part /
├─xvda14 202:14 0 4M 0 part
└─xvda15 202:15 0 106M 0 part /boot/efi
scinia 252:0 0 8G 0 disk
root@ip-172-26-2-6:~#
root@ip-172-26-2-6:~#
root@ip-172-26-2-6:~#We can start using it via typical mounting process. I am just using a temporary mount here, one can extend it as per their preferred method.
root@ip-172-26-2-6:~# mkdir -p /data
root@ip-172-26-2-6:~# mkfs -t ext4 /dev/scinia
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 7ce8de86-4367-47ba-8cc2-343c93a63fe4
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
root@ip-172-26-2-6:~# mount /dev/scinia /data
root@ip-172-26-2-6:~# cd /data
root@ip-172-26-2-6:/data# touch 1.txt 2.txt 3.txt
root@ip-172-26-2-6:/data# ll
total 24
drwxr-xr-x 3 root root 4096 Jan 22 16:45 ./
drwxr-xr-x 20 root root 4096 Jan 22 16:44 ../
-rw-r--r-- 1 root root 0 Jan 22 16:45 1.txt
-rw-r--r-- 1 root root 0 Jan 22 16:45 2.txt
-rw-r--r-- 1 root root 0 Jan 22 16:45 3.txt
drwx------ 2 root root 16384 Jan 22 16:44 lost+found/
root@ip-172-26-2-6:/data#That’s it! Hoping that this read gave you the practical insight of consuming volumes on ubuntu 20.04 EC2 instances from APEX Block Storage for AWS cluster.
Thanks to my colleague Alexander Hoppe within the Software Engineering team to help me overcome some of the nuances within this implementation.
Happy powerflex’ing
cheers,
Ambar@thecloudgarage
#iwork4dell
Additional Information
The default kernel version in EC2 based ubuntu instances is a custom one. We need to change this kernel version before the SDC installation. The process is entirely automated in the above terraform template. The below is just an illustration on how these SDC versions are organized and stored in Dell’s FTP repository as per OS type and kernel versions. FTP into ftp.emc.com with the given user-id and password. This is the site which our scripts automatically fetch the scini.tar file for a specific ubuntu OS and kernel version to configure the SDC. The above terraform template can easily be modified to suit a similar installation for ubuntu 22.04 OS.
ftp://QNzgdxXix:Aw3wFAwAq3@ftp.emc.com/
