VM Disk Space VS Cloud

Managing hard drive space in local virtual machines and cloud servers like EC2 instances is similar in concept. In this article we’ll explore adding storage space to both a virtual machine running locally using VMWare Fusion and an EC2 Ubuntu instance.

VMWare Fusion Storage

Running locally I have an Ubuntu 18 Desktop running with a 20G hard drive.

This is also reflected in the VM using df -h, shown as /dev/sda1

Filesystem      Size  Used Avail Use% Mounted on
udev            957M     0  957M   0% /dev
tmpfs           198M  1.8M  196M   1% /run
/dev/sda1        20G  5.4G   14G  29% /

First thing to notice here is that you cannot up the size of the hard drive attached to the virtual machine while its running. Shut down the VM, and go to Virtual Machine -> Hard Disk (SCSI) -> Hard Disk (SCSI) Settings and up the size of the drive, and click Apply

Notice now that even though the disk size has been increased, the operating system inside the VM needs to know about the size increase. Running df-h shows the same size:

/dev/sda1 20G 5.4G 14G 29% /

In Ubuntu Desktop you can use gpart to resize the partition to fill up all remaining space.

After resizing the volume to fill all space you can run df -h again to see the size reflected with the increase to 30G:

/dev/sda1 30G 5.8G 23G 21% /

Cloud Storage in AWS

Now that we’ve seen how storage works in local virtual machines, lets look at how to manage disk space in EC2 instances using Amazon Web Services. I’m starting off with an EC2 instance running Ubuntu with 20G storage space. When configuring the instance using the launch wizard I set the device storage size to 20G:

Running df -h on this instance shows the size of our root drive at 20G, shown as /dev/xvda1:

df -h

Filesystem      Size  Used Avail Use% Mounted on
udev            2.0G     0  2.0G   0% /dev
tmpfs           395M  740K  394M   1% /run
/dev/xvda1       20G  1.1G   19G   6% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/loop0       88M   88M     0 100% /snap/core/5328
/dev/loop1       13M   13M     0 100% /snap/amazon-ssm-agent/495
tmpfs           395M     0  395M   0% /run/user/1000

Instances in AWS run from volumes, also managed from the EC2 console. Each volume is loaded from a snapshot. When an instance is launched, at minimum a root volume is created from a snapshot of the AMI that is launched. For example in this case, we launched an EC2 instance from an Ubuntu AMI. A snapshot of the root volume fo the AMI is created, and then a volume created from that snapshot and then attached to the image.

Now that our instance is launched, AWS allows us to resize the volume that the instance is running on, similar to resizing the virtual machine drive, but there is no need to reboot the EC2 instance. Visit the volumes screen, find the volume that is attached to your EC2 instance and change the volume size.

After modifying the volume through the AWS console we need to update the instance to let the OS know there is more space available for the drive. More information on how AWS manages volumes and how to increase the size of a volume can be found here.

To resize the volume we first need to know where the volume is mounted in the OS. From our earlier command df -h we see that our root drive with 20G space is mounted at /dev/xvda1

/dev/xvda1 20G 1.1G 19G 6% /

To expand the volume to fill up the entire allocated space we use the command growpart.Note the off syntax where you seperate the 1 from the path, and also note that sudo or root access is required to run this command.

sudo growpart /dev/xvda 1

After running this command running df -h again shows us the updated size

/dev/xvda1 30G 1.1G 19G 4% /

Conclusion

After exploring resizing drives for both local virtual machines and cloud hosted AWS instances we can see that the processes are similar, and almost identical when using command line Ubuntu. The main difference being that VMWare Fusion requires us to shutdown our virtual machine before restarting where AWS allows us to modify volume size without restarting our instances. The underlying Linux concepts are the same for modifying partition size regardless of if you are running your server as a virtual machine in VMWare Fusion or in the Cloud with AWS.