Own Virtualisation Infrastructure

Executive summary

This portfolio documents the deployment of a self-managed virtualisation platform using KVM/QEMU on Ubuntu Server across two physically separated hosts: MTN Nigeria DC, Ojota (Lagos) — primary on-premises node with physical access, and Rimu Hosting (UK) — remote replication node provisioned with root access. VM disk images are stored on a GlusterFS replicated volume over an encrypted site-to-site VPN (pfSense). Core services include web & mail hosting (control panel), NAS, Vyatta routing, pfSense firewall, and a centralised Icinga2 monitoring stack. Routine maintenance is performed during off-hours and 24/7 on-call support is provided.

Key objectives

• Achieve geographical redundancy and fault tolerance for VM disk images via GlusterFS replication across Lagos and UK nodes.
• Host essential services (web, mail, NAS, enterprise apps) as virtual machines on KVM.
• Secure cross-site replication using an encrypted site-to-site VPN (pfSense).
• Implement central monitoring and alerting with Icinga2 and provide 24/7 on-call operations.
• Document repeatable deployment steps suitable for automation (future Ansible playbooks).

High-level architecture

The architecture comprises two primary sites connected by an encrypted tunnel: the Lagos (primary) KVM host and the UK (replica) KVM host. Each host runs Ubuntu Server LTS, KVM/libvirt, GlusterFS bricks and a pfSense VM for the VPN endpoint. VMs are stored on a GlusterFS replicated volume mounted on the libvirt storage directory. Icinga2 runs as the central monitoring master on Lagos with remote checks for the UK node.

Hardware & software

Deployment summary

1. Installed Ubuntu Server on both bare-metal hosts; applied basic hardening and SSH key-based authentication.
2. Installed and configured KVM/libvirt, set up network bridges for VM connectivity.
3. Provisioned pfSense VMs on both sites and established an encrypted site-to-site VPN (IPsec/OpenVPN as required).
4. Installed GlusterFS on both hosts; created a replicated Gluster volume for VM images and started the volume.
5. Mounted the Gluster volume on both hosts as the libvirt storage pool.
6. Deployed VMs (Linux and Windows Server) for web, mail, NAS, Vyatta, and enterprise applications with images stored on the Gluster volume.
7. Installed and configured Icinga2 for host and service monitoring; configured notification channels and escalation runbooks.
8. Tested replication, failover and routine maintenance procedures; scheduled off-hours maintenance windows.

Security, backups and operations

Security is implemented in layers: host hardening & SSH key-only access; pfSense-managed firewall rules and VPN encryption for replication traffic; VM-level security for services (TLS/SSL, secure mail setup). Backups include regular VM snapshots, database dumps sent to an offsite backup repository and periodic Gluster heal checks. An on-call rota ensures 24/7 response to Icinga2 alerts.

• Management interfaces restricted to a management VLAN and bastion hosts.
• Automatic snapshots before major updates; snapshot retention policy defined.
• Encrypted off-site backups of critical data and configuration backups of network appliances.
• Patch & maintenance schedule executed during agreed off-hours.

Recommendations

• Automate deployments and configuration management using Ansible to ensure repeatability and faster recovery.
• Consider Ceph for larger-scale storage and higher resilience (three or more nodes) if capacity grows.
• Add a third geographically-dispersed node for Gluster quorum and improved failover behaviour.
• Instrument detailed telemetry (I/O, latency, replication metrics) and visualise trends to inform scaling decisions.

Concise operational checklist

• Check Gluster peer & volume status: gluster peer status, gluster volume status.
• Verify volume heal information: gluster volume heal gv0 info.
• Confirm VM state: virsh list --all and virt-manager connection.
• Ensure Icinga2 is running: systemctl status icinga2.
• Run snapshot before major updates and test restore on a staging host.
• Validate VPN connectivity and throughput for replication windows.

Network & Service Topology

GlusterFS replication topology

The production configuration uses a replica 2 GlusterFS volume with bricks on both Lagos and UK nodes. Quorum considerations and tie-breaker or arbiter strategies are noted for production-grade deployments (see full report).

Selected concise configuration snippets

Install KVM, libvirt and virt-manager (hosts and desktop)

# On both Ubuntu Server hosts (LTS)
sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virtinst
sudo systemctl enable --now libvirtd

# On the network engineer's Ubuntu Desktop (for GUI management)
sudo apt update
sudo apt install -y virt-manager

# To allow remote libvirt connections from the desktop to the host:
# 1) Configure libvirtd on the host to listen (edit /etc/libvirt/libvirtd.conf):
#    listen_tls = 0
#    listen_tcp = 1
#    auth_tcp = "none"   # prefer sasl or polkit; use SSH tunnel for security
# 2) Restart libvirt:
sudo systemctl restart libvirtd

# Recommended secure method: SSH tunnel from desktop:
# From desktop: establish SSH tunnel and connect virt-manager to qemu+ssh://user@host/system
ssh -fNL 16509:localhost:16509 user@host.mioemi.com -p 22
# Then in virt-manager, open connection: qemu+tcp://localhost/system (or use qemu+ssh)

Basic GlusterFS (peer probe & create replicated volume)

# Install GlusterFS on both hosts
sudo apt update
sudo apt install -y glusterfs-server
sudo systemctl enable --now glusterd

# On Lagos host:
sudo gluster peer probe uk-host.mioemi.com

# Create replicated volume (example)
sudo mkdir -p /srv/gluster/gv0/brick
sudo mkdir -p /srv/gluster/gv0/brick

sudo gluster volume create gv0 replica 2 transport tcp \
  lagos.mioemi.com:/srv/gluster/gv0/brick \
  uk.mioemi.com:/srv/gluster/gv0/brick

sudo gluster volume start gv0
sudo gluster volume status gv0

Mount Gluster volume on host for libvirt storage

# Create mount point and mount (on each host)
sudo mkdir -p /var/lib/libvirt/images
sudo mount -t glusterfs lagos.mioemi.com:/gv0 /var/lib/libvirt/images

# Persist in /etc/fstab (example):
lagos.mioemi.com:/gv0  /var/lib/libvirt/images  glusterfs  defaults,_netdev  0  0

Create a VM using virt-install (CLI) and using virt-manager (GUI)

# CLI example: create a Linux VM with virt-install
virt-install \
  --name web01 \
  --ram 8192 \
  --vcpus 2 \
  --disk path=/var/lib/libvirt/images/web01.qcow2,size=60,format=qcow2 \
  --os-type linux --os-variant ubuntu22.04 \
  --network bridge=br0 \
  --graphics none \
  --console pty,target_type=serial \
  --cdrom /path/to/ubuntu-server.iso

# GUI: open virt-manager on the engineer desktop, connect via SSH or direct libvirt endpoint,
# choose 'New VM', select installation method (ISO), allocate CPU/RAM/disk on the Gluster-mounted pool.

Short summary

This summarises my deployment of a two-site KVM virtualisation deployment with GlusterFS replication, secure VPN connectivity, centralised monitoring and operational processes. It is intended for review prior to producing the full technical report containing exhaustive configuration files, test results and automation artefacts.