VMs can’t start if a node contains more than ~60 VMs

Ref: https://github.com/harvester/harvester/issues/2722

image

Verify Steps:

  1. Install Harvester with any nodes
  2. Login to console, execute sysctl -a | grep aio, the value of fs.aio-max-nr should be 1048576
  3. Update the value by executing:
mkdir -p /usr/local/lib/sysctl.d/
cat > /usr/local/lib/sysctl.d/harvester.conf <<EOF
fs.aio-max-nr = 61440
EOF
sysctl --system
  1. Execute sysctl -a | grep aio, the value of fs.aio-max-nr should be 61440
  2. Reboot the node then execute sysctl -a | grep aio, the value of fs.aio-max-nr should still be 61440
  3. Create an image for VM creation
  4. Create 60 VMs and schedule on the node which updated fs.aio-max-nr
  5. Update fs.aio-max-nr to 1048576 in /usr/local/lib/sysctl.d/harvester.conf and execute sysctl --system
  6. VMs should started successfully or Stopping with error message Too many pods