‘Load Average is an age-old metric reported in various operating systems. It’s often assumed as a metric to indicate the CPU demand only. However, it is not the case. ‘Load Average’ not only indicates CPU demand, but also the I/O demand (i.e., network read/write, file read/write, disk read/write). To prove this theory, we conducted this simple case study.

Load Average Study
To validate this theory, we leveraged BuggyApp. BuggyApp is an opensource java project which can simulate various sort of performance problems. When you launch BuggyApp with following arguments, it will cause the high disk I/O operation on the host.

java -jar buggyApp.jar PROBLEM_IO

Let’s see the source code in the BuggyApp, which is causing this high disk I/O activity.

public class IODemo {
 
  public void start() {
 
	for (int counter =1; counter <= 5; ++counter) {
 
   	  // Launch 5 threads.
	  new IOThread ("fileIO-" + counter + ".txt").start();
	  System.out.println("Starting to write to fileIO-" +counter + ".txt");
	}
   }
}
 
 
public class IOThread extends Thread {
 
	public String fileName;
 
	public static final String CONTENT = 
"Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n"
+ "Hello World! We are building a simple chaos engineering product here. \n";
 
	public IOThread(String fileName) {
		this.fileName = fileName;
	}
 
	public void run() {
 
		int counter = 0;
		// Loop infinitely trying to read and close the file.
		while (true) {
 
			// Write the contents to the file.
			FileUtil.write(fileName, CONTENT);
 
			// Read the contents from the file.
			FileUtil.read(fileName);
 
			if (++counter == 1000) {
 
				System.out.println("Read & write 1000 times to " + fileName);
				counter = 0;
			}
		}
	}
}

Here you can see that BuggyApp is launching 5 ‘IOThread’ in ‘IODemo’ class. You can notice that ‘IOThread’ is going on an infinite while loop. In the loop, thread is writing the content into a file and reading the same content. It is doing these 2 steps repeatedly again and again. Writing contents and reading contents from the disk is a I/O intensive operation.

Load Average in Linux
When we ran this BuggyApp program in AWS EC2 Amazon Linux 4.9.58 instance, below is the ‘top’ command output on this host:



Fig: Top output in Linux

You can notice the Load Average to be ‘5.0’. This AWS EC2 instance has just 1 CPU only. Since it’s 1 CPU, the load on this system instance is 500%. If you aren’t sure how to read the Load Average, you can refer to this article. However, you can see that overall system’s CPU consumption is only 2.1% (i.e., 1.4% user CPU utilization + 0.7% system CPU utilization) as highlighted in the above top output. Thus, even though CPU demand is very less, still Load Average is very high. It clearly indicates that Load Average is not just an indicator of CPU demand. It indicates the I/O demand on the system as well.