In this series of simulating and troubleshooting performance problems in Scala articles, let’s discuss how to simulate CPU consumption to spike up to 100% on a host (or container) and troubleshoot it.

Scala sample program
CPU consumption will spike up whenever a thread goes on an infinite loop. Here is a sample Scala program, which generates the CPU spike:

package com.yc
class CPUSpikeApp {
}
object CPUSpikeApp {
   def main(args: Array[String]): Unit = {
      new AppThread().start()
      new AppThread().start()
      new AppThread().start()
      new AppThread().start()
      new AppThread().start()
      new AppThread().start()
      println("6 threads launched!")
   }
   def action(): Unit = {
   }
   class AppThread extends Thread {
      override def run(): Unit = {
         while (true)
         CPUSpikeApp.action()
      }
   }
}

When a thread goes on an infinite loop, the CPU will start to spike up. In the above program, we have set up 6 threads to go on an infinite loop. Thus when the program is executed, CPU consumption will skyrocket on the machine.

In the program you will notice the ‘CPUSpikeApp’ class, which launches the ‘AppThread’ 6 times. Within the ‘AppThread’ class, there is a ‘while (true)’ loop, which invokes ‘CPUSpikeApp.action()’ method. The action() method has no code in it, which will cause the thread to go on an infinite loop, ultimately causing the CPU to spike up on the machine.

How to troubleshoot Scala CPU Spike?
You can diagnose CPU spike problems in scala either through a manual or automated approach.

Manual approach:
As highlighted in this article, you can use manual approach to troubleshoot :

Capture thread dump from the application
Capture ‘top -H -p {PID}’ output
Marry these #a and #b and identify the root cause of the CPU spike problem

Automated approach:
On the other hand, you can also use yCrash open source script, which would capture 360-degree data (GC log, 3 snapshots of thread dump, heap dump, netstat, iostat, vmstat, top, top -H,…) from your application stack within a minute and generate a bundle zip file. You can then either manually analyze these artifacts or upload it to yCrash server for automated analysis.

We used the automated approach. Once the captured artifacts were uploaded to the yCrash server, it instantly generated the below root cause analysis report highlighting the source of the problem.



Fig: yCrash tool point out lines of code causing the CPU spike

From the above screenshot, you can see the yCrash tool pointing out the 6 threads causing the CPU to spike up: Thread-1, Thread-4, Thread-6, Thread-5, Thread-2, Thread-3. It also reports the CPU consumption by each thread. These are the 6 threads that were created in the above Scala program.

When you click on the thread names in the report, you will see the stack traces of those threads. You can also notice that the tool is pointing out exact lines of code i.e., com.yc.CPUSpikeApp$AppThread.run(cpu.scala:33), where the threads are infinitely looping & causing the CPU to spike up. Equipped with this information one can easily go ahead and fix the CPU spike problem.