Understanding Queue Depth, IOPS and Latency

Under the storage section on your instances activity page you will see IOPs, and Queue length.

Input/output operations per second (IOPS) is an input/output performance measurement used to characterize computer storage devices like hard disk drives (HDD), solid state drives (SSD), and storage area networks (SAN).

Queue Depth (queue length) is the number of I/O requests (SCSI commands) that can be queued at one time on a storage controller. Each I/O request from the host's initiator HBA to the storage controller's target adapter consumes a queue entry.  (Note: this number will vary as it shows how much your system is maxing at that moment)

Calculating your IO rate:

Latency is a measure of how long it takes for a single I/O request to happen from the application’s viewpoint.

For example: If your charts show that IOPS are at 160,000 then that's 160k IO occurring in 1 second which makes 1 IO 0.00000625 of a second.

Using Centos with FIO Marketplace template to determine Latency: [The best test to run is a Direct I/O which means an IO access where you bypass the cache (as buffered can be misleading)]

You will need to stop the automatic FIO that is running and manually run it. 

  1. Download the Centos with FIO template from the marketplace and open it
  2. Run -  systemctl stop runfio
  3. To check what the parameters are set at, you can run - Cat rw_test.fio 
  4. Run - /usr/bin/fio /root/rw_test.fio
  5. To stop the test at any time you can "ctrl C"

The test will look similar to the example below. Here you can see the same information provided on the activity page, including in depth details of your environment.


Key sections of the results and how to read them:

'Read and Write' 

The first line of the 'read' or 'write' sections state the IO requests, bandwidth, iops, and run time that occurred within the duration of the test.

[In the above example, fio did a total of 43GB of IO at 167.2MB/s for a total of 41796 IOPS, and ran for 4 minutes and 39 seconds.]

'slat' or submission latency (measured in usec) measures how long it took to submit this IO to the kernel for processing

'clat' or completion latency (measured in usec) shows the time between submitting a request and it being completed. 

Below the Read and Write sections you will find:

'lat' or latency distribution which shows how long a certain percentage of IO requests took.

[In the above example it is saying that 0.01% of requests took 100 microseconds, 0.01% of requests took 250 microseconds, 0.79% of requests took 500 microseconds, and so on.]

'IO depths' shows the amount of IO depths reached at what percentage of the time.

'Submit and complete' represent the number of submitted IOs at a time by fio and the number completed at a time.







Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request