Cloud Economics: Computational Load Testing and Stress Testing for Azure App Service Web Api (Performance Benchmark)

App Services, Service Fabric and Serverless Azure Function are becoming flagship computational services for Microsoft Azure. I am trying to find an economical benchmark for Computing in Azure. It is not a straight-forward task, but I am employing a simple methodology to find cost/computing rational between all three computational platforms.


Following is the highest throughput and zero error load test’s summary out 5 various load patterns.

Hourly Cost (3 x Nodes)0.305159022863118 per HourAzure Pricing
Hourly Test Count687600 Api CallsLoad Test Stats & Profile
Zero-Fault Average CPU Consumption88.234625%Performance Stats

Load Test – Detail Breakdown

Test Methodology

I ran user load of 600, 1300, 500, 500, 500, 50, 50, 50 respectively. My objective was to find Zero-Fault maximum throughput from the Service Fabric cluster.

The user load was well distributed with planned load spikes for a successful run.

Service Fabric Cluster Profile

Node Type1
Node Count3
Application Type1
ApplicationStateless Web Api (OWIN)

Machine Profile

"sku": {
  "name": "Standard_DS2_v2",
  "tier": "Standard"

Machine Configuration

CPU cores2
Memory GiB7
Local SSD GiB14
Max data disks4
Max cached disk throughput IOPS / MBps (cache size in GiB)8,000 / 64 (86)
Max uncached disk throughput: IOPS / MBps6,400 / 96
Max NICs / Network bandwidth2 / high

Microsoft’s Quote DSv2-series [as of 25th April 2017]

DSv2-series* (ACU: 210-250)

The A0 size is over-subscribed on the physical hardware. For this specific size only, other customer deployments may impact the performance of your running workload. The relative performance is outlined below as the expected baseline, subject to an approximate variability of 15 percent.

Azure Pricing [as of 25th April 2017]

  • 0.101719674287706 per Hour per Node – Based on Detail Usage Report from Azure Portal
  • 0.101719674287706 x 3 = 0.305159022863118 per Hour per Service Fabric Cluster (3xNode)

Web API Harness – Sample Code

Web Api deploy two test endpoints,

  • APIEndpoint1 – light-weight and high-volume scenario.
  • APIEndpoint2 – CPU Cruncher with complex graphics and mathematical calculations.
Generate random Guid and apply list and hash operations.
# API Endpoint - 1

var guidList = new List<dynamic>();
for (var i = 0; i < 300000; i++)
    var randomGuid = Guid.NewGuid().ToString();
    var guid = randomGuid;
    var hash = randomGuid.GetHashCode();
    var reverseGuid = Reverse(randomGuid);

for (var i = 0; i < 1000; i++)
    var randomGuid = Guid.NewGuid().ToString();
    guidList.Add(new { Guid = randomGuid, Hash = randomGuid.GetHashCode(), ReverseGuid = Reverse(randomGuid) });

// Return GuidList using JsonConvert

Accept Base64 2048×2048+300dpi High Resolution image and applying some rigorous image manipulation.

# API Endpoint - 2

ISupportedImageFormat format = new JpegFormat { Quality = 70 };
Size size = new Size(150, 0)
using (MemoryStream inStream = new MemoryStream(photoBytes))
    using (MemoryStream outStream = new MemoryStream())
        using (ImageFactory imageFactory = new ImageFactory(preserveExifData:true))
                        .Hue(180, true)

        // Converting Outstream to Base64 and add into HttpResponse.

Load Test Stats & Profile

I find TestId/2006 was the best match for comparison. Following is the summary of the test run.

Max User Load500 (No warm up or think time)
Tests/Sec191 (687,600 Tests per Hour)
Tests Failed0
Avg. Test Time (sec)0.31 (310 ms)
Avg. Http Response Size (KiB)137.21
Throughput/Sec (KiB)26257.8777 (25.6425 MiB)

Azure Metrics Query">{{your</a> resource group}}/providers/Microsoft.Compute/virtualMachineScaleSets/{{your sf node}}/providers/microsoft.insights/metrics?api-version=2016-09-01&$filter=(name.value eq 'Percentage CPU') and aggregationType eq 'Average' and startTime eq 2017-04-25T20:30:00Z and endTime eq 2017-04-25T20:50:00Z and timeGrain eq duration'PT1M'

Performance Stats

You can also find raw result set at Gist  .

TimestampAverage CPU


The views expressed on this site are personal opinions only and have no affiliation. See full disclaimerterms & conditions, and privacy policy. No obligations assumed.