Mobile Performance Test

NSW Government Agency

Case Study – NSW Government Agency

 

Background:

A NSW government agency had a requirement to performance test a new version of their emergency fires/hazards mobile in readiness for the fires season. The main performance requirement was to simulate the performance of the backend infrastructure for a worse case scenario and also for resiliency against 25 million watchzones within all the states in Australia and to create 25 million notifications in under 30 minutes or as near to that target.  The backend infrastructure was on AWS using Lambda for serverless, SQS queues and SNS for notifications. S3 buckets were there to store the feeds using the CAP-AU  standard format from the various Jurisdictions.  Spatial data was created via R to simulate the creation of watchzone based on Geolocation data which form the input into Golang script that used to create the 25 million watchzones for faster processing within the infrastructure. Jmeter was then used to create the load scripts and then to execute the performance test.  Several bottlenecks were found in the early cycles and recommendations were given to the vendor to improve on performance to reach the acceptable NFR.  Some design constraints were found as part of the performance testing. Other tests that were performed were resilient tests to ensure redundancy to ensure notifications don’t get lost and can be resubmitted if they go to a specific retransmission queue. 

Problem Statements:

5

Due to issues with current mobile emergency during black summer (2019) there were number of inefficiencies found with performance and notification management thus a new app was created. A key requirement was to ensure that performance and the inefficiencies were thoroughly tested

5

The client’s documentation was not sufficient and a new knowledgebase was created around functional and non functional testing

5

No baselines for performance testing existed and performance scripts did not exist

Client outcomes:

After multiple NFT tests the NFRs were met and signed off for production release candidate

Through the NFT performance testing it also confirmed that components within the system could scale to handle even bigger loads

Activities:

=

TL Consultants managed the whole performance delivery from start to the end from Discovery, test planning, environment configuration, test preparation and scripting, execution, defect triaging and reporting

=

Multiple workshops were required with various teams such as architecture, development, and business were held to agree on expected NFRs

=

Updated documentation & scripting around performance.

=

Performance scripts for real-time workloads were developed in JMeter and Golang scripts were maintained in GIT following best practices.

=

AWS X-RAY and Cloudwatch were used for monitoring

=

Tuning recommendations were provided on real-time workloads

Measurable Benefits:

=

On initial run of 1 million notifications, it took 3 days. With monitoring and performance scripting logs we were able to determine design issues and operational issues and were able to identify where the bottlenecks were residing in the system and also configuration recommendations on Lambda settings. This then brought the run down to 15 minutes

=

Further recommendations through SQS optimizations then brought the 25 million test to 40 minutes which was accepted

=

Scripts for spatial and geolocation data can be reused easily for future performance tests

Uplift your DevSecOps capability today

;

Contact Us

TALK TO AN EXPERT