Case Study

Data center monitoring platform with IoT analytics on AWS cloud

For a leading provider of turnkey datacenter solutions

Style

Post-info

Lack of expertise to understand the current data center Performance and Operations
All incidents at data centers were logged and tracked manually, there was an urgent need for automation
High CAPEX investment required for platforms (including customization) to monitor and track data center operations
Effective utilization of Datacenter personnel by giving targeted (value add) information for troubleshooting
Strict regulatory government requirements on data center standards

The solution was developed as Cloud-native on AWS cloud with a Microservices architecture making use of AWS services (IoT, Kinesis Streams, Lambda, DynamoDB, Elastic Search, Redshift, Cognito, CloudFront, CodeCommit, CodeBuild & CodeDeploy)
Leveraged AWS CodeCommit, CodeBuild and CodeDeploy to create a custom DevOps pipe-line
Created actionable insights through Co-relation and Causation Analysis using Spark ML (Bayesian probability algorithms)
Created near real-time dashboards and visualizations (SVG-based)
Phase 2 to incorporate Cognitive Services (including vision & audio)

Enabled the client to start small scale and then expand to a multi-node cluster based on demand, using Amazon Redshift, a highly scalable Massively Parallel Processing (MPP) Architecture.
Reduced license costs on expensive proprietary software, through extensive use of open source tools.
Concurrent processing of large data volumes, increasing the throughput of the data load using AWS EMR.
A generic intermediary data structure (domain model) enabled the organization to add any new source systems with minimal effort and less cost.
High availability, fault tolerance and resilience using AWS EMR, Redshift and S3