Customer is an IT solutions provider for leading mutual fund companies in India. The customer provides the backend transaction processing and customer care for over 60% of the companies in the fund management business. With the constant increase in new fund companies, the total number of users and end user expectations with regards to turnaround times, the customer’s infrastructure was constantly under stress to process and send out periodic account statements as quickly as possible. This was resulting in high IT overhead costs to procure and manage the infrastructure.
Implemented a cloud based parallel processing architecture and job scheduler (POC on key accounts & then full production). Tooling integration with existing legacy systems was completed in short order without disrupting current operations. Predicted time and cost savings for processing were also tested and proved. A new data dump was used to create a testing database using masked user information. Minor modifications to batch processing code were made to integrate with the Parallel and Batch processing engine. The initial POC was setup and completed in one week. This included data masking, code updates and database setup.
- 10X+ reduction in overall processing times.
Processing of 100,000 items in 12 hours was reduced to 50 minutes
- 68% savings in processing costs.
All instances procured from AWS were spot instances with heavy discounts
- Parallel processing at scale to reduce time.
Approximately 250 threads over 20 instances were used instead of 10 threads in the original infrastructure
- Optimal utilization of resources
Healthy utilization of 65% (cluster averaged) across all instances
- 800 records processed per minute at peak.
- 10 cents from 1000 items