About The Organization.
OpSight is our Washington DC-based client which provides identity matching information to healthcare industry was looking to improve its service efficiency. The client collects identities from multiple sources and a provides collapsed and searchable database of patient and healthcare professionals. The customer had over 12TB data and performed a monthly batch job to reconcile the identities.
Completely migrate and automate service in AWS
Ability to create custom clusters to support monthly batch job
Each cluster needs to have all dependencies, applications, users, storage created
Batch job has to be completed within 3-business days to offset the additional cost
Need to comply with HIPAA and PII requirements
When the client approached us, they were looking for a reliable means of completing the monthly batch job. The number of data points were increasing every month and the customer had a goal of reconciling 50K identities every hour to make the increased spend in AWS worthwhile. This would allow them to complete the job in 72 hours. They also didn’t want to make drastic changes to the HIPAA- approved architecture or software stack.
Our team got to work immediately and met with software, network, data architects to understand the workflow and component interactions. We understood the authentication, data flow, application roles, deployment, monitoring, KPIs and scaling challenges.
We presented AWS architecture, our approach and strategy, scaling tactics, pricing, gap analysis, TCO and ROI analysis. Our approach and attention to detail was well received and after incorporating feedback from customer, we began Phase 2 of the project.
Our team used a modular method to build each layer of the cluster (Java / app tier, SOLR search service, MongoDB database, auxiliary services layer et al) one at a time. Every item of the application stack was configurable: the instance size, storage allocated, number of instances, software versions, users and roles to be created, location of application artifacts. We then used Terraform and Chef infrastructure automation tools to provision in AWS. The clusters were brought up in a particular sequence, configured to allow inter-node communication, dependencies installed and application deployed. The customer would then point to the latest data store and kick-off the batch job.
These efforts enabled the customer to reorganize their IT practices and adopt a “Cloud First” policy. The customer now has:
- A well-defined application profiles and can spin up app clusters on demand with a single click
- Migrated off of a data center and retired over 400 servers
- Gained new sources of revenue by offering this service as a SaaS solution
- Batch job that takes 3-days to complete and monthly bill is reduced by over 65%
- Can accurately charge-back their end clients as cost of each app cluster is known in advance