Our technology group is a dynamic, fast-paced environment, with exciting changes on the horizon under new senior leadership. We are looking for a data engineer /developer to build next generation data platform from ground up on AWS as part of a small focused team. Our ideal candidate will have strong knowledge of computer science fundamentals with programming experience in Scala / Python and Spark. The right candidate for this role will identify this challenge as a unique and valuable opportunity to help drive our global technology transformation.
What you will do:
What You Will Need:
- Build complex data ingestion pipelines using Scala, Spark, Parquet and S3
- Design scalable processes in event driven architecture to support Fixed Income applications
- Develop near real time streaming analytics using Kafka/Kinesis
- Act as Subject Matter Expert and help rest of the team in leveraging the platform and migrating applications to it
- Establish end to end data lineage and data catalogue. Work with data governance team to setup data quality checks and metrics
- Create self-service notebook environment with Zeppelin/Jupyter for exploratory data analytics and rapid interactive development
- Troubleshoot any performance issues and ensure efficient data organization
- Build efficient web-based tools for monitoring and tracking
What we would love to see:
- Bachelor’s degree in computer science or related field required. Master’s degree preferred
- Knowledge of data structures, algorithms, and functional programming
- Passion to learn new things, experiment with new ideas and build world class data platform
- 5+ years of experience in programming with Scala, Python or Java
- 2+ years of experience in Scala, Spark, and functional programming. Deep knowledge of spark internals such as partitioning, DAG, lazy evaluation.
- Strong experience with relational data bases, SQL, and query optimization. Knowledge of data warehousing, dimensional data model and business intelligence is a plus.
- Knowledge/experience with event driven programming and Akka actor model
- Excellent verbal and written communication skills
- Experience with AWS infrastructure, docker, ECS/EKS, EMR, Kafka/Kinesis.
- Front-end development experience with Angular, Java Script, Reactive Programming.
- Knowledge or desire to learn Investment Management, Fixed Income and Finance
- Familiarity with NoSQL and ElasticSearch