The right candidate enjoys working collaboratively to solve complex data engineering challenges that support our Business needs
Responsibilities
Design, implement and maintain large-scale data pipelines using Python, Databricks
Provisioning, developing, maintaining and tuning Amazon Oracle RDS
Work with cross-functional teams to integrate solutions that meet our data processing and analysis needs
Ensure performance, reliability, security and maintainability of data pipelines
Develop monitoring and alerting mechanisms to proactively identify and resolve issues
Continuously look for opportunities to improve data engineering processes and practices
Requirements
Minimum of 3+ years with expertise in designing and maintaining large-scale data pipelines
Strong proficiency in building and maintaining stored procedures, triggers and functions in Oracle DB
Strong proficiency in using DMS to sync data from RDS to Databricks
Strong proficiency in Python and Databricks(AWS) in production environment
Excellent understanding of Big Data technologies, primarily Spark
Strong understanding of Data Architecture and modelling concepts
Highly knowledgeable about data warehousing principles and practices
Good to have skills
Knowledge in State Street internal Datawarehouse ESP
Certification on AWS, Databricks
Educational qualifications
BS in Computer Science with 5 years of experience or MS in Computer Science with 3 years of experience
Salary Range: $100,000 - $160,000 Annual
The range quoted above applies to the role in the primary location specified. If the candidate would ultimately work outside of the primary location above, the applicable range could differ.
