Schedule: Hybrid (3 days onsite), after 4 weeks the position will be remote
Salary: $140k - $150k + 5% bonus
Join our dynamic team as a Data Engineer and spearhead the evolution of our data management platform! We're on a mission to revolutionize our data landscape, empowering innovation, efficiency, and ongoing enhancement as the keystones of our effort. Leveraging tools and languages such as Python, Java, and SQL, our goal is to craft streamlined data pipelines in a way that is as platform agnostic as possible. As part of this journey, you'll craft an enterprise-wide data model, facilitating our business-centric analysts in generating insightful reports across our entire data ecosystem. We're seeking individuals with a knack for tackling intricate data challenges, coupled with a passion for continuous learning and adaptation to emerging technologies. In this role, you'll work in a collaborative environment, supporting a variety of teams, stakeholders, and products with their data requirements. Join us and be at the forefront of driving data-driven success!
ESSENTIAL FUNCTIONS: Essential duties and responsibilities may include, but are not limited to, the following:
Product and thrive in a collaborative development environment.
Participate in decisions around data warehouse architecture and, more specifically, data models and database design.
Design, build and maintain optimal data pipeline architecture.
Maintain and optimize the data infrastructure required for accurate extraction, transformation, and loading of data from a wide variety of data sources.
Follow standard practices, patterns, and guidelines for database deliverables including conceptual design, deployment scripts, data migration plan and performance tuning.
Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
Provide access to datasets using REST and Python APIs.
Design and develop the extraction, transformation, and loading of data from a wide variety of data sources using Python or Java, SQL and cloud technologies.
Monitor data systems performance and implement optimization strategies.
Actively participates in the inception, technical design, development, testing and delivery of data solutions.
Ability to understand and disseminate requirements information from requirements documents into practical work instructions.
Estimate level of effort, evaluate new options of similar technology, and offer suggestions to improve processes.
Contribute to long-term strategic vision of platform and tools.
Support data management efforts and continued improvements to our internal technical infrastructure.
QUALIFICATIONS:
Required Knowledge/Skills/Abilities:
7+ years of relevant working experience.
Bachelor's degree in Information Systems, Computer Science, or related field is required.
Strong proficiency in object-oriented languages: Python, Java, C#, Scala.
Ability to design, build and unit test applications on Spark framework on Python.
Build PySpark based applications for both batch and streaming requirements.
Develop and execute data pipeline testing processes and validate business rules and policies.
Able to provide technical consultation and coaching to coworkers.
Experience working with cloud Data Warehouse solutions (e.g., Snowflake, Redshift, Azure
