We are seeking a highly skilled Senior Data Engineer This role involves extensive coding, infrastructure management, and automation, with a focus on building and maintaining robust ETL pipelines and tools. The ideal candidate will be a key individual contributor, working closely with the director and mentoring junior team members.
Key Responsibilities:
ETL Pipeline Development: Build and manage ETL pipelines using Python, Kafka, Snowflake, and AWS, ensuring data is efficiently processed and delivered.
Heavy Coding: Focus on Python development to implement data processing, analysis, and automation tasks.
Infrastructure as Code (IaC): Utilize SQL-based IaC tools (CloudFormation, Serverless, Terraform) for infrastructure management.
Automation & Optimization: Automate manual processes, optimize data delivery, and redesign infrastructure to enhance scalability and performance.
End-to-End Development: Take ownership of all aspects of development, from application code to cloud deployment and ongoing observability.
Testing & CI/CD: Develop automated unit and integration tests, and create and maintain CI/CD pipelines using GitHub Actions.
Mentorship: Act as a go-to resource for junior team members, providing guidance and support.
Collaboration: Work closely with the director to align on project goals and contribute to the overall success of the team.
Required Skills and Qualifications:
Python Development: Extensive experience in Python coding, particularly for ETL and data processing.
AWS & ETL Tools: Proficiency with AWS services, Kafka, Snowflake, and experience in building and managing ETL pipelines.
Infrastructure as Code: Experience with Terraform, CloudFormation, and Serverless frameworks for infrastructure management.
DevOps & CI/CD: Strong understanding of DevOps practices and experience with CI/CD pipelines, including GitHub Actions.
SQL & Databases: Experience with relational databases, SQL, and building unit tests.
Version Control: Advanced skills with Git or other version control systems.
Docker & Linux/Unix: Experience with Docker and working in Linux/Unix environments.
Beacon Hill is an Equal Opportunity Employer that values the strength diversity brings to the workplace. Individuals with Disabilities and Protected Veterans are encouraged to apply.
California residents: Qualified applications with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.
If you would like to complete our voluntary self-identification form, please click here or copy and paste the following link into an open window in your browser: https://jobs.beaconhillstaffing.com/eeoc/
Completion of this form is voluntary and will not affect your opportunity for employment, or the terms or conditions of your employment. This form will be used for reporting purposes only and will be kept separate from all other records.
Company Profile:
Beacon Hill Technologies, a premier National Information Technology Staffing Group, provides world class technology talent across all industries utilizing a complete suite of staffing services. Beacon Hill Technologies' dedicated team of recruiting and staffing experts consistently delivers quality IT professionals to solve our customers' technical and business needs.
Beacon Hill Technologies covers a broad spectrum of IT positions, including Project Management and Business Analysis, Programming/Development, Database, Infrastructure, Quality Assurance, Production/Support and ERP roles.
Learn more about Beacon Hill and our specialty divisions, Beacon Hill Associates, Beacon Hill Financial, Beacon Hill HR, Beacon Hill Legal, Beacon Hill Life Sciences and Beacon Hill Technologies by visiting www.bhsg.com .
