Zapata Technology is seeking an experienced Data Engineer, for a hybrid role in Monterey, California. The Data Engineer will lead the migration of our data infrastructure from Confluent with manual ETL processes to Apache NiFi. The ideal candidate will have extensive experience in data engineering, ETL processes, and managing data pipelines in cloud-native environments.
Check out the details below and our awesome benefits here.
Job Qualifications:
Qualifications and Skills include:
Design and develop modular ETL solutions which ingest and deliver data from source systems to analytics platforms (structured, semi-structured, and unstructured; queue / batch and real-time)
Experience working with programming languages
Experience in developing and maintaining scalable data storage systems
Proficiency in building software to retrieve, parse, and process structured and unstructured data
Experience in designing scalable ETL and ELT workflows for reporting and analytics
Strong ability to collaborate and create solutions within cross-functional teams
Skilled in developing scripts and programs to convert various data types into usable formats, support scaling project teams, and manage data platforms
Hold Documenting administrative metadata and assigning compliance documents to newly received entities.
Manually updating the status in the roadmap each month for assigned exchanges.
Ensuring compliance with HRPP/Privacy/HIPAA/SOP prior to submission to the DLA for processing.
Documenting exchange receipt dates in the roadmap.
Investigating, documenting, and resolving Data Quality Assurance issues discovered by project staff.
Creating documentation and storing it in the RFL-specified repository per Appendix D of this document.
Reporting any discovered human use concerns or Pil leaks.
Conducting Quality Assurance tasks and reports as directed.
Transferring, downloading, scrubbing, and loading data, including the development of new and updated metadata files.
Coding updates to existing data transformations as needed.
CompTIA Security+ certification, demonstrating commitment to security and compliance.
Computing Environment Certification
Preferred Qualifications and Skills include:
Specialized Agency Experience with Apache NiFi (Niagara Files) is preferred, and training will be required if that skillset is not already in your portfolio
Experience with low-code/no-code platforms
Proficiency in application development using SQL or other such applications
Familiarity with public cloud platforms (e.g., AWS, Azure)
Experience with distributed data processing tools such as Spark, Databricks, Hive, or Kafka
Experience working with real-time data and streaming applications
Knowledge of data warehousing concepts
Proficiency in basic commands and shell scripting
Experience with Agile methodologies
Education/Experience includes:
4+ years of ETL experience
Working Conditions:
Prolonged periods sitting at a desk and working on a computer
Position Type/Expected Hours of Work:
Full time position
Clearance:
This position requires an active Secret clearance with the ability to obtain TS/SCI.
AAP/EEO Statement:
Equal Opportunity Employer – M/F/Disabled/Veteran
Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status.
Zapata Technology, Inc. provides reasonable accommodation to applicants who are veterans or who have disabilities and are unable to fully use our company application system. If you need a reasonable accommodation for any part of the application and hiring process, please notify Christina Hall, EEO Coordinator at ChristinaHall@ZapataTechnology.com
