Job Description
Synopsis
This individual will work with cross‑functional product teams to design, implement, and support modern data platforms leveraging Databricks and related technologies. The role requires expertise in Delta Lake, Unity Catalog, Delta Live Tables (DLT), and Lakeflow, combined with strong skills in PySpark, Structured Streaming, Autoloader, and DQX. A solid background in CI/CD, Terraform, DevOps, and DataOps practices is essential. The engineer will drive platform reliability, scalability, and automation across cloud environments, ensuring robust analytics and data science capabilities.
Accountabilities
- Treat the data platform as a core product, ensuring scalability, reliability, and innovation.
- Collaborate with Architecture on the development and prioritization of the Databricks‑based platform roadmap.
- Integrate new services such as Delta Lake, Unity Catalog, and DLT into the platform ecosystem.
- Prepare and maintain documentation for Databricks workflows, CI/CD pipelines, and infrastructure automation.
- Partner with stakeholders to resolve platform‑related technical issues and identify areas for improvement.
- Implement internal process improvements: automate ingestion with Autoloader, optimize streaming with Structured Streaming, and enforce data quality with DQX.
- Adopt Agile and DevOps practices, ensuring CI/CD pipelines and Terraform‑based infrastructure are production‑ready.
- Provide operational support for Databricks clusters, jobs, and pipelines.
- Perform fallout & impact analysis, ensure recovery of impacted transactions, and provide business impact reports for production incidents.
- Propose new ideas and technologies to foster continuous improvement and enhance user experience.
- Define, share, and drive adoption of best practices for DataOps and cloud‑native data engineering.
Qualifications
Education & Experience
- 6+ years of experience as a Data/Platform Engineer, with proven expertise in Databricks and modern data lakehouse architectures.
- Graduate degree in Computer Science, Information Systems, or a related quantitative field.
- Hands‑on knowledge of PySpark, Structured Streaming, Autoloader, DQX, and Databricks workflows.
- Strong experience with CI/CD pipelines, Terraform, DevOps, and DataOps practices.
- Ability to troubleshoot and solve complex problems across distributed data systems.
- Experience working with Linux, shell scripting, public cloud platforms, and big data technologies.
- Strong communication skills and experience collaborating with cross‑functional teams in Agile environments.
- Ability to work independently with minimal supervision and eagerness to learn new technologies.
Additional Information
Etihad Airways is more than the UAE’s national airline - we’re shaping the future of global aviation. From our hub in Abu Dhabi, we connect people across continents and cultures, while driving our “Journey 2030” ambition to double our fleet, expand our network and grow our people for a new era of excellence. Join us and become part of a vision where the world is your workplace and extraordinary experiences take flight.
To learn more visit www.etihad.com
Recruitment Fraud Alert
Beware of fraudulent job offers from individuals or organizations claiming to represent the Etihad group. We will never ask for personal information, bank details, or payment during the recruitment process. Interviews are conducted face-to-face or via video/telephone before any formal offer. If you are asked for money, please treat it as fraudulent.
