Descripción del puesto / Funciones
- Design, develop, and maintain ELT data pipelines.
- Deploy and manage data solutions in a cloud environment.
- Participate in data platform migration projects.
- Optimize performance of data processing jobs and clusters.
- Troubleshoot and resolve production issues.
- Collaborate with cross-functional data teams (architects, analysts, data scientists).
- Create and maintain technical documentation for data pipelines and processes.
- Implement data quality checks and validation processes.
- Develop data integration and processing workflows.
- Manage version control and implement CI/CD pipelines.
- Prepare training materials and support knowledge sharing within the team.
Requisitos mínimos
- Minimum of 4 years of experience as a Data Engineer.
- Proven experience building, deploying, and maintaining ETL/ELT pipelines in cloud environments.
- Hands-on experience with Databricks and Unity Catalog.
- Experience with Azure data services, including Azure Data Lake Storage and Azure Data Factory.
- Strong knowledge of Apache Spark (job and cluster optimization), Databricks Workflows and Jobs, and Delta Lake.
- Solid understanding of medallion architecture and Lakehouse environments.
- Strong data modeling skills.
- Proficiency in Python (PySpark).
- Advanced SQL skills.
- Good understanding of development best practices, version control, and CI/CD.
- Professional level of English.
Requisitos valorables
Knowledge of Scala programming.
Experience with other cloud platforms such as AWS or GCP.
Experience building ETL processes from third-party APIs.
Familiarity with data streaming technologies (e.g., Spark Streaming).
Experience with Infrastructure as Code (IaC).
Understanding of data governance principles.
Idiomas
English is a must
Ubicación
Barcelona (hybrid)
