MidFull TimeENMiddle East
About ShipIn:
At ShipIn Systems, we empower maritime leaders with our cutting-edge Visual Fleet Management Platform. Leveraging patented computer vision technology and real-time visual analytics, our platform provides proactive alerts to shipowners, managers, and seafarers, enhancing safety, efficiency, and operational excellence. By delivering unparalleled visibility into onboard activities, ShipIn is transforming maritime operations and modernizing the global supply chain.
Position Description:
We are looking for a Data Engineer to join our new Data team. In this role, you will take ownership of the existing data pipelines, develop and maintain On-Premises data pipelines running on vessels, handling large-scale, real-time video streams operating 24/7. While most of the work is On-Premises, some AWS integrations will be required for data management.
You will join a new team and work closely with architects, DevOps, product managers, and algorithm engineers. Initially, you'll focus on maintaining the existing pipeline, with future responsibilities including redesign and optimization. Given our hybrid On-Premises and cloud architecture, this role requires adaptability to complex systems.
Key Responsibilities:
Design, implement, and optimize the product data pipelines on the on-prem environments.
Collaborate with data scientists to deploy and scale ML models, ensuring they operate efficiently in real-time environments.
Develop high-performance data pipelines for both batch and stream processing.
Integrate the on-prem data pipeline with AWS cloud infrastructure, for both data pipeline SW deployment as well as publishing the data pipeline output to the cloud.
Write clean, efficient code using Python to maintain the data pipelines.
Collaborate with cross-functional teams, including computer vision \ dev ops \ product managers and other development teams, to establish the required infrastructure in the data pipelines for supporting new product features and improve the overall system performance.
Participate in design review and code reviews, providing constructive feedback, and maintaining high-quality standards across the team.
Stay current with the latest developments in data engineering, applying new insights to improve existing processes.
Troubleshoot and resolve issues related to the data pipelines, ensuring minimal downtime and optimal performance.
Qualifications / Experiences:
2+ years of experience working with data pipelines
Proficiency in Python.
Understanding of how to integrate machine learning models into large-scale production environments.
Experience with working with streaming platforms such as Kafka, Redis streams, Rabbit MQ, Google Pub Sub or their equivalence.
Excellent communication and collaboration skills, to collaborate with cross-functional teams.
Advantage - experience with image\video processing
Advantage - Hands-on experience working with Postgres
