Data Engineer

Requirements

• 5+ years of commercial experience in Data Engineering, ideally in senior-level positions.
• Strong hands-on experience with Microsoft Azure and Azure-based data solutions.
• Strong proficiency in Python.
• Solid experience with Apache Spark / PySpark or other distributed data processing frameworks.
• Strong understanding of Data Engineering best practices, principles, and patterns.
• In-depth knowledge of data concepts, data structures, and data modelling techniques.
• Experience designing and provisioning data to support different data consumption and business needs.
• Experience applying Test-Driven Development (TDD) practices.
• Strong understanding of the full data lifecycle, from ingestion and transformation to production and consumption.
• Good level of English, both written and spoken.
• Availability to work 1 day per week from the office in Matosinhos.

Responsibilities

• Design, develop, and maintain scalable and reliable data solutions using Azure and modern Data Engineering technologies.
• Deliver business value into Production, ensuring data solutions meet business and technical requirements.
• Develop and optimize data processing pipelines using Python and PySpark / Apache Spark.
• Apply Data Engineering best practices to ensure code quality, scalability, maintainability, and performance.
• Design appropriate data structures and data models according to business and technical requirements.
• Build and provision data solutions that support different consumption needs and use cases.
• Ensure data solutions are production-ready, reliable, and scalable.
• Apply TDD and automated testing practices throughout the development lifecycle.
• Collaborate with technical and business stakeholders to translate requirements into effective data solutions.