Data Engineer

Senior Data Engineer

Python | SQL | AWS / Azure | Databricks

London, UK | Hybrid working

Overview:

The focus is on engineers who can build scalable data pipelines, develop cloud-based data platforms and deliver reliable data infrastructure to support analytics, machine learning and AI-driven applications.

I'm particularly interested in professionals who enjoy solving complex engineering problems, improving data reliability and taking ownership of technical solutions from design through to production.

Key responsibilities

• Design, build and maintain scalable ETL/ELT pipelines using Python and SQL.
• Develop and optimise data processing workflows across cloud-based environments.
• Build and maintain data platforms using technologies such as Databricks, Snowflake and cloud-native services.
• Work with large datasets using Apache Spark and distributed processing frameworks.
• Implement data quality checks, testing, monitoring and pipeline observability.
• Collaborate with analytics, software engineering, product and machine learning teams.

Essential experience

• Strong commercial experience with Python and SQL.
• Experience building and maintaining production data pipelines.
• Hands-on experience with AWS, Azure or GCP.
• Understanding of data modelling, warehousing and ETL/ELT architecture.
• Experience troubleshooting data quality, performance and reliability issues.
• Familiarity with Git, testing and software engineering best practices.