Databricks Data Engineer

Databricks Data Engineer - London (Hybrid) – Up to £60,000

Databricks - Azure - PySpark - Unity Catalog - AI & Machine Learning

Are you looking to join a growing Data & AI team delivering some of the UK's most ambitious transformation programmes.

Due to continued growth, we're hiring multiple Databricks Data Engineers to join a rapidly expanding Data & AI practice. Whether you're currently working within a consultancy or delivering Databricks solutions in industry, we'd love to hear from engineers passionate about modern data platforms, AI and cloud technologies.

Our client is building one of the UK's leading Databricks practices and is investing heavily in Data & AI. You'll work on Databricks-led transformation programmes, helping enterprise organisations design and build modern Lakehouse platforms that support advanced analytics, machine learning and AI initiatives.

You'll join a high-performing team of architects, engineers and AI specialists, solving complex technical challenges for large-scale clients. This is an opportunity to work with cutting-edge Databricks technologies, gain exposure to major Data & AI programmes and accelerate your career within a collaborative and highly respected consulting environment.

If you enjoy combining hands-on engineering with stakeholder engagement and solving real-world business challenges through data, this is a fantastic opportunity to make an impact.

You Will Work With

• Designing and developing Databricks Lakehouse solutions
• Building batch and streaming data pipelines using PySpark and Spark SQL
• Creating trusted, AI-ready data products
• Implementing Delta Lake, Unity Catalog and Databricks SQL solutions
• Supporting end-to-end delivery from requirements gathering through to deployment
• Optimising data solutions for performance, scalability and reliability
• Working alongside architects, data scientists and cloud specialists
• Supporting machine learning and AI-enabled data initiatives
• Applying modern engineering practices including CI/CD, automated testing and Infrastructure as Code
• Contributing to technical best practices and continuous improvement across delivery teams

Benefits

• Annual performance bonus
• Hybrid working model
• Access to enterprise-scale Data, AI and Cloud programmes
• Clear and structured career progression
• Industry-recognised training and certifications
• Exposure to Databricks, AI and cloud technologies
• Private healthcare
• Pension scheme
• Flexible benefits package
• Inclusive and collaborative working culture

Key Experience

Essential

• Commercial experience in Data Engineering
• Experience working with Databricks in a production environment
• Strong SQL and Python and/or PySpark skills
• Experience building and supporting ETL/ELT pipelines
• Understanding of Delta Lake, Lakehouse Architecture or Medallion Architecture
• Experience working with cloud platforms such as Azure, AWS or GCP
• Strong communication and stakeholder engagement skills

Desirable

• Azure Data Factory, Airflow or Databricks Workflows
• Unity Catalog
• Databricks certifications
• CI/CD and Infrastructure as Code
• MLflow, Feature Engineering or AI-related workloads
• Consultancy or client-facing experience