Mission Context
As part of a data engineering project, our client is looking for a hands-on Databricks Engineer to build and support data pipelines on top of the raw layer owned by the Platform team.
Missions
- Build and optimize Databricks pipelines using PySpark, SQL and Delta Lake
- Transform Raw data into curated and business-ready layers
- Own production deployments through Azure DevOps (ADO), including code reviews, testing and releases
- Track work in ADO Boards
- Monitor, troubleshoot and tune jobs for performance, reliability and cost
- Implement data quality checks and governance using Unity Catalog
- Partner with business users and the Platform team to turn requirements into technical solutions
- Maintain clear documentation and support runbooks
Required Profile
- 8+ years of experience in data engineering
- 5+ years of hands-on Databricks experience in production
- Strong hands-on experience with Databricks, including notebooks, jobs, workflows and clusters
- Proficiency in PySpark, Python and advanced SQL
- Solid understanding of Delta Lake, data modeling and ETL/ELT patterns
- Experience supporting production workloads, with strong debugging and data validation skills
- Working knowledge of Git and CI/CD, ideally Azure DevOps (Repos, Pipelines, Boards)
- Bachelor's degree in Computer Science, Engineering or related field, or equivalent experience
- Excellent written and verbal English
- Self-starter who takes initiative, works independently and proactively solves problems
- Ability to own work end to end, from requirements to production support, and meet commitments
- Comfortable working with business users and explaining technical topics simply
- Collaborative communicator who shares proactive updates, raises risks early and works well across time zones
Additional Information
- Location: Brazil (Hybrid 2x days in office per week)
- Working Hours: Must be willing to work US CST hours
- Work type: Contract/freelance