Role Summary
- Experience Required: 4-6 years
- Primary Role: Data Engineer / API Role with AI Builder
- Cloud Platform: Azure Cloud
- Role Type: Individual Contributor
Core Technical Skills
- Azure Databricks
- PySpark
- Scala
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Delta Lake
- Unity Catalog
- Python
- SQL
- GitHub
- CI/CD
Data Engineering Responsibilities
- Design, develop, and optimize scalable data pipelines.
- Build cloud-based data solutions using Azure.
- Develop ETL/ELT data processing workflows.
- Work with large-scale datasets in enterprise environments.
- Perform data modeling and performance tuning.
- Implement and maintain Delta Lake solutions.
- Work with Databricks, ADF, and PySpark for data engineering workloads.
- Develop end-to-end data solutions independently.
- Apply data governance and security best practices.
API Development Skills
- Strong hands-on experience building REST or GraphQL APIs.
- Understanding of:
- API design and contracts
- API security
- Error handling
- API performance optimization
- Experience deploying APIs using Kubernetes, preferably Azure Kubernetes Service (AKS).
AI / ML Skills
- Exposure to AI/LLM technologies.
- Experience or exposure to agentic frameworks.
- Familiarity with:
- Databricks AI/ML
- Codex
- Claude
- Other LLM/AI development frameworks
Cloud & DevOps
- Strong hands-on experience with Azure Cloud.
- Experience with CI/CD practices.
- GitHub experience.
- Knowledge of cloud-native data platforms.
- Understanding of Kubernetes/AKS deployment.
Data Governance & Security
- Strong understanding of data governance.
- Knowledge of data security and access control.
- Experience with Unity Catalog is preferred.
- Ability to implement secure and governed data solutions.
Essential Skills
- Databricks
- Azure Data Factory (ADF)
- PySpark
- Azure Cloud
- ADLS Gen2
- Delta Lake
Desirable Skills
- Scala
- REST APIs / GraphQL
- Kubernetes / AKS
- AI/LLM
- Agentic frameworks
- Databricks AI/ML
- CI/CD
- Unity Catalog
Key Responsibilities
- Own end-to-end solution delivery.
- Analyze and solve complex data engineering problems.
- Collaborate with cross-functional teams and business stakeholders.
- Build scalable and reliable data pipelines.
- Optimize data processing and API performance.
- Follow enterprise standards for governance, security, and deployment.