Job Title: Azure Data Engineer 8+ years
Location: Dallas, TX
Employment Type: Full-Time / W2
About the Role: We are seeking a highly skilled and experienced Azure Data Engineer to design, develop, implement, and maintain scalable cloud-based data engineering solutions using the Microsoft Azure ecosystem.
Key Responsibilities:
- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory (ADF), Azure Databricks, Python, PySpark, and SQL.
- Develop and optimize cloud-based data solutions using Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure SQL Database, and Azure Blob Storage.
- Build robust batch and real-time data pipelines for ingesting, transforming, cleansing, and integrating structured and unstructured data from multiple sources.
- Develop complex SQL queries, stored procedures, views, functions, and performance optimization strategies for large-scale data processing and analytics.
- Implement data transformation and processing frameworks using Databricks, Apache Spark, PySpark, and distributed computing technologies.
- Design and implement data warehouse and dimensional modeling solutions, including Star Schema, Snowflake Schema, Fact and Dimension tables, SCD Type 1/Type 2, and data marts.
- Perform data integration across relational databases, APIs, flat files, SaaS applications, enterprise systems, and cloud data sources.
- Implement data quality, validation, reconciliation, error handling, logging, monitoring, and alerting mechanisms across enterprise data pipelines.
- Develop reusable and parameterized Azure Data Factory pipelines, including triggers, linked services, datasets, integration runtimes, and pipeline orchestration.
- Implement CI/CD and DevOps practices data engineering solutions using Azure DevOps, Git, YAML pipelines, and automated deployment processes.
- Apply Azure security, governance, access control, encryption, and data protection best practices using technologies such as Azure Key Vault, Microsoft Entra ID, RBAC, and managed identities.
Required Skills & Qualifications:
- 8+ years of experience in Data Engineering, Data Warehousing, ETL/ELT, and cloud-based data platforms, with strong hands-on experience in Microsoft Azure.
- Strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), Azure SQL Database, and Azure Blob Storage.
- Advanced proficiency in SQL and experience with query optimization, stored procedures, views, functions, indexing, and performance tuning.
- Strong programming experience with Python and PySpark, including data processing, transformation, automation, and distributed computing.
- Hands-on experience developing ETL/ELT pipelines, data ingestion frameworks, data integration, data transformation, and pipeline orchestration.
- Strong knowledge of Apache Spark, Databricks, Delta Lake, Lakehouse Architecture, and large-scale data processing.
- Experience with Data Warehousing, Dimensional Modeling, Star Schema, Snowflake Schema, Fact/Dimension modeling, SCD Type 1/Type 2, and enterprise data architecture.