Senior Specialist - Data Engineering

ob Title Data Architect Azure Databricks

Experience

11 Years

Location

India HybridRemote

Job Summary

We are seeking an experienced Data Architect with 11 years of experience in designing and implementing enterprisescale data platforms and analytics solutions on Microsoft Azure The ideal candidate should possess strong expertise in Azure Databricks Azure Data Factory ADF SQL Python and PySpark with proven experience in building scalable secure and highperformance data ecosystems supporting business intelligence advanced analytics and AIML initiatives

Mandatory Skills

Azure Databricks

Microsoft Azure Cloud

Azure Data Factory ADF

SQL

Python

PySpark

Data Architecture Design

Design and implement scalable cloudbased data architectures on Azure

Define enterprise data models data integration patterns and data governance standards

Architect modern Data Lakehouse solutions using Databricks and Delta Lake

Create conceptual logical and physical data models

Data Engineering Integration

Design and develop batch and realtime data pipelines using ADF and Databricks

Build scalable ETLELT frameworks using Python and PySpark

Integrate data from multiple source systems including databases APIs SaaS platforms and files

Optimize Spark jobs Databricks clusters and SQL workloads for performance and cost efficiency

Data Governance Security

Implement data quality lineage metadata and governance frameworks

Establish security controls rolebased access and compliance standards

Define best practices for data lifecycle management and monitoring

Stakeholder Engagement

Work closely with business stakeholders product owners and development teams to translate business requirements into technical solutions

Provide technical leadership and architectural guidance to data engineering teams

Participate in architecture reviews solution design sessions and technology evaluations

Delivery Leadership

Lead endtoend implementation of enterprise data platforms

Mentor data engineers and enforce engineering best practices

Drive continuous improvement and innovation in cloud data solutions

Required Qualifications

Bachelors or Masters degree in Computer Science Information Technology Engineering or related field

11 years of experience in Data Engineering Data Warehousing and Data Architecture

5 years of handson experience in Azure Cloud and Databricks ecosystem

Strong experience designing Lakehouse architectures and enterprisescale data platforms

Expertise in SQL optimization and advanced query development

Strong programming background in Python and PySpark

Experience with CICD DevOps and Agile delivery methodologies

Technical Competencies

Azure Databricks

Azure Data Factory ADF

Azure Data Lake Storage ADLS Gen2

Delta Lake

Spark SQL

Python

PySpark

Azure SQL Database

Data Modeling

ETLELT Design

Data Governance Security

Performance Tuning

Azure DevOps Git

Azure Synapse Analytics

Microsoft Fabric

Data Mesh Data Products Architecture

Streaming technologies Kafka Event Hub

AIML Data Platform experience

Databricks Unity Catalog

Certification in Azure Data Engineering or Azure Solutions Architecture

Soft Skills

Strong communication and stakeholder management

Excellent analytical and problemsolving abilities

Leadership and mentoring capabilities

Ability to drive architecture decisions across distributed teams

Key Deliverables

Enterprise Data Architecture

Lakehouse Data Platform Design

Scalable Data Pipelines

Data Governance Frameworks

Performance Optimized Analytics Solutions

Cloud Migration Modernization Strategies