Enabling Areas National Office Manager Data Engineering
Deloitte India (South Asia) · Consulting & Professional Services
- Bengaluru, India
- On-site
- Posted today
- Apply by 30 Oct
- Data & AI
About the job
Enabling Areas - National Office - Manager - Data Engineering
Job requisition ID : 116330
Location : Bengaluru
Entity : Deloitte Shared Services India LLP
The team
The Office of the COO, Deloitte India is powered by a dynamic team known for its agility and results. Despite its size, the team manages critical projects, consistently delivering high-impact outcomes.
Collaboration is the key to the team’s success, with members working closely together while taking ownership of their responsibilities. They operate in a fast-paced environment, engaging directly with senior leadership (CXOs’) to implement key initiatives and drive strategic outcomes.
Your work profile
Data Lake Architecture & Engineering
Design, develop, and maintain scalable data lake solutions using Microsoft Azure technologies.
Build and manage data ingestion, transformation, and processing pipelines to integrate data from multiple enterprise applications and source systems.
Develop and maintain structured data layers within the data lake, ensuring data is organized, accessible, and optimized for downstream consumption.
Work extensively with Azure Data Lake Storage (ADLS Gen2), Azure Data Factory (ADF), Azure SQL, and other relevant Azure data services.
Implement efficient data storage, partitioning, transformation, and processing strategies to improve performance and scalability.
Support the development and maintenance of enterprise data models catering to financial, operational, and management reporting requirements.
Evaluate existing data architecture and recommend improvements to enhance performance, reliability, and maintainability.
SQL Development & Data Transformation
Develop, maintain, and optimize complex SQL queries, stored procedures, views, functions, and data transformation logic.
Design and implement robust ETL/ELT processes for large-volume enterprise datasets.
Perform SQL query tuning, performance optimization, and database troubleshooting to ensure efficient data processing.
Develop reusable data transformation components and standardized data engineering frameworks.
Translate complex business requirements into technical data structures, transformation rules, and data models.
Ensure accurate handling of historical data, incremental loads, data reconciliation, and changes in source data structures.
Troubleshoot complex data issues involving inconsistent source data, transformation logic, and downstream reporting discrepancies.
Data Integration & Pipeline Management
Design and implement automated data pipelines to support integration between enterprise systems, data lake environments, and downstream applications.
Work with structured and semi-structured data from databases, APIs, flat files, and enterprise applications.
Develop and manage scheduled and event-driven data ingestion processes.
Implement appropriate error handling, logging, monitoring, and alerting mechanisms across data pipelines.
Ensure timely availability of data for business-critical reporting and analytics platforms.
Investigate and resolve data pipeline failures, processing delays, and integration issues.
Coordinate with source-system teams to manage changes in data structures, interfaces, and integration requirements.
Data Quality, Validation & Governance
Establish and implement data quality frameworks covering completeness, accuracy, consistency, and integrity of information.
Develop automated validation and reconciliation mechanisms between source systems, data lake layers, and downstream reporting applications.
Identify root causes of data discrepancies and implement sustainable technical solutions.
Maintain appropriate documentation of data pipelines, transformation logic, source-to-target mappings, and data lineage.
Implement data access controls and security practices in alignment with organizational information security policies.
Support data governance initiatives, including standardization of data definitions and master data across systems.
Ensure engineering solutions comply with applicable data privacy, security, and retention requirements.
Enterprise Analytics & Reporting Enablement
Collaborate with business and technology teams to understand enterprise reporting and analytics requirements.
Develop and maintain data structures that support financial planning, business performance reporting, and operational dashboards.
Support integration of data lake information with downstream applications such as Power BI, Anaplan, and other enterprise analytics platforms.
Work with large datasets covering financial and operational metrics such as revenue, billing, collections, workforce, utilization, costs, and profitability.
Optimize datasets and data transformation processes to improve reporting performance and data refresh timelines.
Support the implementation of new reporting dimensions, business hierarchies, and changes in organizational structures.
Provide technical support for data reconciliation and issue resolution during implementation, testing, and production operations.
Technical Delivery & Team Management
Independently lead the technical design, development, testing, and implementation of data engineering solutions.
Manage multiple development initiatives and ensure delivery within agreed timelines and quality standards.
Review SQL code, pipeline designs, data models, and technical solutions developed by team members.
Provide technical guidance and mentoring to data engineers and other members of the development team.
Establish and enforce coding standards, documentation practices, version control, and deployment processes.
Collaborate with IT infrastructure, application development, data architecture, and business teams to resolve technical dependencies.
Proactively identify technical risks, performance bottlenecks, and areas requiring architectural improvements.
Ensure appropriate testing, release management, and production support processes are followed.
Continuous Improvement & Automation
Identify opportunities to automate manual data processing, reconciliation, and operational activities.
Develop reusable frameworks and utilities to improve engineering productivity and reduce development effort.
Implement improvements to reduce data processing time, infrastructure costs, and recurring operational issues.
Evaluate emerging Azure technologies and data engineering capabilities for potential adoption.
Drive continuous improvement in data pipeline reliability, scalability, observability, and maintainability.
Key skills required
8–12 years of overall experience in data engineering, database development, data integration, or related technology roles.
Minimum 4–5 years of hands-on experience working with Microsoft Azure data engineering technologies.
Strong experience in SQL development, query optimization, and enterprise database management.
Proven experience designing and implementing end-to-end ETL/ELT pipelines and data lake solutions.
Experience working with large-scale enterprise datasets and integrating information from multiple source systems.
Prior experience managing technical deliverables and providing guidance to small engineering teams.
Experience in financial services, professional services, or enterprise finance and operations reporting would be advantageous.
Previous experience in a Big 4 organization or a large multinational enterprise would be an added advantage.
Mandatory Technical Skills
Microsoft Azure Data Engineering
Azure Data Lake Storage Gen2 (ADLS Gen2)
Azure Data Factory (ADF)
Azure SQL Database / SQL Server
Azure data integration and pipeline orchestration
Data lake architecture and data processing frameworks
Pipeline monitoring, troubleshooting, and performance optimization
SQL & Database Development
Advanced SQL and T-SQL
Complex queries, joins, CTEs, and window functions
Stored procedures, functions, and views
Query performance tuning and execution plan analysis
Database design, indexing, and optimization
Data validation and reconciliation
Data Engineering & Integration
ETL/ELT architecture and development
Data ingestion and transformation
Data warehousing and dimensional modelling
Incremental loading and change data capture concepts
Data quality management
Source-to-target mapping and data lineage
Integration with APIs, databases, and flat files
Handling large datasets and complex transformation logic
Good-to-Have Technical Skills
Azure Databricks
Python and PySpark
Apache Spark
Microsoft Fabric
Azure Synapse Analytics
Delta Lake and Medallion Architecture
Power BI datasets and semantic models
Azure DevOps, Git, and CI/CD pipelines
REST APIs and application integration
Experience integrating data with SAP, ERP, or enterprise finance systems
Knowledge of cloud security, access management, and infrastructure cost optimization