…

Data Engineering Lead

Optum (UnitedHealth Group) · Healthcare & Pharma

  • Noida, Uttar Pradesh, India
  • On-site
  • Posted today
  • Data & AI

About the job

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities:

Data Modeling & Analytics Engineering - Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas
Build curated data products that support reporting, analytics, AI, and machine learning use cases
Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains
Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication
Collaborate with business stakeholders to translate requirements into analytical and reporting solutions
Databricks Platform Engineering - Design and implement modern data architectures leveraging Databricks Lakehouse principles
Develop and maintain Bronze, Silver, and Gold data layers
Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows
Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed (CDF)
Delta optimization techniques - Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions
Context Layer & Semantic Layer Development
Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics
Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms
Enable self-service analytics through Power BI, Tableau, and AI-driven applications
Manage semantic models and governance processes to ensure reporting consistency
Establish metric certification, lineage tracking, and data governance standards
Performance Optimization & Cost Management - Analyze query execution plans and identify performance bottlenecks
Optimize Spark workloads for scalability, reliability, and cost efficiency
Implement cluster sizing, autoscaling, caching, broadcast joins, and Adaptive Query Execution (AQE) strategies
Tune Delta Lake tables through file compaction, partition optimization, and storage management
Monitor warehouse and cluster utilization to improve performance and control cloud expenses
Enhance dashboard and reporting performance through optimized data models and query design
Data Governance & Quality Management - Develop and implement data quality frameworks, validation processes, and automated monitoring
Ensure compliance with enterprise security, governance, and regulatory requirements
Support metadata management, data cataloging, and lineage initiatives
Establish monitoring and alerting for data pipelines, processing failures, and data anomalies
Partner with governance teams to drive data stewardship and certification processes
Technical Leadership & Collaboration - Collaborate with data scientists, analysts, engineers, product owners, and business leaders
Mentor and guide junior analytics engineers and data engineers
Conduct architecture reviews and recommend technology best practices
Drive adoption of enterprise data standards, frameworks, and governance processes
Lead technical design discussions and influence data platform strategy and roadmap
Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications:

Graduate degree or equivalent experience
Bachelor's degree in Computer Science, Information Systems, Data Engineering, Analytics, or a related field
7+ years of experience in Data Engineering, Analytics Engineering, Business Intelligence, or related disciplines
3+ years of hands-on experience with Databricks and Delta Lake technologies
Hands-on experience with Power BI, Tableau, or similar BI platforms
Experience building enterprise-scale ETL/ELT pipelines
Experience implementing semantic layers and enterprise reporting solutions
Experience with performance tuning of Spark workloads and large-scale analytics environments
Experience with cloud platforms such as Azure, AWS, or Google Cloud
Knowledge of CI/CD, DevOps practices, and version control systems such as Git
Solid understanding of Databricks Lakehouse architecture and Medallion (Bronze/Silver/Gold) design patterns
Solid understanding of data governance, metadata management, and data quality frameworks
Solid expertise in dimensional data modeling, including Star and Snowflake schema design
Advanced proficiency in SQL and Spark/PySpark development
Proven excellent problem-solving, communication, and stakeholder management skills

Preferred Qualifications:

Master's degree in Computer Science, Data Science, Analytics, or a related discipline
Databricks certifications (Data Engineer Associate, Professional, or equivalent)
Experience with Databricks Unity Catalog and enterprise data governance implementations
Experience supporting AI/ML, Generative AI, or advanced analytics use cases
Experience designing enterprise semantic models and metric layers
Healthcare, insurance, or highly regulated industry experience
Demonstrated experience leading technical teams, mentoring engineers, and driving enterprise data strategy
Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation

Key Competencies:

Data Architecture & Modeling
Analytics Engineering
Databricks & Delta Lake
Spark/PySpark Development
Data Governance & Quality
Performance Optimization
Semantic Layer Design
Cloud Data Platforms
Technical Leadership
Stakeholder Management

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.