Lead I - DevOps Engineering
UST · IT Services & Consulting
- Chennai, Bangalore
- On-site
- Posted yesterday
- IT & Infrastructure
About the job
8+ years in data platform engineering (DataBricks), DataOps, platform engineering, DevOps or SRE. • At least 5+ years of hands-on Terraform and IaC experience, including modules, state management, imports, plans, provider and version control, environment separation, drift detection and CI/CD. • Strong Python engineering for APIs, automation, testing, packaging, structured logging, error handling and reusable tooling. • Production experience operating Databricks and Unity Catalog. • Strong production Apache Airflow experience. • Experience operating dbt environments, jobs and CI/CD workflows. • AWS IAM, S3, VPC/private connectivity, KMS, Secrets Manager and CloudWatch. • Git, CI/CD, REST APIs, SQL, Linux, OAuth, SAML/SCIM and service-principal authentication. • Experience with monitoring, incident response, change control, rollback and cost management. • Relevant Databricks platform training or certification is highly preferred. Responsibilities • Provision and manage platform resources through reusable Terraform modules and configuration as code. • Operate Databricks accounts and workspaces, including identities, groups, service principals, entitlements, compute policies, clusters, SQL warehouses and Jobs/Workflows. • Configure Unity Catalog metastores, catalogs, schemas, external locations, storage credentials, workspace bindings, ownership and least-privilege access. • Operate dbt projects, environments, connections, jobs, permissions, service tokens, CI quality gates and promotion controls. • Engineer production Astro/Airflow capabilities covering DAG standards, dependencies, retries, timeouts, concurrency, backfills, ing and safe recovery. • Orchestrate and integrate Fivetran, dbt, Databricks and Atlan using APIs, webhooks and workload identities. • Implement Fivetran usage monitoring, cost thresholds, non-production lifecycle guardrails and configuration portability. • Enable governed Atlan and Tableau connectivity, metadata, lineage, access and operational workflows. • Implement ADLC controls for branching, testing, approval, environment promotion, deployment evidence and rollback. • Integrate identity and access workflows with Okta or equivalent identity providers and ServiceNow. • Establish platform and pipeline SLOs; monitor reliability, schema changes, data quality, usage and cost. • Troubleshoot incidents, perform root-cause analysis and automate recurrence prevention. • Maintain runbooks, configuration inventories, access evidence and recovery procedures