Data Engineer II
American Express · Banking & Financial Services
- Gurugram, HR, India
- On-site
- Posted today
- Apply by 29 Oct
- Data & AI
About the job
Data Engineer II facilitates seamless data integration and model maintenance, enhances data pipelines and database designs, supports production environments, ensures compliance with enterprise standards, and fosters collaboration while implementing basic partitioning and indexing solutions for Big Data management.
Evaluates data requirements and stories, documents them for seamless integration into existing architectures, and maintains data models to support business needs
Builds and enhances data pipelines and database designs to meet performance, scalability, and security requirements
Should have good hands on with GCP technologies like : BigQuery, DataProc and other services of GCP.
Have good working knowledge of SQL and data evaluation tools.
Collaborates in design reviews and testing, and provides production environment support with guidance from peers and leaders
Completes work reviews and fosters a collaborative learning environment with guidance from peers and leaders
Communicates and collaborates with business and product teams to facilitate changes and implementation
Completes Big Data/Spark and Other distributed frameworks requirements by implementing basic partitioning and indexing solutions
Collaborates and co-creates effectively with teams in product and the business to align technology initiatives with business objectives
Education Qualifications:
Bachelor’s degree in Computer Science, Computer Engineering, and/or comparable experience; advanced degree preferred
Knowledge of Big Data / NoSQL design and development with variety of data stores.
Knowledge of database management system products and ecosystem – e.g., storage formats, access algorithms, data management processes, administration, data maintenance, replication, high availability, encryption, etc.
Knowledge of distributed (multi-tiered) systems, algorithms, and relational & non-relational database
Work Experience:
Data engineering and 4+ years of experience in development.
Have good experience in performance tuning of large data processing in both batch and real-time.
Experience in SQL and Python (or any other programming language like Java)
Experience in design and development across one or more database management systems (e.g., HBase, Hive, GCP tools, NoSQL, BigTable, BigQuery, Cloud Spanner) as appropriate.
Experience in NoSQL databases (e.g., Hbase, BigTable)
Experience in performance tuning in the Big Data space
Experience in distributed (multi-tiered) systems, algorithms, and application development
Experience in database development and support of online transaction processing and/or online analytical processing systems
Experience in software delivery methodology (Agile, Scrum, Agile XP, Test-Driven Development etc.)
GCP Certificate will be good add-on.