Job description

Design, Develop, Test and maintain efficient, scalable data pipelines and ETL process.
Translate customer requirements to develop effective, robust data solutions in line with customer Data Engineering standards
Work with other Data Engineers and Architects in tuning and optimizing the data pipelines/ETL processes
Effective stakeholder engagement at all stages of the Data Engineering project lifecycle, from pre-feasibility through to delivery
Implement processes to monitor performance and data quality

R equirements

Must have experience in developing, building and deploying solutions based on Microsoft Azure SQLDW, TSQL, Databricks, Data Lake, Azure Data Factory, Blob Storage Spark/Spark SQL, Azure Logic Apps, Azure Log analytics.
Demonstrable experience and understanding of end-to-end data delivery techniques including data modelling, ETL building, Quality Assurance and performance tuning
Experience in designing and building cloud based ETL/ELT pipelines and data migrations and dealing with high volume systems Good understanding of monitoring and tuning data pipeline workloads, preferably in ADF/ Azure Synapse Experience in building pipelines and loading data into Azure SQL Data Warehouse / Azure Synapse or SQL Server BI Toolset (SRSS, SSIS, SSAS)
An in-depth understanding of data management (e.g., permissions, recovery, security and monitoring) and capable of choosing the right data stores/data processing for the job (relational and non-relational stores, batch, and Real Time processing)
Experience in dealing with different data schemas (including Microsoft SQL Server i.e., MSSQL, NoSQL, API Based) is required
Experience of working with MI / BI Analysts / Visualisers to understand their requirements
Experience in agile delivery methodologies, CI/CD practices
Any knowledge of the following is desirable: Teradata, Azure pipelines, Azure DevOps and Linux.

Role:

Data EngineerIndustry Type:

Full Time, PermanentRole Category:

Data Science & Machine Learning

EducationUG:

Any GraduatePG:

Any Postgraduate

Job Category: IT Jobs

Job Type: Full Time

Job Location: Hyderabad

Azure Data Engineer