Sr. Azure Data Engineer

Job highlights
Bachelor’s degree in Information Systems, Computer Science, or Engineering; 8+ years in ETL processes; 5+ years with Azure ETL tools
Design and implement data frameworks and pipelines; analyze and validate data sharing requirements; prepare project deliverables

Job description

Skills

   Azure Data Factory, Azure Data Lake, Azure SQLData Warehouse, Azure Databricks, Azure Synapse, Spark, Python, Azure AnalysisServices, Azure DevOps

   Experience working in ETL process for at least 8 years

   Experience developing ETL process using AzureETL tools for at least 5 years

Qualifications

   Bachelors degree in information systems,Computer Science, Engineering, or related field

   Proficient in designing and implementing datasourcing and transformation processes (ETL) in a large, distributed environmentusing cloud services e.g., Azure Data Factory,Azure Data Lake, Azure SQL Data Warehouse, Azure Databricks, Azure Synapse,Spark, Python, Azure Analysis Services and Azure DevOps

   Hands-on experience with developing complex ETLpipelines that ingest and transform data from various source systems into Azurecloud operational databases and data warehouse environment

   Proficient in developing and operationalizingvarious data distribution patterns such as APIs, event based, publish/subscribemodels

   Well versed with data models, source targetdefinitions, identify/fixing discrepancies adhering to enterprise datagovernance standards

   Adept with Azure DevOps-CI/CD (ContinuousIntegration and Continuous Deployment) and Test-Driven Development practices

   Design, develop and implement scalable batch /real-time data pipelines to integrate data from a variety of sources into anAzure database / data warehouse and data lake

   Experience migrating data lakes from on-premisesto cloud

   Experience with agile delivery using AzureDevOps

   Experience monitoring performance of dataworkflows and infrastructure through use of monitoring tools and enablingalerts

   Functional knowledge of data visualization tools

Responsibilities

   Design and implement data frameworks andpipelines to process data from on-premises and cloud data sources to feed intothe Azure Cloud, monitor data quality and implement related controls

   Evaluate existing designs, improve methods, andimplement optimizations

   Analyze and validate data sharing requirementswithin and outside data partners

   Analyze and interpret collected data;identifying trends; writing reports and recommendations for internal orexternal clients

   Use statistical practices to analyze current andhistorical data to make predictions, identify risks, and opportunities enablingbetter decisions on planned/future events

   Work with business leadership to understand datarequirements; propose and develop solutions that enable effectivedecision-making and drive business objectives

   Prepare project deliverables that are valued bythe business and present them in such a manner that they are readily understoodby project stakeholders

   Prepare advanced project implementation plansusing Agile methodologies, which highlight milestones and deliverables,leveraging standards, practices, and work planning tools

   Recognize potential issues and risks during theanalytics project implementation and recommend mitigation strategies

   Document best practices for data analysis, dataengineering, and evangelize their usages

   Work with stakeholders, including the executivesas well as product engineering teams to assist with data-related technicalissues and support their data infrastructure needs

   Create data tools for data scientist team toassist them in building and optimizing our product into an innovative industryleader

   Work with data and analytics experts to strivefor greater functionality in our data capabilities

   Experience inoffshore/ onsite coordinationRole & responsibilities

Preferred candidate profile

Perks and benefits

Role: IT Support – Other
Industry Type: IT Services & Consulting
Department: IT & Information Security
Employment Type: Full Time, Permanent
Role Category: IT Support
Education
UG: Any Graduate
Key Skills
Skills highlighted with ‘‘ are preferred keyskills
pythonAzure DatabricksAzure Data LakeETL
Datafactory