MetLife

We are seeking a highly skilled, hands-on Azure Data Engineer with a strong DevOps / CI/CD mindset to build, deploy, and operate modern data pipelines on Azure.

This role emphasizes automation-first delivery for data workflows—using Azure DevOps or GitHub Actions to implement repeatable CI/CD, environment promotion, validation, and monitoring for production-grade data pipelines.

The engineer will develop and support ETL/ELT workflows using Azure Data Factory (ADF) and Azure Databricks (PySpark/SQL/Delta Lake), enabling reliable and scalable data products aligned to governance expectations.

Job Responsibilities:

CI/CD, DevOps, and Automated Delivery (Primary Focus)

• Establish and maintain CI/CD pipeline development, deployment, and testing for data workflows using Azure DevOps or GitHub Actions.

• Implement Git-based source control practices for data engineering assets (ADF pipelines, Databricks notebooks/jobs, SQL scripts), including branching and release management across environments.

• Automate deployments with validation and rollback strategies and set up monitoring/alerting for pipeline and platform health.

• Build and maintain YAML-based pipelines and standardized build/deploy patterns (e.g., reusable templates, triggers, approvals).

• Ensure production deployments follow required change governance and audit-friendly release controls.

Data Engineering & ETL/ELT Pipelines (ADF + Databricks)

• Architect, build, and maintain ETL/ELT pipelines using Azure Data Factory and Azure Databricks (PySpark, SQL, Delta Lake).

• Optimize ingestion, transformation, and storage patterns for performance, scalability, reliability, and cost efficiency.

• Implement orchestration standards, error handling, retries, and operational guardrails to meet SLA/SLO expectations.

• Develop robust transformations/aggregations to support downstream analytics and reporting needs.

Azure Platform Operations & Reliability

• Support administration and optimization of core Azure data services used by the platform (e.g., ADF, Databricks, ADLS, Azure SQL and/or Synapse where applicable).

• Monitor platform health, performance, and cost usage; implement automation to improve efficiency and reliability.

Data Governance, Security & Compliance

• Implement data access controls, auditing, and security best practices using Azure-native capabilities (e.g., RBAC, secrets management, and governance tooling such as Purview or equivalent where applicable).

Education:

• Graduate in Computer Applications / Computer Science

Experience:

• 2-4 years relevant experience

Technical Skills: