Note: The job is a remote job and is open to candidates in USA. Mayo Clinic Rochester is seeking a talented Senior Data Engineer to join their Advanced Data Lake team. The role focuses on building and operating enterprise data Lakehouse platforms and architecting automated data pipelines for large-scale analytics and digital transformation.
Responsibilities
- Build and operate enterprise data Lakehouse platforms that support large-scale analytics and digital transformation
- Architect and maintain automated data pipelines for ingesting, transforming, and integrating complex datasets
- Use DataStream for real-time data movement and Dataflow for processing at scale
- Leverage Composer/Airflow for seamless scheduling, monitoring, and automation of pipeline operations
- Handle infrastructure provisioning and workflow management with Terraform and Dataform
- Manage all code and pipeline assets through git repositories, with CI/CD automation and streamlined releases enabled by Azure DevOps (ADO)
- Govern changes by ServiceNow processes to ensure traceability, auditability, and operational compliance
- Work with cross-functional teams to translate business needs into pipeline specifications
- Build and optimize data models for advanced analytics
- Maintain data quality and security throughout all processes
- Automate workflow monitoring and proactively resolve data issues
Skills
- A Bachelor's degree in a relevant field such as engineering, mathematics, computer science, information technology, health science, or other analytical/quantitative field and a minimum of five years of professional or research experience in data visualization, data engineering, analytical modeling techniques; OR an Associate's degree in a relevant field such as engineering, mathematics, computer science, information technology, health science, or other analytical/quantitative field and a minimum of seven years of professional or research experience in data visualization, data engineering, analytical modeling techniques
- In-depth business or practice knowledge will also be considered
- Ability to manage a varied workload of projects with multiple priorities and stay current on healthcare trends and enterprise changes
- Interpersonal skills, time management skills, and demonstrated experience working on cross functional teams
- Strong analytical skills and the ability to identify and recommend solutions and a commitment to customer service
- Excellent verbal and written communication skills, attention to detail, and a high capacity for learning and problem resolution
- Advanced experience in SQL
- Strong Experience in scripting languages such as Python, JavaScript, PHP, C++ or Java & API integration
- Experience in hybrid data processing methods (batch and streaming) such as Apache Spark, Hive, Pig, Kafka
- Experience with big data, statistics, and machine learning
- Ability to navigate linux and windows operating systems
- A GCP Professional Data Engineer certification is required
- Knowledge of workflow scheduling (Apache Airflow Google Composer)
- Infrastructure as code (Kubernetes, Docker)
- CI/CD (Jenkins, Github Actions)
- Experience in DataOps/DevOps and agile methodologies
- Experience with hybrid data virtualization such as Denodo
- Working knowledge of Tableau, Power BI, SAS, ThoughtSpot, DASH, d3, React, Snowflake, SSIS, and Google Big Query
- Google Cloud Platform (GCP) certification
- Hybrid or multi-cloud experience
- Familiarity with enterprise data governance, metadata, and lineage tools
- Experience working in large, regulated environments
Benefits
- Medical: Multiple plan options.
- Dental: Delta Dental or reimbursement account for flexible coverage.
- Vision: Affordable plan with national network.
- Pre-Tax Savings: HSA and FSAs for eligible expenses.
- Retirement: Competitive retirement package to secure your future.
Company Overview