Senior Lead Data Engineer - R01557349
Senior Lead Data Engineer
- Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL
- Hadoop and Spark SQL
- Key Responsibilities:
- Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL
- - Manage and optimize the infrastructure ensuring reliability and performance
- -Design and implement data models and schemas for enterprise data warehouse and data marts
- -Write and maintain scripts in Python for various applications
- -Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases
- - Work closely with analysts and product managers to understand data requirements and deliver high-quality data
- -Provide technical guidance and mentorship to other team members fostering best practices in data engineering
- -Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability
- -High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing
- -Good knowledge in Python for data manipulation and scripting
- -Mastery of advanced SQL and deep experience working with relational databases and datawarehouses
- -Hands-on experience with workflow management tools like Apache Airflow
- -Familiarity with Unix shell scripting for automation and system management
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
Lead Quality Engineer role focused on mobile and API automation, driving BDD/TDD practices and test data management for a product-focused engineering organization in Saint Louis.
Lead the design and delivery of OCC’s cloud analytics semantic layer and ETL pipelines to provide reliable, query-ready data for enterprise analytics and risk teams.
Help power international streaming analytics at NBCUniversal by developing reliable, observable batch and streaming data pipelines that enable fast business insights.
Experienced AI/ML Data Engineer needed to build and operate scalable ML data pipelines, feature/embedding stores, and model enablement infrastructure for College Board's remote Aquifer team.
Experienced AI/ML Data Engineer needed to design, build, and deploy large-scale data pipelines and models using Python, PySpark, Microsoft Fabric, Power BI, and SQL for a 12+ month contract in Berlin, NJ.
Work at MongoDB as an Analytics Engineering Intern building end-to-end analytics pipelines, writing complex SQL, and contributing to data-driven projects in a hybrid NYC environment.
Lead the design and implementation of scalable analytics and data engineering solutions that power clinical and operational insights across the University of Maryland Medical System.
AUMOVIO seeks a detail-oriented Catalog Data Specialist to manage ACES/PIES electronic catalogs, vehicle application data, and supplier/customer catalog communications for multiple product brands.
Verkada is hiring an on-site Analytics Engineer in San Mateo to build scalable dbt data models, develop Looker dashboards, and translate marketing data into growth-driving insights.
Cellares is looking for a Senior Data Quality Engineer to lead automated validation, reconciliation, and monitoring of Databricks Lakehouse pipelines that power its cell therapy manufacturing platform.
Our mission is to bring innovation to the digital world and provide the best customer experience possible through true partnerships and integration with our clients.
10 jobs
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
