Senior Data Engineer
Sodexo's Digital & Data Factory's foundation is based on a unique mix of skills: strong business knowledge, data science / software/ cloud expertise, and agile methods day to day practice. This allows us to deliver projects with strong business and operational value thanks to teams composed of the best experts in their fields.
The performance of our data services is based on a real technical expertise in AI, software engineering, data engineering and site reliability engineering. We aim at ensuring an end-to-end product vision from the delivery of a software solution to its run in our production environment.
We are searching for a Senior Data Engineer who will participate in the design, development, and evolution of data pipeline, aiming at serving data to data consumers part of the product team. Then, you will work in partnership with data scientists and data analysts to deliver the monitoring and the alerting settings. You are on the first line to ensure every product is running smoothly and support feature teams in the issue resolution.
You will be implementing Data Engineering best practices at a Digital & Data Factory level to ensure a high level of service to very diverse tools from a functional and technical standpoint.
This is a hybrid position, we are seeking candidates located in the Irvine, CA region.
The successful candidate will:
- Design and implement batch and real-time data ingestion pipelines.
- Use Databricks (with PySpark) for big data processing tasks.
- Clean, transform, and enrich raw data to make it analytics ready.
- Design and implement database schemas, tables, and views.
- Optimize storage formats for querying, such as Parquet or Delta Lake.
- Enforce data quality checks and data lineage documentation.
- Implement partitioning, bucketing, and indexing strategies for efficient data retrieval.
- Work closely with data scientists to provide data in appropriate formats for machine learning and advanced analytics.
- Assist data analysts with SQL queries, views, and report generation.
- Collaborate on the deployment of machine learning models to production.
- Implement role-based access controls and data encryption (at-rest and in-transit).
- Comply with industry and organizational data standards, privacy regulations, and best practices.
- Regularly audit data access and usage.
- Set up and maintain Azure cloud infrastructure.
- Configure and optimize Azure Data Lake Storage, Blob Storage, and other Azure storage solutions.
- Deploy and manage Databricks clusters for processing tasks.
- Implement and maintain data pipelines using Azure Data Factory.
- Monitor and troubleshoot infrastructure-related issues.
- Onboard new team members, providing access and initial training on tools.
- Create documentation and knowledge bases for data pipelines, best practices, and tooling.
- Stay updated with the latest advancements in data engineering technologies.
- Propose and implement optimizations for current workflows and systems.
- Proactively identify areas of improvement and automation.
- Regularly update team on new features or changes in Azure, Databricks, or related technologies.
Is this the right opportunity for you? We are looking for candidates that have/are:
- Strong coding skills in Python, especially with libraries related to data manipulation (e.g., Pandas) and interfacing with databases. Very important to master software engineering techniques: unit testing, integration and e2e testing, design patterns, …
- Advanced knowledge of SQL for data querying, transformation, and aggregation.
- Ability to design scalable and robust data pipelines considering functional and non-functional requirements.
- Knowledge of data architectures to ensure reliable and efficient data flow.
- Proficiency in Azure Data Lake, Azure Data Factory, Azure Blob Storage, Azure SQL Database, and other related Azure services.
- Familiarity with setting up, configuring, and managing virtual networks, VMs, security groups, and related components on Azure.
- Mastery of PySpark for data processing, including the RDD and DataFrame APIs.
- Understanding of Delta Lake format for reliable data lakes.
- Knowledge of both relational (SQL Server) and NoSQL databases (like Cosmos DB).
- Familiarity with different data storage formats such as Parquet, JSON, CSV, and Avro.
- Skill in designing, deploying, and managing data integration solutions with ADF.
- Ability to design efficient and scalable database schemas for both operational and analytical use cases. Should know Dimensional Modelling, 3NF (Third Normal Form) Data Modeling and Data Vault Modeling
- Knowledge in designing Extract, Transform, Load (ETL) processes
- Model, transform and test data on specific data products
- Skills in optimizing complex SQL queries
- Knowledge in optimizing data processing pipelines, particularly in Databricks/Spark.
- Knowledge of encryption techniques, both at-rest and in-transit
- Understanding of role-based access controls and integration with Azure Active Directory.
- Familiarity with tools like Azure Resource Manager (ARM) templates or Terraform is a plus
- Experience with continuous integration and continuous deployment tools like Azure DevOps.
- Basic understanding of Docker and Kubernetes, especially as they might integrate with Azure services is a plus.
- Familiarity with tools like Azure Monitor, Log Analytics, or other monitoring solutions.
- Ability to troubleshoot and debug issues within data pipelines and storage systems.
Sodexo offers a full array of benefits including paid time off, holidays, medical, dental, vision, 401K and access to ongoing training and development programs, tuition reimbursement, plus health and wellness programs.
Not the job for you?
At Sodexo, we have numerous IS&T positions that support this and other initiatives with similar goals. Continue your search for IS&T jobs.
Working for Sodexo:
Sodexo fosters a culture committed to the growth of individuals through continuous learning, mentoring and career growth opportunities. Our IS&T team supports 13,000 locations across North America and collaborates with the entire Sodexo Group, spanning 72 countries. Sodexo empowers its employees who have developed a thorough understanding of the organization to create their own career path
What We Offer
Sodexo offers fair and equitable compensation, partially determined by a candidate's education level or years of relevant experience. While the budgeted range for the position is posted, Sodexo salary offers are based on a candidate's specific criteria, like experience, skills, education and training.
In collaboration with a cross-functional feature team (AI Product Manager, Data Engineer, DevOps, etc), the Data Engineer Data Lab objective is to design and build scalable and reliable ML/AI solutions integrating and optimizing algorithms at every stage of the Product life cycle (e.g. MVM, MVP, Industrialization, etc).
- Provide support during the scoping and Minimum Viable Model phases
- Co-realize the data and tech due diligence and SoTA* analysis
- Guarantee correct exploratory environment setup
- Provide workload estimation on the product backlog to the AI Product Manager
- Design and co-implement functional blocks to ensure a scalable and reliable ML/AI solution all along the product life cycle
- Design and build pipelines on an ad hoc basis
- Measure cost effectiveness of the different transformation pipelines scenario
- Embed monitoring metrics into products-Design MVP target architectures-Package, document code and infrastructure
- Optimize ML/AI algorithm performances in distributed systems (e.g. model response time, optimization of the parallelism, etc)
- Drive innovation within Sodexo recommending & testing new AI/ML technologies
- Conduct technological watch
- Test new technologies in the context of use cases MVMs & MVPs
- Communicate on new market technologies which could bring value to Sodexo
- Participate to selecting new service providers (e.g. RFP)
Qualifications & Requirements
Basic Education Requirement: Bachelor’s Degree or equivalent experience
Basic Management Experience: 0–7 years
Basic Functional Experience: 7 years
Sodexo is an EEO/AA/Minority/Female/Disability/Veteran employer.