Senior Data Engineer ‎- Atlanta, GA

Position Overview

Cargill’s size and scale impact the world by nourishing it in a safe, responsible, and sustainable manner. As a family company that connects farmers to markets and customers to ingredients, Cargill delivers essential products to billions of lives each day. In the role of Senior Professional, Data Engineering, you will design, build, and maintain complex data systems that enable actionable insights. With minimal supervision, you will ensure that large sets of data are processed efficiently and made accessible for decision making.

Key Responsibilities

• Prepares data infrastructure to support efficient storage and retrieval of data.
• Examines and resolves appropriate data formats to improve usability and accessibility.
• Develops complex data products and solutions using advanced engineering and cloud-based technologies, ensuring scalability, sustainability, and robustness.
• Builds and maintains streaming and batch data pipelines for seamless data ingestion, transformation, and storage in data lakes or warehouses.
• Reviews existing data systems and architectures to identify areas for improvement and optimization.
• Collaborates with multi-functional data and analytic teams to gather requirements and ensure data solutions meet various partners’ needs.
• Builds prototypes to test new concepts and implements data engineering frameworks that enhance processing capabilities.
• Develops automated deployment pipelines to enhance the efficiency of code deployments with proper governance.
• Performs complex data modeling to ensure sustainable performance and accessibility.

Required Qualifications

• Minimum of 4 years of relevant work experience (typically 5 years or more).
• Note: The business will not sponsor applicants for work visas for this position.

Preferred Qualifications

• Experience developing data systems on major cloud platforms (AWS, GCP, Azure).
• Hands-on experience building modern data architectures, including data lakes, data lakehouses, and data hubs, along with capabilities such as ingestion, governance, modeling, and observability.
• Proficiency in data collection and ingestion tools (Kafka, AWS Glue) and storage formats (Iceberg, Parquet).
• Experience developing data pipelines with streaming architectures and tools (Kafka, Flink).
• Expertise in data transformation and modeling using SQL-based frameworks and orchestration tools (dbt, AWS Glue, Airflow) with knowledge of SCD and schema evolution.
• Strong background in using Spark for data transformation, including streaming, performance tuning, and debugging.
• Advanced programming skills in Python, Java, Scala, or similar languages; expert-level proficiency in SQL.
• Experience in DevOps practices, including code management, CI/CD, and deployment strategies.
• Solid understanding of data governance principles, encompassing data quality, privacy, and security.