IT Data Engineer

Job Summary

An agricultural technology organization is hiring a Senior Data Engineer to architect and build distributed analysis capabilities that process large-scale scientific and operational datasets across agricultural systems. This senior-level role focuses on designing scalable data architectures, developing data-intensive APIs, and implementing real-time stream processing solutions to accelerate crop system development decisions. The position offers the opportunity to apply advanced software engineering expertise while collaborating cross-functionally to deliver innovative, high-impact solutions in sustainable agriculture.

Responsibilities

• Design and implement distributed analysis capabilities for large-scale, diverse datasets across agricultural operations
• Architect, build, and maintain data-intensive RESTful APIs supporting critical business functions
• Develop and optimize real-time stream processing pipelines using Apache Kafka
• Build and maintain containerized application deployments using technologies such as Docker
• Create and manage scalable cloud-based infrastructure on major cloud platforms
• Apply advanced knowledge of algorithms and data structures to improve system performance and drive innovation
• Model data architectures for large-scale relational and NoSQL databases, optimizing for performance and scalability
• Collaborate with cross-functional teams to identify optimal technology stacks for varying dataset requirements
• Contribute to technical presentations and publications at industry conferences
• Mentor team members and promote best practices in software development methodologies

Qualifications

Required:

• Bachelor’s degree in Computer Science, Software Engineering, or related technical field, or equivalent practical experience
• Minimum 7 years of professional software engineering experience
• At least 2 years of hands-on experience with Go programming language
• Proven experience building and maintaining data-intensive RESTful APIs
• Demonstrated experience with stream processing using Apache Kafka
• Practical experience creating and maintaining containerized applications
• Proven ability to build and maintain cloud-based infrastructure on AWS, Azure, or Google Cloud Platform
• Experience with data modeling for large-scale relational and NoSQL databases
• Advanced proficiency in Go
• Strong expertise in RESTful API development
• Proficiency in unit testing and Test Driven Development methodologies
• Deep understanding of algorithms and data structures
• Experience with containerization technologies and deployment strategies
• Cloud infrastructure design and management capabilities
• Strong problem-solving abilities for complex technical challenges
• Excellent collaboration and communication skills