Job Information
Cargill Senior Platform Engineer-Data Discovery in Bengaluru, India
Cargill’s size and scale allows us to make a positive impact in the world. Our purpose is to nourish the world in a safe, responsible and sustainable way. We are a family company providing food, ingredients, agricultural solutions and industrial products that are vital for living. We connect farmers with markets so they can prosper. We connect customers with ingredients so they can make meals people love. And we connect families with daily essentials — from eggs to edible oils, salt to skincare, feed to alternative fuel. Our 160,000 colleagues, operating in 70 countries, make essential products that touch billions of lives each day. Join us and reach your higher purpose at Cargill.
Job Purpose and Impact
The Platform Engineer will take the lead in building the data and analytics platform for modern business applications in the company. In this role, you will apply your in-depth knowledge of modern infrastructure and cloud software engineering practices to build, secure and maintain the core capabilities used by our data and application teams to drive business value. You will also coach and mentor junior engineers to deliver highly scalable and resilient systems using infrastructure as code across our data centers and cloud environments. This role will specifically work in the discoverability space designing and maintaining the Data Discovery Platform which consists of a metadata lake, data catalog, data mesh register, and data marketplace. The Data Discovery Platform team will lead efforts to ensure data and data products are accessible and well-organized which will aid in better governance and compliance.
Key Accountabilities
Build the platforms, systems and infrastructure using in depth knowledge of software development and infrastructure as code practices for the Data Discovery Platform and Metadata Lake.
Partner closely with Product & Engineering teams, proactively identifying areas of opportunity for the Data Discovery Platform and Metadata Model.
Keep up-to-date with regulatory requirements that influence Data Discovery.
Take the lead to design, develop, test, deploy, support and enhance the complex and varied automated infrastructure and platform components.
Take the lead to drive large efforts, stories and tasks to completion.
Participate in the engineering community to maintain and share relevant technical approaches and modern skills and present best code practices.
Build prototypes to test new concepts and provide ideas on reusable framework and components to help promote adoption of new technologies.
Independently handle complex issues with minimal supervision, while escalating only the most complex issues to appropriate staff.
Works across engineering teams to plan installations and upgrades and ensures subsequent maintenance in accordance with IT controls and policies.
Monitors Data Discovery Platform to achieve optimum performance levels.
Collaborates with engineering teams by providing technical input for workflows and configurations relating to the access, security, and protection of data.
Other duties as assigned
Qualifications
MINIMUM QUALIFICATIONS:
Bachelor’s degree in Computer Science, Information Systems, or related field.
Minimum of four years experience in data engineering and operationalizing data ingestion pipelines.
Minimum of four years experience with data integration efforts from both on-premise and cloud to targets both on-premise and cloud.
Strong understanding of database storage concepts (data lake, relational databases, NoSQL, Graph, data warehousing).
Experience with Git / Azure DevOps.
Proficiency in understanding modern data architectures and concepts such as cloud services (AWS, Azure, GCP), real-time data distribution and processing (Kafka, Flink), and modern data warehouse tools sql (Snowflake, Databricks).
Programming knowledge in languages such as SQL, Python, Java, Scala, or equivalent
PRIOR BENEFICIAL EXPERIENCES
Design, deploy and enhance moderately complex automated infrastructure and platform components
Implementation or management of discoverability tools and practices – data catalog, metadata lake
Knowledge of master data, metadata, reference data, and business intelligence principles and processes
Knowledge of data governance and compliance
Experience with CI/CD and other automation for cloud deployments
Familiarity with infrastructure as code
Cloud infrastructure management and use of serverless technologies (AWS preferred)r
Equal Opportunity Employer, including Disability/Vet.
Cargill
-
- Cargill Jobs