Job Information
CloudIngest Inc. Lead Data Engineer, Multiple Positions in Alpharetta, Georgia
Lead Data Engineer, Multiple Positions: Alpharetta, GA, and various unantic client sites thruout U.S. Resp: Implement and mge Multinode Hadoop clusters on Cloudera virtual machines, collaborating w/Hadoop admin team to configure user groups; Develop ETL pipelines to extract, transform, and load data from source datalakes using Python, Spark, and Hive, as well as migrating data from on-premises Oracle and SQL Server to Hadoop servers using Sqoop and Spark; Script w/ PySpark and Python to automate validation, logging, and alterations for Spark apps, along w/develop shell wrapper scripts for automation; Write OOZIE Workflow scripts for job orchestration, develop Terraform scripts for deploy Cloud Function configs and pub/sub topic creation in the GCP platform, and optimize and monitor performance of Spark apps. Reqs min of MS or equiv in CS, CIS, IT-related Engineering, or related, w/1 yr of exp in position offered or rel. Extended travel and/or relocation thruout U.S. Mail resumes: CloudIngest Inc., Job LDE, 310 Maxwell Rd., Suite 600, Alpharetta, GA 30009.