Data Engineer
The main function of the Data Engineer is to develop, evaluate, test and maintain architectures and data solutions within our organization. The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organization’s data assets.
Job Responsibilities:
• Building end to end solution for data pipeline using technologies like python, Hadoop and spark.
o Identify and resolve the problem if current data pipeline fails
o Continuous improvement of a new product
o Building new code from an older project
• Communication with data scientists, strategists, product team, etc. and finding the data gap if exists.
• Will need to be flexible to assist on other projects where their expertise is required
• Will need to help Data Scientist in developing, validating and operationalizing the Models built by them
Must have:
• 4+ years of experience as a Data Engineer or related field
• 4+ years of experience with Minio, Trino, Airflow, and Kubernetes
• 4+ years of experience with Python & Spark
• 4+ years of experience with Hadoop, Shell scripting, Strong SQL knowledge,
Nice to have
• FI experience
Soft skills:
• Excellent written, presentation, and verbal communication skills
• Ability to work as part of a team, as well as work independently or with minimal direction..
• Collaborate with data architects, modelers and IT team members on project goals.
Education/Experience:
• Bachelor's degree in a technical field such as computer science, computer engineering or related field required.