Big Data Engineer

Job Title: Big Data Engineer
Contract Duration: 6 months + extension
Location Address: WFH (Toronto, ON)

This Contractor will help to develop and manage this data platform under the leadership of the technical lead.

Project Summary: This initiative is for setting up a data platform to land and ingest all source data that is required for finance and risk applications.

Candidate Value Proposition
This is from the ground up and starting from scratch – building out a new team. You will be part of something new, which gives rise to a lot of opportunities to learn and grow within the bank.
Our team will be using state of the art open source technologies.

Job Responsibilities:
– Responsible for the building, monitoring, tuning, and troubleshooting the data platform E.g. installation and configuration, tooling integration, process automation, and platform management.
– Test, monitor, administer, optimize, and operate multiple big data clusters on premise and/or in cloud.
– Documents and maintains operational procedures and processes. (Experience with Confluence

Must Have Skills
1. 3+ years of experience setting up, configure and manage, monitoring, integrating data platforms using Hadoop, Ranger, Hive Metastore & NiFi.
2. 3+ years of experience with security authentication, authorization, and encryption specifically AD/LDAP integration
3. 1+ year of experience with S3 object storage technology. E.g. MinIO
4. 4+ years of DBA experience with RDBMS: Oracle and PostgreSQL DB support.
5. 3+ years of experience troubleshooting data platform performance and capacity bottlenecks.
6. 3+ years of DevOps experience using technologies such as Ansible Tower, Ansible, Docker, Git, Jenkins.
7. 3-5 years of experience with Linux shell scripting and system administration.
8. Strong communication skills

Nice-To-Have Skills:
1. Experience setting, configure and manage data platforms using Kafka, Presto
2. Familiarity with networking, firewalls and load balancing.
3. NoSQL is an asset
4. Experience with Agile methodologies such as confluence, Jira and scrum
5. Banking or FI Experience

Degrees or certifications
– Bachelor's degree in a technical field such as computer science, computer engineering or related field