Data Engineer
Typical Day in Role:
• Guides the data architecture for creation of data assets to be used for modelling, BI, Marketing operations and other analytic functions within the scope of Canadian Banking Analytics
• Performs data engineering activities to create and maintain the data assets required. Leads the creation of data pipelines and design of ETL work to transform and manage data at scale. Responsible for the orchestration of the data pipelines / ETL processes and monitoring of these processes for accuracy, reliability and timeliness.
• Enables the migration of data stores and analytical datasets from the data warehouse and enterprise data lake to global data & analytics platform.
• Executes data management activities to design and build dataset for analytics and bring in data from multiple data platform and environments. Assemble large, complex data sets that meet functional / non-functional business requirements.
• Executes on data governance activities to ensure good data quality and integrity for critical datasets used by the CBA team across the various analytical use cases.
• Responsible for ingesting new/alternative sources of data to generate business value.
• Champions a high-performance environment and contributes to an inclusive work environment.
• Participate on special project teams as required to support the delivery of strategic Bank initiatives.
• Actively pursues effective and efficient operations of his/her respective areas, while ensuring the adequacy, adherence to and effectiveness of day-to-day business controls to meet obligations with respect to operational risk, regulatory compliance risk, AML/ATF risk and conduct risk, including but not limited to responsibilities under the Operational Risk Management Framework, Regulatory Compliance Risk Management Framework, AML/ATF Global Handbook and the Guidelines for Business Conduct.
Candidate Requirements/Must-Have skills:
1. 5-7+ years of technical experience as a Data Engineer.
2. Expert ability (5+ years) with data processing language: Python, SPARK.
3. Expert ability (5+ years) with one or more query language (SQL, PL/SQL, HiveQL, SparkSQL). Advanced SQL skills with experience in querying large and complex data sources and creation of highly optimized and efficient SQL queries.
4. Hands on experience with Big Data ecosystem tools (e.g. Hadoop, Hive, Spark) and data processing within the EDL for ‘big data’ data pipelines, architectures & data sets.
5. Experience performing root cause analysis for data issues in order to identify opportunities for improvement.
Nice-To-Have Skills:
• Hands on experience with using Data Platform such as Datameer.
• Experience with managing / functioning with ETL/ data engineering Dev Ops.
• Experience with container technologies (Docker).
• Expert ability to manage and organize information from data environments, including interpretation of the results for management presentation purposes.
Soft Skills:
• Excellent written, presentation, and verbal communication skills to be able to work well with technical peers and business stakeholders at different levels within the organization.
• Must be flexible to adapt to a dynamic environment and make quick and sound decisions under pressure.
• Must be reliable, pro-active, results-oriented, customer-focused and attentive to details.
• Must possess excellent time management and organizational skills in order to deliver initiatives in a timely manner and deal with conflicting priorities and tight deadlines.
Education:
• University Degree in STEM or related fields required.