Job Title: Data Engineer
Location : Toronto
Duration : 6 months
Our large financial client is seeking a Sr. Data Engineer / Data Engineer reporting into the Director, Advanced Analytics and AI Engineering & Enablement Lead. Located in Toronto, Canada the role will (1) champion and support strategic &/or global data initiatives that strengthens the global data and advanced analytic capabilities, (2) foster cross-segment collaboration and communication helping build an agile data insight driven culture, and (3) lead and nurture open data design and architecture establishing conditions for successful technical and analytic innovation.
The Sr. Data Engineer / Data Engineer will develop, maintain, and test data pipelines, application framework, infrastructure for data generation; works closely with Data Scientists to enable their work using modern data architecture and tools.
We have a clear vision for a Global Data Strategy. By liberating and strengthening data capabilities we will enable deeper insights, better product and service design, and more effective business processes. The result will be exceptional experiences for our customers.
Mandate covers:
• Leveraging new & existing Big Data & Cloud technologies contributing to the innovative design, development and management of data analytics labs supporting to increase knowledge and insight from enterprise data
• Perform technical systems and data flow development in a variety of projects for complex front, middle and back office applications, with a focus on the reporting and analytics environment; this environment is built on a Microsoft Azure cloud and is underpinned by a Hortonworks Hadoop stack
• Perform technical systems and data flow design for small-to-medium sized projects
• Work with multiple project execution and deployment teams (e.g. Development, Business Analysis, Architecture, Release Management, Production Support)
• Work closely with the technical leads and architecture teams, and align solutions that meet the departmental architectural vision
• Assess the completeness and accuracy of data, identifying gaps in data, provide feedback to business and system owners with guidance and options to obtain missing information
• Design, build and implement modern data architectures in development and production environments (data orchestration pipelines, data sourcing, cleansing, augmentation and quality control processes)
• Translates business needs into data engineering and architecture solutions
• Contributes to overall solution, integration and enterprise architectures
• Build and support deploying machine learning models in development and production environments
• Provide proactive data ingestion and analysis of large structured and unstructured datasets involving a wide range of systems across Group Functions (i.e., Finance, Treasury, Risk, Human Resources, Brand & Communications)
• Evaluating existing and proposed data models and how to best access and query them as well as existing and proposed data interfaces and how to clearly document them, including specification of data flow models, data flow timing, data mapping, and data transformation rules including data validations and controls
Required Knowledge and Skills:
• Demonstrated 2-5 years of professional experience in related industry experience in working in bigdata/data management & understanding big data analytic tooling and environments including a University degree and or Master’s degree in Engineering, Computer Science or equivalent quantitative program
• Experience in Big Data, Analytics and Business Intelligence technologies to support design, build and implementation for advanced analytics and business intelligence reporting;
• Experience working with Cloudera and/or Hortonworks Hadoop stack
• Experience with big data processing frameworks and techniques such as HDFS, MapReduce, Syncsort, Sqoop, Oozie, Storage formats (Avro, Parquet), Stream processing (NiFi, Kafka), etc.
• Understanding of relational and warehousing database technology working with Hadoop and other major databases platforms (e.g., Hadoop, Oracle, SQLServer, Teradata, MySQL, or Postgres)
• Experience in data technologies and use of data to support software development, advanced analytics and reporting. Focus on Cloud (Azure), Hadoop-based technologies and programming or scripting languages like Java, Scala, Linux, C++, PHP, Ruby Python, R and SAS.
• Knowledge regarding different databases such as Hawq/HDB, MongoDB, Cassandra or Hbase.
• Working knowledge of modern data streaming using Kafka, Apache Spark and data ingestion frameworks: NiFi, Hive and Pig
• Experience writing complex SQL and NoSQL jobs to analyze data in both traditional DBMS (MS-SQL, Oracle) and Big Data environments (i.e., HADOOP, SPARK, or similar open source and commercial technologies)
• Knowledge of non-relational (Cassandra, MongoDB) databases preferred
• Predictive analytics and machine learning experience (scikit-learn, Tensorflow, MLlib, recommendation systems) preferred
• Experience with integrating to back-end/legacy environments
• Experience integrating business and technology teams
• Knowledge and familiarity with machine learning models application and production pipelines
• Collaborative attitude, willingness to work with team members; able to coach, participate in code reviews, share skills and methods
• Remains current with emerging technologies, innovations and practices within the data and analytics industry
• Good organizational and problem-solving abilities that enable you to manage through creative abrasion
• Good verbal and written communication; effectively articulates technical vision, possibilities, and outcomes
• Strong work ethic, results oriented, and accuracy / attention to detail are critical; ability to work in agile or scrum delivery environments
• Exceptional oral, written and interpersonal communication skills with the ability to simplify complex technical concepts into business & value-focused language. A key requirement is to communicate clearly and consistently keeping stakeholders well-informed of progress and challenges
• Excellent organizational and time management skills, strong business presence with ability to multi-task and effectively deal with competing priorities. Ability to work with minimal or no supervision while performing duties; has the ability and initiative to organize various functions and be a strong team player.