- Work with large data sets, experience working with distributed computing (MapReduce, Hadoop, Hive, Pig, Apache Spark, etc).
- Profile and analyze source data to determine the best reporting structures to build.
- Design and develop ETL pipelines using multiple sources of data in various formats according to business requirements.
- Conduct ETL, SQL and DB performance tuning, troubleshooting, support, and capacity estimation to ensure highest data quality standards.
- Conduct dimensional modelling, metadata management, data cleaning and conforming, and warehouse querying.
- Use sound agile development practices (code reviews, testing, etc) to develop and deliver data products
- Provide day-to-day support and technical expertise to both technical and non-technical teams
- Work with other engineers to brainstorm solutions to problems and support others in their goals.
- Exhibit sound judgement, keen eye for details and tenacity for solving difficult problems.
- Use strong analytical skills and support use of data for sound decision making.
- Help us build data expertise and data focused mindset throughout the enterprise
- Translate business needs into technical requirements
- Degree in Engineering, Computer Science or Mathematics/Statistics.
- 5+ years industry experience with big data technology (Hadoop, Hive, Spark, SQL, Kafka, etc.)
- Industry experience as a Data Engineer
- Data modelling and warehousing experience
- Proficiency with relational databases (Oracle, DB2, Redshift, etc.)
- Strong proficiency in SQL
- API development experience
- Experience working in an Agile team environment