Big Data Engineer
Position Description:
The Big Data Engineer will be a key member of the Field Services BI team. You will work closely with Hadoop Administrators, Data Scientists, and business stakeholders.
Job Duties/Accountabilities:
- Maintain and optimize the data infrastructure required for accurate extraction, transformation, and loading of data from a wide variety of data sources
- Works to identify gaps and improving the platform’s quality, reliability, robustness, maintainability, and speed
- Contributes design, code, configurations, and documentation for components that manage data ingestion, real time streaming, batch processing, data extraction, transformation, and loading across multiple systems
- Manage service desk inquiries by debugging, suggesting, implementing changes and/or improvements following business guidelines and procedures
Critical Qualifications/Competencies:
- Training/Education
- Bachelor’s degree in Engineering, Computer Science, Operations Research, Science, Mathematics or equivalent experience and / or education
- Skills
- Autonomous, resourceful, strong tolerance to ambiguity
- Strong analytical, communication and time management skills
- Strong troubleshooting skills
- Excellent ability to articulate and document technical concepts
- Technical Knowledge
- Experience working with distributed data technologies (e.g. Hadoop, MapReduce, Spark, Kafka)
- Experience building streaming and batch jobs in Spark
- Experience with Enterprise class ETL tools such as IBM DataStage
Working Conditions:
- Ability to work efficiently in virtual teams with geographically dispersed team members
- Fast paced, continuous learning environment with new technologies
- Good work / life balance opportunities