Provide technical leadership to clients in a team that designs and develops pathbreaking large-scale cluster data processing systems. Mentor sophisticated organizations on large-scale data and analytics and work with client teams to deliver results. Additionally the Senior Data Engineer assists Think Big Analytics establish thought leadership in the big data industry by contributing white papers, technical commentary and representing the company at industry conferences.
• Designing and developing code, scripts and data pipelines that leverage structured and unstructured data integrated from multiple sources
• Software installation and configuration
• Participating in and helping lead requirements and design workshops with our clients
• Developing project deliverable documentation
• Leading small teams of developers and coordinating development activities.
• Mentoring junior members of the team in software development best practices
• Experience with Linux, SQL, NoSQL, relational database design and methods for efficiently retrieving data, strongly desired.
• Other duties as assigned
Must have 4-10+ years experience programming in Java, Python or C/C++, demonstrate proven expertise in production software development. Must be proficient in Linux, SQL, relational database design and methods for efficiently retrieving data. Must also have experience building complex and non-interactive systems (batch, distributed, etc.) Must have strong analytical skills and be a creative problem-solver. Must possess excellent verbal and written communications skills and be a strong team player capable of working in a demanding start-up environment.
Preferred Knowledge, Skills
Prior consulting experience is strongly desired. Proficient in NoSQL is strongly desired. Experience with Hadoop, Hive, Pig, Avro, Thrift, Protobufs as well as JMS: ActiveMQ, RabbitMQ, JBoss, etc. Dynamic and/or functional languages experience such as Python, Ruby, Scala, and Clojure are also desirable. Experience designing and tuning high performance systems and prior experience with data warehousing and business intelligence systems. A professional or academic background that includes mathematics, statistics, machine learning and data mining is a plus. Prior work and/or research experience with unstructured data and data modeling as well as familiarity with different development methodologies (e.g. agile, waterfall, XP, scrum, etc.)
Asia Pacific/Japan-Australia-Australian Capital Territory-Canberra