In order to execute our vision, we need to grow our team of best-in-class data engineers. We are looking for developers who conduct impeccable data practices and implement high quality data infrastructures. We value hard workers who are comfortable improvising solutions to big data challenges while building a system that can stand the test of time. Our ideal candidate has experience building data infrastructure from the ground up, contributes innovative ideas and ingenious implementations to the team, and is capable of planning out scalable, maintainable data pipelines.
As a data engineer, you would at first work primarily on our Hive Media product, taking real-time data from hundreds of television streams and turning them into a combination of real-time and scheduled outputs, especially our signature ads feed. Your work would improve the quality of our results while reducing computational cost and latency. Expect truly novel challenges.
- Writing scheduled Spark pipelines that perform sophisticated queries on the entirety of our datasets
- Writing real-time pipelines that execute complex operations on incoming data
- Synchronizing large amounts of data between unstructured and structured formats on various data sources
- Creating testing and alerting for data pipelines
- Building out our data infrastructure and managing dependencies between data pipelines
- Defining and implementing metrics that provide visibility into our data quality
- An undergraduate and / or graduate degree in computer science or a similar technical field, with a sound understanding of statistics
- 1-2 years of industry experience as a data engineer
- Hands-on experience doing ETL and have written data pipelines in either Spark, Hadoop, or similar technologies
- A sound understanding of SQL
- You have worked with data lakes such as S3 or HDFS
- You have worked with various databases, such as Postgres, Cassandra, or Redshift before, and understand their pros and cons
- A working knowledge of the following technologies, or are not afraid of picking them up on the fly: Mesos, Chronos, Marathon, Jenkins
- Fluent in at least one scripting language (preferably NodeJS or python) and one compiled language (such as Scala, Java, or C)
- Great communication skills and ability to work with others
- Strong team player with a do-whatever-it-takes attitude
Who We Are
We are a group of ambitious individuals who are passionate about creating a revolutionary AI company. At Hive, you will have a steep learning curve and an opportunity to contribute to one of the fastest growing AI start-ups in San Francisco. The work you do here will have a noticeable and direct impact on the development of the company.
Thank you for your interest in Hive and we hope to meet you soon!