Senior Data Engineer (Spark) - Big Data Startup
These products will accompany the startup’s core product, a self-service big data governance and insights platform that is used by employees at enterprise clients. The platform is essentially an Excel spreadsheet on steroids for non-technical users to parse and manipulate large data sets.
The Senior and Lead Data Engineer will be building streaming ingestion pipelines to connect with data lakes at their clients. These pipelines will bring in new data which is then processed by Spark for analytics.
The company is approaching 1 PB of data. This would be entirely green field development. And you’d use all modern technologies with the language of your choice.
Required Skills & Experience
- Proficient in Scala, Java, or Python
- Strong experience with Spark
Desired Skills & Experience
- Understanding of AWS cloud environment
- Experience with Kafka, SQS, or similar
- Exposure to Flink or other streaming big data technologies
- Experience with Scala
- MS or PhD in CS or mathematics related discipline
What You Will Be Doing
- 100% new development in language of choice – some existing Spark jobs in Java
- 100% open source
- End-to-end project-based design and implementation of new ingestion pipelines
- 100% hands-on, flat hierarchy, first engineers hired for new product team will
- Competitive salary: Up to $200K per year, depending on experience
- Equity package
- Well-funded startup with paying Fortune 100 clients
- New development of two new high-profile products
Applicants must be currently authorized to work in the United States on a full-time basis now and in the future.
Workbridge Associates, part of the Motion Recruitment network, provides IT Staffing Solutions (Contract, Contract-to-Hire, and Direct Hire) in major North American markets. Our unique expertise in today’s highest demand tech skill sets, paired with our deep networks and knowledge of our local technology markets, results in an exemplary track record with candidates and clients.