Job ID: JOB_ID_7608
Job Title: Software Engineer (Big Data / Hadoop developer)
We are seeking a skilled Software Engineer with strong experience in large-scale data systems and distributed processing. The ideal candidate will design, build, and optimize robust data pipelines and real-time streaming architectures that power critical business insights. Experience with Big Data technologies like Hadoop, Spark, Flink, and Kafka is essential.
Key Responsibilities:
- Design and develop scalable data pipelines for batch and real-time processing.
- Build and maintain streaming applications using modern data processing frameworks.
- Collaborate with cross-functional teams to define data requirements and deliver reliable solutions.
- Optimize data workflows for performance, scalability, and cost-efficiency.
- Ensure data quality, integrity, and availability across systems.
- Troubleshoot and resolve production issues in data infrastructure.
Required Skills & Technologies:
- Strong experience with Apache Spark for large-scale data processing.
- Hands-on experience with Apache Flink for real-time stream processing.
- Proficiency in workflow orchestration tools like Apache Airflow.
- Experience working with Apache Cassandra or other distributed NoSQL databases.
- Knowledge of modern data lake formats such as Apache Iceberg.
- Experience with Apache Kafka for building real-time data pipelines and event-driven architectures.
- Strong programming skills in Java, Scala, or Python.
- Experience with distributed systems and cloud platforms (AWS, GCP, or Azure preferred).
Qualifications:
- Bachelors or Masters degree in Computer Science, Engineering, or related field.
- 3+ years of experience in software engineering or data engineering roles.
- Solid understanding of data structures, algorithms, and system design.
Additional Requirements:
- Need Only Local To Bay Area Profiles.
- Ex-Apple experience is a plus.
Location:
San Francisco Bay Area (SFO)
Duration:
Contract
Special Requirements
Local to Bay Area profiles only. Ex-Apple experience is a plus.
Compensation & Location
Salary: $60 – $80 per year (Estimated)
Location: San Francisco Bay Area, CA
Recruiter / Company – Contact Information
Recruiter / Employer: Diverse Lynx
Email: l.pandey@diverselynx.com
Recruiter Notice:
To remove this job posting, please send an email from
l.pandey@diverselynx.com with the subject:
DELETE_JOB_ID_7608