Job ID: 2757
Role Overview
Nukasani Group Inc is currently seeking a highly proficient and dedicated Big Data Engineer for a long-term contract position. This role is designed for a technical expert who can support Cobb Systems Group in their mission to build and optimize large-scale data platforms. As a Big Data Engineer, you will be at the forefront of designing distributed processing systems that handle massive datasets, ensuring that the organization can leverage data-driven insights for strategic decision-making. This position is strictly onsite, with locations available in either Rockville, Maryland, or Tysons Corner, Virginia. Candidates must be local to the Maryland, Virginia, or Delaware (MVD) area to be considered for this opportunity.
Key Responsibilities and Duties
- Design, develop, and maintain robust, large-scale data pipelines using industry-leading technologies such as Apache Spark, Hadoop, Hive, and Trino.
- Architect and implement distributed platforms capable of processing and analyzing petabyte-scale datasets with high efficiency.
- Build scalable solutions for the entire data lifecycle, including ingestion, complex transformations, storage, and downstream analytics.
- Optimize existing data pipelines to improve performance, enhance scalability, reduce operational costs, and ensure maximum reliability.
- Implement comprehensive automated testing frameworks and continuous validation processes to maintain the highest standards of data quality.
- Develop and execute unit, integration, and end-to-end test strategies specifically tailored for complex data platforms.
- Collaborate closely with cross-functional teams, including data scientists, analysts, and product managers, to translate business needs into technical solutions.
- Support data science initiatives by delivering production-ready datasets that are optimized for machine learning and advanced analytics.
- Monitor and troubleshoot production data pipelines, resolving issues promptly to minimize downtime and data loss.
- Apply advanced Spark tuning techniques, such as partitioning strategies, caching, and broadcast joins, to optimize resource utilization.
- Promote and adhere to strong software engineering practices, including version control (Git), code quality reviews, and CI/CD automation.
- Stay abreast of emerging trends in big data, cloud computing, and artificial intelligence to continuously evolve the data architecture.
- Drive the adoption of AI-enabled development practices, including the use of prompt engineering and AI-assisted coding tools.
- Ensure all data solutions comply with regulatory requirements, corporate governance, and financial data integrity standards.
Required Technical Qualifications
- A Bachelor’s degree in Computer Science, Information Systems, or a related technical discipline is required; a Master’s degree is preferred.
- Minimum of 5 years of professional experience in designing and implementing big data solutions and distributed systems.
- Deep expertise in Apache Spark, including a thorough understanding of executors, stages, DAGs, and task scheduling.
- Hands-on experience with the Hadoop ecosystem, Hive, and Trino for large-scale data processing.
- Proficiency in programming languages such as Python, Scala, or Java for developing scalable and modular code.
- Advanced SQL skills, with the ability to write complex joins, window functions, and sophisticated aggregations.
- Proven experience working with large-scale datasets and troubleshooting performance or scalability bottlenecks.
- Direct experience with AWS cloud services, specifically S3, EMR, Glue, Lambda, and Athena.
- Strong understanding of CI/CD pipelines and automated testing methodologies within a data engineering context.
- Familiarity with Agile methodologies, including Scrum and Kanban, to manage project workflows effectively.
Preferred Skills and Experience
The ideal candidate will also possess experience in the financial services sector or other highly regulated industries. Exposure to Kubernetes (EKS) or serverless architectures is a significant plus. We value candidates who are proactive in using AI-assisted development tools like GitHub Copilot or ChatGPT to enhance productivity. Strong communication skills are essential, as you will be interacting with various stakeholders to define and deliver critical data infrastructure. This project offers a long-term engagement with a focus on cutting-edge technology and professional growth within a collaborative environment.
Special Requirements
Onsite requirement. Candidates must reside in Maryland, Virginia, or Delaware (MVD) area. Submission requires last 5 digits of SSN, DOB (mm/dd), and Work Authorization details.
Compensation & Location
Salary: $145,000 – $195,000 per year (Estimated)
Location: Rockville, MD
Recruiter / Company – Contact Information
Recruiter / Employer: Nukasani Group Inc
Email: bhavani@nukasanigroup.com
Recruiter Notice:
To remove this job posting, please send an email from
bhavani@nukasanigroup.com with the subject:
DELETE_2757