Job ID: JOB_ID_8518
Job Summary:
This role is focused on building and operating robust, scalable data pipelines that ingest and process 30+ TB of data daily, primarily using Python on Google Cloud Platform (GCP). The engineer will collaborate closely with business partners, researchers, and trading teams to onboard high-value datasets that directly power systematic trading and research workflows. The ideal candidate is highly hands-on, production-focused, and comfortable operating in a high-performance, data-intensive environment.
Essential Skills:
- 3+ years of professional experience as a Data Engineer or in a similar role
- 3+ years of hands-on experience building ETL pipelines in production environments
- Strong Python programming skills for data processing and pipeline development
- Practical experience with cloud-based data platforms, preferably Google Cloud Platform (GCP)
- Solid understanding of data operations, including ingestion, processing, storage, quality, and lifecycle management
- Strong SQL skills and familiarity with data modeling concepts
Nice-to-Have Skills:
- Experience with Snowflake as a cloud data warehouse
- Exposure to Spark or other distributed data processing frameworks
- Familiarity with Lakehouse concepts (Delta Lake or similar formats)
- Experience with event-driven or streaming data pipelines
- Background working with financial, market, or alternative datasets
- Knowledge of data observability, lineage, and governance tooling
Key Responsibilities:
- Work closely with business stakeholders to understand data requirements and usage patterns
- Collaborate with engineers, researchers, and portfolio managers to onboard new and complex datasets
- Design, build, and support production-grade ETL and data ingestion pipelines using Python
- Operate and scale data pipelines running on Google Cloud infrastructure
- Ensure strong standards around data quality, reliability, monitoring, and operational support
- Handle large-scale batch data ingestion volumes (30TB+ per day)
- Extend and enhance the existing data onboarding framework to support new data sources and formats
- Troubleshoot and resolve pipeline failures and data quality issues in production
- Contribute to documentation, operational runbooks, and engineering best practices
The candidate should be able to work from the office 5 days in a month.
Special Requirements
Onsite 5 days in a month
Compensation & Location
Salary: $70,000 – $120,000 per year (Estimated)
Location: New York, NY
Recruiter / Company – Contact Information
Recruiter / Employer: Arthur Grand Technologies
Email: mrichard@arthurgrand.com
Recruiter Notice:
To remove this job posting, please send an email from
mrichard@arthurgrand.com with the subject:
DELETE_JOB_ID_8518