Job ID: JOB_ID_548
Role Overview
VBeyond Corporation is seeking a highly skilled and hands-on Lead Data Engineer to spearhead the development of advanced Azure-based data platforms for our premier client in Chicago, IL. This is a pivotal leadership role requiring a deep technical understanding of modern data stack components, specifically Snowflake and Python. As a Lead, you will not only contribute to the codebase but also guide the architectural direction and ensure the scalability of data pipelines.
Key Responsibilities
- Lead the end-to-end development and implementation of Azure-based data platforms utilizing ADLS, ADF, and Databricks.
- Design and optimize complex data pipelines using Snowflake, Snowpipe, and SQL to ensure high-performance data processing.
- Develop robust streaming solutions using Kafka-based architectures and EventHub for real-time data ingestion.
- Implement CI/CD automation workflows to streamline the deployment of data infrastructure and code.
- Perform advanced query optimization and performance tuning within Snowflake environments.
- Evaluate and implement data transformation tools, specifically comparing and justifying the use of DBT against alternative solutions.
- Mentor junior developers and provide technical oversight on complex engineering challenges.
- Collaborate with stakeholders to translate business requirements into technical specifications.
Technical Requirements
- Extensive experience with Snowflake architecture, including Snowpipe, Triggers, and Tasks.
- Deep understanding of the differences between virtual warehouses and serverless compute within Snowflake.
- Proficiency in Python and PySpark for large-scale data processing and transformation.
- Hands-on experience with Azure cloud services (ADLS Gen2, Azure Data Factory, Azure Databricks).
- Strong background in streaming technologies such as Kafka and EventHub.
- Proven track record in query optimization and managing large-scale data warehouses.
- Experience with DBT (Data Build Tool) for modular data modeling.
Professional Expectations
The ideal candidate will possess strong communication skills and the ability to lead cross-functional teams. You should be comfortable working in a fast-paced environment and have a passion for staying current with emerging data technologies. This role requires a local presence in Chicago to facilitate close collaboration with the core engineering team. We are looking for a visionary who can balance immediate delivery needs with long-term architectural integrity.
Special Requirements
Local candidates preferred; requires expertise in Snowflake and Azure domain.
Compensation & Location
Salary: $165,000 – $215,000 per year (Estimated)
Location: Chicago, IL
Recruiter / Company – Contact Information
Recruiter / Employer: VBeyond Corporation
Email: shraddhap@vbeyond.com
Recruiter Notice:
To remove this job posting, please send an email from
shraddhap@vbeyond.com with the subject:
DELETE_JOB_ID_548