Job ID: JOB_ID_1944
Role Overview
We are seeking a highly skilled Lead Data Engineer to join our dynamic team in Austin, TX. This is a leadership-heavy technical role where you will be responsible for designing, building, and optimizing large-scale data pipelines and warehousing solutions. The ideal candidate will have a deep understanding of modern data architecture and the ability to lead complex projects from inception to delivery while collaborating with offshore teams.
Technical Requirements
- SQL Mastery: Hands-on experience writing complex SQL queries using Joins, Self Joins, Views, Materialized Views, and Cursors. Proficiency in SQL performance tuning and the use of GROUP BY and PARTITION BY functions is mandatory.
- Data Modeling: Extensive experience with ETL processes and Dimensional Data Modeling, including Slowly Changing Dimensions (Type 1, 2, and 3).
- Python Ecosystem: Expert-level Python scripting skills with a focus on Pandas, PyParsing, and Apache Airflow. Experience setting up scripts on DataLab and connecting with DataLake/S3 is required.
- Snowflake Architecture: Deep understanding of Snowflake design aspects, performance tuning, time travel, warehouse concepts, scaling, and clustering. Experience with SnowSQL and SnowPipe is essential.
- Streaming & Messaging: Proficiency in Apache Kafka using client libraries like Confluent’s librdkafka or kafka-python to produce and consume messages for streaming applications.
Key Responsibilities
- Lead the development of automated data pipelines and microservices.
- Interact directly with business stakeholders to understand requirements and perform thorough impact analysis.
- Own overall project delivery and provide technical guidance to junior engineers and offshore partners.
- Optimize Snowflake and Vertica/SingleStore environments for maximum performance.
- Ensure data integrity and scalability across all data platforms.
Preferred Qualifications
Candidates local to the Austin, TX area are strongly preferred. A valid LinkedIn profile must be submitted with the application. Experience with Vertica or SingleStore is a significant plus.
Special Requirements
Submission must have LinkedIn profile; Preference for candidates local to Austin, TX; Experience collaborating with offshore teams required.
Compensation & Location
Salary: $165,000 – $215,000 per year (Estimated)
Location: Austin, TX
Recruiter / Company – Contact Information
Recruiter / Employer: KK Software Associates
Email: ajay.e@kksoftwareassociates.com
Recruiter Notice:
To remove this job posting, please send an email from
ajay.e@kksoftwareassociates.com with the subject:
DELETE_JOB_ID_1944