Job ID: JOB_ID_11572
Role: Azure Data Engineer (Databricks, PySpark)
Location: Dallas, TX
Experience: 15+ years total IT experience, with 8+ years in Big Data & Cloud Engineering.
Availability: ASAP
Interview: Anytime
Professional Summary:
Highly experienced Data Engineer with over 15 years in IT and 8+ years specializing in Big Data and Cloud Engineering. Proven expertise in Azure Databricks, PySpark, Delta Lake, ADLS Gen2, and Azure Data Factory. Skilled in building scalable ETL pipelines, data ingestion frameworks, and data transformation solutions. Experience with Airflow, Apache NiFi, Kubernetes (AKS), and CI/CD pipelines (Jenkins). Successfully delivered enterprise data solutions for major clients including Starbucks, Citi Bank, Nike, and Wells Fargo. Possesses strong knowledge of AWS (S3, EMR, EC2) and hybrid cloud environments.
Key Skills:
- Big Data: PySpark, Spark SQL, Hadoop, Hive
- Azure Stack: Databricks, ADF, ADLS, Delta Lake, AKS
- Data Engineering: ETL Pipelines, Data Modeling, Data Quality Frameworks
- Tools: Airflow, NiFi, Jenkins, Terraform
- Databases: Oracle, PostgreSQL, SQL Server
- Cloud: Azure, AWS
- Other: CI/CD, Kubernetes
Client Experience:
- Starbucks
- Citi Bank
- Nike
- Wells Fargo
Technical Proficiencies:
- Data Processing: Spark, PySpark, SQL
- Cloud Platforms: Microsoft Azure (Databricks, ADF, ADLS Gen2, AKS), Amazon Web Services (S3, EMR, EC2)
- Data Warehousing & Lakes: Delta Lake, Data Lake Storage Gen2
- Orchestration & Workflow: Apache Airflow, Apache NiFi
- DevOps & CI/CD: Jenkins, Terraform, Kubernetes (AKS)
- Databases: Oracle, PostgreSQL, SQL Server
- Programming Languages: Python (PySpark)
Role Responsibilities:
- Design, develop, and maintain scalable and robust data pipelines using Azure Databricks and PySpark.
- Implement data ingestion processes from various sources into Azure Data Lake Storage (ADLS Gen2).
- Build and optimize ETL/ELT processes using Azure Data Factory.
- Develop and manage data models for analytical and reporting purposes.
- Ensure data quality and implement data governance best practices.
- Deploy and manage data solutions using Kubernetes (AKS).
- Implement CI/CD pipelines for data engineering workflows using Jenkins and Terraform.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
- Monitor and troubleshoot data pipelines to ensure optimal performance and reliability.
- Stay up-to-date with the latest trends and technologies in big data and cloud engineering.
Special Requirements
Work Authorization: Not specified. Relocation: Open. Interview: Anytime.
Compensation & Location
Salary: $100,000 – $140,000 per year
Location: Dallas, TX
Recruiter / Company – Contact Information
Recruiter / Employer: Ventechsoft
Email: c2chotlisth1b@googlegroups.com
Recruiter Notice:
To remove this job posting, please send an email from
c2chotlisth1b@googlegroups.com with the subject:
DELETE_JOB_ID_11572