NEWPosted 2 hours ago

Job ID: JOB_ID_3549

Job Summary

We are seeking a Databricks professional who can work across engineering and architecture responsibilities, designing, building, and scaling enterprise data solutions using Databricks and Apache Spark on cloud platforms (AWS / Azure / GCP). The role involves hands-on development as well as solution design and technical leadership.

Key Responsibilities (Combined Role)

  • Design, develop, and maintain end-to-end data pipelines using Databricks.
  • Build and optimize Spark (PySpark / Spark SQL) jobs for large-scale data processing.
  • Implement Delta Lake, data versioning, and data quality frameworks.
  • Handle batch and streaming data pipelines.
  • Architect scalable data lake / lakehouse solutions on cloud.
  • Perform Spark performance tuning and cost optimization.
  • Define data ingestion, transformation, and consumption strategies.
  • Implement CI/CD pipelines for Databricks workloads.
  • Ensure data security, governance, and access control.
  • Collaborate with stakeholders and provide technical guidance.
  • Mentor team members and lead design discussions.

Required Skills

  • Strong hands-on experience with Databricks & Apache Spark.
  • Expertise in PySpark, Spark SQL.
  • Experience with AWS / Azure / GCP cloud services.
  • Solid understanding of data lakes, data warehousing, and lakehouse architecture.
  • Experience with Delta Lake.
  • Strong SQL and data modeling skills.
  • Experience with ETL/ELT frameworks.
  • Knowledge of DevOps / CI-CD concepts.

Nice to Have

  • Databricks certification.
  • Experience with Kafka, Airflow, Azure Data Factory, Glue.
  • MLflow and machine learning pipeline exposure.
  • Infrastructure as Code (Terraform).
  • Client-facing experience.

Role Expectation by Level

  • Senior Engineer: Ownership, optimization & mentoring.
  • Architect: Solution design, scalability, governance & leadership.

Special Requirements

Remote position. No specific visa constraints mentioned. No specific screening steps or interview modes mentioned.

Compensation & Location

Salary: $100,000 – $160,000 per year (Estimated)

Location: Remote

Recruiter / Company – Contact Information

Email: info@aetalentsgroup.com

Interested in this position?
Apply via Email

Recruiter Notice:
To remove this job posting, please send an email from
info@aetalentsgroup.com with the subject:

DELETE_JOB_ID_3549

to delete@join-this.com.