Job ID: JOB_ID_1416
Role Overview
We are seeking a highly skilled Database Databricks Architect to join our team in a remote capacity. This senior-level position requires a Databricks expert with 10 to 12 years of experience in architecting enterprise-grade data solutions. You will be at the forefront of designing and delivering scalable, high-performance cloud data platforms on AWS, specifically tailored for the Capital Markets domain. As the financial industry increasingly relies on real-time data and advanced analytics, your expertise in Databricks and AWS will be instrumental in driving our data strategy forward. This role offers the flexibility of remote work while providing the challenge of working on complex, high-stakes financial data systems.
Key Responsibilities
- Architect and design comprehensive cloud-based data platforms leveraging Databricks on the AWS ecosystem.
- Build and optimize scalable data architectures, including Lakehouse, Delta Lake, and Spark-based solutions.
- Define and implement data models and integration standards to support enterprise-wide analytics and reporting.
- Provide strategic architectural guidance and mentorship to data engineering and analytics teams.
- Ensure the highest standards of data performance, reliability, security, and governance across all platforms.
- Collaborate closely with business and technical stakeholders to ensure data solutions align with organizational objectives.
- Lead the transition of legacy data systems to modern, cloud-native architectures.
- Develop and maintain automated data pipelines using AWS Glue, Lambda, and other serverless technologies.
- Implement robust monitoring and alerting for data platforms to ensure high availability and performance.
- Stay current with the latest advancements in Databricks and AWS, evaluating new features for potential adoption.
- Optimize Spark jobs and Databricks notebooks for maximum efficiency and cost-effectiveness.
Required Technical Skills
- Expert-level proficiency in Databricks, including Apache Spark, Delta Lake, and performance tuning.
- Strong experience with AWS services such as S3, EC2, Lambda, IAM, and Glue.
- Advanced knowledge of RDBMS concepts, complex SQL, and data modeling techniques.
- Proficiency in programming languages including Python and/or Scala.
- Experience with Unix/Linux operating systems and shell scripting.
- Familiarity with CI/CD pipelines and DevOps practices for data engineering.
- Understanding of data security best practices, including encryption at rest and in transit.
Domain and Soft Skills
- Extensive experience in the Capital Markets domain, understanding trading, risk, and regulatory reporting.
- Exceptional communication and stakeholder management abilities, with the capacity to explain complex technical concepts to non-technical audiences.
- Proven leadership skills with the ability to make critical architectural decisions under pressure.
- Ability to work effectively in a fully remote, distributed team environment, maintaining high levels of productivity and collaboration.
- Strong analytical and problem-solving skills, with a focus on delivering high-quality results.
Modern Data Architecture Trends
The shift towards the Data Lakehouse architecture is transforming how organizations manage and analyze their data. In this role, you will be a key architect of this transformation, helping our clients move away from traditional data warehouses towards more flexible and scalable solutions. Your work will enable advanced analytics, machine learning, and real-time reporting, providing a competitive edge in the fast-paced world of Capital Markets. We are looking for a thought leader who can not only design these systems but also inspire others to embrace the future of data.
Data Governance and Security
In the financial sector, data governance and security are paramount. You will be responsible for ensuring that all data solutions comply with industry regulations and internal security policies. This includes implementing fine-grained access controls, data lineage tracking, and robust auditing mechanisms. Your expertise will help build trust in our data platforms and ensure that sensitive financial information is protected at all times.
Collaborative Culture
Despite being a remote role, we emphasize a highly collaborative culture. You will participate in regular architectural reviews, brainstorming sessions, and knowledge-sharing forums. We believe that the best solutions come from diverse perspectives and open communication. Joining our team means working with some of the brightest minds in data engineering and finance to solve some of the industry’s most challenging problems.
Special Requirements
Domain: Capital Markets (Trading, Risk, Regulatory reporting). Remote role.
Compensation & Location
Salary: $185,000 – $240,000 per year (Estimated)
Location: Remote
Recruiter / Company – Contact Information
Recruiter / Employer: Global IT Family LLC
Email: amishra@globalitfamily.com
Recruiter Notice:
To remove this job posting, please send an email from
amishra@globalitfamily.com with the subject:
DELETE_JOB_ID_1416