Job ID: 3189374
Job Overview:
We are seeking a skilled and experienced Kafka Administrator to join our dynamic team. The Kafka Administrator will be responsible for the installation, configuration, management, and monitoring of Apache Kafka clusters. This role is crucial for ensuring the high availability, security, and optimal performance of our Kafka environment. You will manage all aspects of the Kafka ecosystem, including brokers, topics, and partitions, and collaborate closely with other engineering teams to integrate Kafka with various systems and applications, facilitating real-time data streaming and ingestion.
Key Responsibilities:
- Kafka Cluster Management: Install, configure, and maintain Kafka clusters, implementing strategies for high availability and disaster recovery to ensure business continuity.
- Performance Optimization: Continuously monitor Kafka cluster performance, identify potential bottlenecks, and fine-tune configurations to achieve optimal throughput and low latency.
- Security Management: Implement and manage robust security configurations, including access control (ACLs), authentication mechanisms (SASL, SSL/TLS), and data encryption to protect sensitive information.
- Monitoring and Alerting: Set up, configure, and maintain comprehensive monitoring and alerting systems (e.g., Prometheus, Grafana, Kafka-specific tools) to proactively identify, diagnose, and address potential issues before they impact production.
- Troubleshooting: Diagnose and resolve complex issues within the Kafka ecosystem, including performance degradation, connectivity problems, data loss, and configuration errors.
- Integration: Collaborate effectively with development, data engineering, and other IT teams to integrate Kafka with other systems and applications, supporting data ingestion pipelines and real-time data streaming initiatives.
- Documentation: Create, update, and maintain detailed documentation for Kafka configurations, operational procedures, best practices, and troubleshooting guides.
- Automation: Develop and implement automation scripts (e.g., using Python, Bash) to streamline routine cluster management tasks, deployments, and maintenance activities.
- Capacity Planning: Assess current and future resource requirements (CPU, memory, disk, network) for the Kafka environment, ensuring the cluster can scale effectively to meet growing demands.
- Collaboration: Work closely with cross-functional teams to understand their data streaming needs and provide Kafka expertise and support.
Qualifications:
- Proven experience as a Kafka Administrator or in a similar role managing large-scale Kafka clusters.
- Strong understanding of Apache Kafka architecture, components, and best practices.
- Experience with Kafka installation, configuration, and maintenance.
- Proficiency in monitoring Kafka clusters using tools like Prometheus, Grafana, or similar.
- Experience with Kafka security features, including authentication and authorization.
- Scripting skills (e.g., Bash, Python) for automation.
- Familiarity with cloud environments (AWS, Azure, GCP) is a plus.
- Excellent troubleshooting and problem-solving skills.
- Strong communication and collaboration abilities.
Compensation & Location
Salary: $100,000 – $150,000 per year
Location: New York, NY
Recruiter / Company – Contact Information
Recruiter / Employer: GAC Solutions
Email: suhel@gacsol.com
Recruiter Notice:
To remove this job posting, please send an email from
suhel@gacsol.com with the subject:
DELETE_3189374