Job ID: JOB_ID_2556
Position Summary: Senior Observability Engineer
We are seeking a Senior Infrastructure Monitoring and Observability Engineer for a long-term engagement in Harrisburg, PA. This role is critical for the modernization of enterprise monitoring systems, shifting from traditional reactive monitoring to proactive, data-driven observability. The incumbent will serve as a Subject Matter Expert (SME) on monitoring tools and processes, collaborating with agency teams and vendors to implement actionable reporting and automated workflows. This is a hybrid position requiring at least one day per week in the Harrisburg office, specifically targeting local Pennsylvania candidates.
Core Responsibilities and Duties
- Drive the evolution of monitoring practices by identifying gaps and implementing automation-first strategies to reduce manual intervention.
- Maintain and optimize endpoint monitoring connectivity across hybrid networks, managing telemetry ingestion via SNMP, WMI, APIs, and agents.
- Develop and maintain comprehensive documentation, including runbooks, SOPs, and service maps, within version-controlled repositories.
- Integrate observability context into incident management, utilizing ServiceNow to capture monitoring data and produce detailed post-incident reviews.
- Collaborate with Enterprise Change and Incident Management teams to ensure standardized risk assessments and communication plans.
- Monitor service restoration performance, tracking key metrics such as MTTR and SLAs to ensure high availability of critical applications.
- Design and execute disaster recovery (DR) plans for monitoring infrastructure, defining RTO/RPO and performing periodic validation exercises.
- Stay at the forefront of emerging technologies, including KQL, Azure Monitor, and advanced dashboarding tools like SquaredUp.
Technical Qualifications and Expertise
The successful candidate will have over 12 years of IT experience, with at least 5 years specifically focused on infrastructure monitoring and observability in hybrid cloud environments. Proficiency in PowerShell and at least one other scripting language (Python, Bash, or SQL) is required. Hands-on expertise with the Microsoft monitoring stack, including SCOM, Azure Monitor, and Log Analytics, is essential. Candidates should be well-versed in ITIL 4 practices and have experience implementing CI/CD pipelines for automation. Knowledge of API integrations and secure authentication protocols is vital for maintaining a secure and robust monitoring ecosystem. We highly value certifications such as Microsoft Certified: Azure Administrator Associate or ITIL 4 Foundation.
Strategic Impact and Environment
This role is not just about maintaining tools; it is about transforming how the organization views system health. You will be responsible for ensuring compliance with Commonwealth IT policies while recommending updates to improve reliability and cost efficiency. During catastrophic incidents, you will fulfill Continuity of Government (CoG) obligations, ensuring that critical monitoring services remain operational. This position offers the chance to work on large-scale, complex infrastructure that supports public services, requiring a professional who is dedicated to operational excellence and continuous improvement. Your work will directly impact the stability and performance of applications serving millions of users.
Special Requirements
Locals only to PA. Hybrid schedule (1 day/week onsite). Screening requires Passport Number, SSN (last 4), and DOB (MM/DD).
Compensation & Location
Salary: $145,000 – $185,000 per year (Estimated)
Location: Harrisburg, PA
Recruiter / Company – Contact Information
Recruiter / Employer: Concept Software & Services Inc
Email: manoj.k@concept-inc.com
Recruiter Notice:
To remove this job posting, please send an email from
manoj.k@concept-inc.com with the subject:
DELETE_JOB_ID_2556