Senior Software Engineer
Splunk (Cisco) · San Francisco
Jun 2020 — present
- ▸Resolved an Aurora connection-overhead bottleneck that crashed clusters under load by introducing AWS RDS Proxy for backend connection pooling — raised per-cell ingest from ~50 TB/day to a sustained 75 TB/day (headroom to ~110 TB/day), unblocking scale-out to 15+ production cells.
- ▸Drove Pulsar → SQS migration with a custom Java SQS wrapper — exposed perf-test-derived throughput tunables, batched messages per tenant to fit SQS's 64 KB compressed billing unit, and decommissioned legacy Pulsar topics; cut messaging infra cost by ~70%.
- ▸Built and shipped a new Risk Scoring system for Splunk Enterprise Security end-to-end — backend scoring on ML baseline-model output plus a React + Python Splunk app that surfaces scored anomalies as insider-threat alerts for SOC analysts; rolled out to all ~80 ES customers and demoed at Splunk .Conf Innovation Labs.
- ▸Led design and rollout of a high-throughput micro-batching service (approved through CCAB/CCF) — consumes JSON events from SQS, writes compressed Parquet to S3 with dual size/time flush and Delta-style partitioned, column-sorted layouts that let downstream Spark EMR jobs skip row groups and prune partitions; instrumented end-to-end with Prometheus + Grafana for oncall.
- ▸Performance-tuned JVM microservices on Kubernetes end-to-end — built a Python harness that spins up K8s pods to simulate customer event-rate ingest, identified throughput bottlenecks, and tuned HPA to scale on custom Prometheus signals (queue depth, thread busyness) instead of CPU/memory defaults.
- ▸Built reusable Terraform modules — multi-region deployment patterns and an automated AWS Secrets-Manager credential-rotation flow — standardizing infra across teams and cutting ~4 hrs/month of manual rotation toil per engineer.
- ▸Built team-facing DX tooling — a CI/CD-integrated Cypress E2E framework gating PRs against UI regressions, plus a Mend Renovate dependency bot that auto-bumps versions across a 3-tier internal package cascade (dependency-manager → common libs → microservices) on upstream publish.
- ▸Own oncall response for the team's 5–6 product services end-to-end — incident triage, SLA restoration (latency, error rate, freshness), cross-team war-room coordination, P0 post-incident reviews, runbook authorship, and alert tuning for noise reduction.
- ▸Built tooling to accelerate oncall onboarding — packaged team admin-action runbooks into reusable skills, and prototyped a Claude-backed oncall assistant with RAG over runbooks, past incidents, and PIRs so new oncalls can self-serve answers instead of paging senior engineers.
- ▸Regular presenter in Splunk's CCAB / CCF architecture reviews — pitch designs to PM, UX, and peer engineering teams, and drive approved designs end-to-end through delivery.
- ▸Mentor 4 junior engineers on the team — own their design reviews, code feedback, and promo prep; one mentee promoted to the next level.
JavaPythonReactAWSTerraformKubernetesSparkPrometheusGrafanaClaudeRAG