Software Engineer ShyftLabs
Built a multi-provider LLM gateway with an OpenAI-compatible API that routes across more than 250 models from over 45 providers, including OpenAI, Anthropic, Gemini, Vertex AI, and AWS Bedrock. Implemented cost-aware auto-routing, model tier selection, prompt complexity classification, capability filters, and per-organization model controls.
Developed a model routing registry with 120 entries, quality scoring by task complexity, automatic cooldowns for unhealthy or rate-limited models, and budget controls that shift traffic toward lower-cost models as usage grows. Added LLM evaluation, automated quality scoring, and normalized provider responses for tool calls and token usage.
Deployed more than 10 MCP tool servers for agent workflows, including image generation, code execution, data visualization, web search, and workplace-tool connectors. Integrated vector retrieval with batched ingestion and retries for internal knowledge search.
Operated multi-agent workloads on Kubernetes across development and production environments with tenant isolation, IAM-scoped access, RBAC, and OIDC single sign-on. Established observability with OpenTelemetry, Prometheus, Grafana, and Sentry, and implemented GitHub Actions CI/CD for backend, frontend, Helm, AWS, and Azure Kubernetes releases.