LLMOps & Evaluation: Build systematic evaluation harnesses to track retrieval precision, extraction accuracy, and hallucination rates. Set up comprehensive observability for token spend, quality drift, and overall system health.
API & Infrastructure Ownership: Drive senior-level software development by designing, building, and maintaining robust APIs. Understand DevOps best practices and manage scalable deployments utilizing Kubernetes.
Technical Delivery: Contribute hands-on to the codebases making up the retrieval, extraction, and orchestration platform. Drive reliable, high-quality delivery through planning, code reviews, and pairing. Write technical tickets to bridge functional intent and execution, and collaborate with tech leads to ensure cross-team alignment.
Core Requirements
Experience: 6+ years of software engineering experience, including 2+ years specifically building, operating, and scaling LLM-backed systems in production.
Core Skills: Strong Python expertise (e.g., FastAPI) and solid fundamentals in cloud infrastructure (Azure/GCP), CI/CD, and modern architecture.
AI/Search Tech: Hands-on experience with vector databases (pgvector, Qdrant, Elasticsearch, etc.), LLM provider APIs, and orchestration frameworks such as LangGraph or Google ADK.
Applied AI Knowledge: Proven track record of owning production RAG systems, complex document extraction pipelines, and multi-step agentic workflows.
Operational Mindset: Deep understanding of LLM versioning, rate limiting, cost control, and automated evaluation for non-deterministic systems.
Bonus: Experience in ESG, data-intensive applications, or regulated SaaS.
What We Offer
Impact: Advance Novisto’s mission to drive economic, environmental, and social sustainability through better ESG data management.
Compensation & Equity: Stock option plan in an early-stage, well-financed company.
Flexibility & Time Off: Remote-friendly flexible schedules, 4 weeks of vacation, and summer hours.
Health & Wellness: 100% insurance coverage and access to Dialogue’s telemedicine app.