Benchmarks & Evaluation
Contribute to APIFlow-Bench, our open-source benchmark for real API-development work: design and review benchmark tasks and their mock API environments, extend the evaluation harness and task-generation pipeline in Python, and help maintain the public multi-model leaderboard with statistical confidence intervals.
Help build a new action-level AI safety benchmark: instead of grading what a model says, it scores what an agent actually does inside a simulated enterprise API environment. You’ll work on scenario design, threat modeling (prompt injection, data exfiltration, permission overreach), and auditable evaluation design.
Model Training & Efficiency
Fine-tune open-weight models for tool calling and agentic tasks (SFT, distillation, and RL) using PyTorch and the open-source training ecosystem, on both managed training platforms and self-managed cloud GPUs.
Design and run experiments with rigor: evaluate every training run on our benchmarks, support ablation studies and error analysis, track experiments, and report results honestly, including cost.
Evaluate ultra-low-bit quantized models for on-device use: extend our quantized vs. full-precision benchmark comparisons and analyze where and why they diverge.
Agent Systems & Engineering Practice
Help build the next generation of Postman’s in-product AI agent (Agent Mode): a deliberately minimal agent architecture that calls LLM APIs directly (tool loops, multi-step execution, checkpointing), primarily in TypeScript. No prior TypeScript is required; strong Python fundamentals transfer quickly.
Read the source code of open-source agent harnesses and turn what you learn into design specs and prototypes.
Document experiments, design decisions, and runbooks so your work is legible to the next person; flag safety, fairness, or privacy concerns you observe in model or agent behavior.
About You
Currently pursuing a BS, MS, or PhD in Computer Science, Data Science, or a related quantitative field.
Hands-on experience training or evaluating ML models: course projects, research, hackathons, or a prior internship all count.
Solid Python fundamentals: data structures, functions, basic testing; comfortable writing and reviewing code outside of notebooks.
Working knowledge of at least one deep-learning framework (PyTorch preferred).
Clear written and verbal communication, and a habit of documenting what you build.
Preferred Qualifications (none required; the more of these you have, the better)
Experience fine-tuning open-weight LLMs (SFT, LoRA, RL, or distillation), with the improvement measured on a benchmark.
Experience building LLM agents (tool calling, multi-step loops) or LLM evaluation harnesses/benchmarks, and reporting results with statistical rigor.
A track record of shipping real software end-to-end: APIs and services, CLIs, Docker, CI/CD, cloud; public code on GitHub is a big plus.
Interest or experience in AI safety and robustness: red-teaming, prompt injection, agent security, fairness, or interpretability.
Exposure to model-efficiency work: quantization, low-bit inference, or serving optimization.
Evidence of rigor and initiative: publications, technical blog posts, ablation studies, or self-driven side projects with quantified results.
Fluency with AI coding tools (Claude Code, Cursor, Codex) to ship fast while still deeply understanding the systems you build.
What Else?
In addition to Postman's pay-on-performance philosophy, and a flexible schedule working with a fun, collaborative team, Postman offers a comprehensive set of benefits, including full medical coverage, flexible PTO, wellness reimbursement, and a monthly lunch stipend. Along with that, our wellness programs will help you stay in the best of your physical and mental health. Our frequent and fascinating team-building events will keep you connected, while our donation-matching program can support the causes you care about. We’re building a long-term company with an inclusive culture where everyone can be the best version of themselves.
At Postman we value in person collaboration. We are in office 5 days a week for all roles based out of our hubs in San Francisco Bay Area, Boston, Austin, New York City, Tokyo and London. For roles based in Bangalore, employees currently work in the office three days a week and will transition to five days per week by the end of the year. We were thoughtful in our approach which is based on collaboration and grounded in feedback from our workforce, leadership team, and peers. The benefits of our in office model will be shared knowledge, brainstorming sessions, communication, and building trust in-person that cannot be replicated via zoom.
Our Values
At Postman, we create with the same curiosity that we see in our users. We value transparency and honest communication about not only successes, but also failures. In our work, we focus on specific goals that add up to a larger vision. Our inclusive work culture ensures that everyone is valued equally as important pieces of our final product. We are dedicated to delivering the best products we can.
Equal Opportunity
Postman is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Headhunters and recruitment agencies may not submit resumes/CVs through this website or directly to managers. Postman does not accept unsolicited headhunter and agency resumes. Postman will not pay fees to any third-party agency or company that does not have a signed agreement with Postman.
