Outcomes & KPIs Key Outcome(s) Owned
Automated, continuous generation and curation of training, evaluation, and false-positive (FP) datasets across web, file, and macro threat surfaces.
Standardized, reproducible evaluation benchmarks that guide model improvements before deployment.
Automated data pipeline uptime and ingestion throughput from web/file sources.
Size, diversity, and labeling accuracy of threat dataset repositories (PI, Shadow-AI, PII, Macros).
Mean time to ingest live false-positive/false-negative feedback into training evaluation loops.
Accuracy and benchmark coverage across historical vs. newly trained SLMs.
What You'll Do
Design and operate automated data collection, mining, and web-scraping pipelines to gather threat samples from surrogate traffic and web sources.
Build data processing, annotation, and dataset curation workflows, training and re-training various models, e.g. Prompt Injection (PI) SLMs, Shadow-AI detection models, Malicious Macro classifiers, and PII detection.
Build mechanisms to capture false-positive (FP) samples from live user/DDR flows and represent / integrate them into model retraining pipelines.
Develop automated benchmarking and evaluation harnesses to continuously test model detection quality, runtime performance, and accuracy against baseline models (e.g., Vertex AI).
Build automated pipelines for model training for various models.
Work closely with research and software engineering teams to turn raw detection telemetry into ongoing actionable model improvements.
Required
2–4 years of experience in software engineering, data engineering, or automated QA/evaluation roles.
Strong programming proficiency in Python. Adv: C++, SQL.
Experience in model training and various ML, AI algorithms.
Experience building data pipelines, web scrapers, or automated data processing systems.
Familiarity with data structures, storage formats, and database configurations.
Methodical approach to dataset labeling, validation, bug logging, and metric tracking.
Preferred / Nice to Have
Hands-on experience with ML evaluation frameworks, dataset curation, or annotation tools.
Familiarity with text extraction, NLP preprocessing, or document parsing techniques.
Understanding of cybersecurity concepts (Prompt Injection, Malicious Macros, DLP/PII).
Basic experience with cloud storage, object stores, and basic containerization (Docker).
Experience with AWS, GCP, Terraform, Kubernetes, and Linux.
Follow us on LinkedIn https://www.linkedin.com/company/menlo-security/!
Why Menlo?
At Menlo, we don't settle for the status quo — in our technology or our culture. How we think and act is just as important as what we build. Our culture is defined by five core mindsets: Proactive Leadership, Straight Talk, United Impact, Elevated Talent, and Customer-Compelled. We take ownership and drive outcomes without waiting to be told. We communicate directly and seek hard truths. We break down silos and win together. We hold a high bar for ourselves and the people around us. And we treat every customer interaction as mission-critical. If you're someone who sees it, owns it, solves it, and does it — you'll thrive here.
All qualified applicants will receive consideration for employment without regard to race, sex, color, religion, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability.
TO ALL AGENCIES: Please, no phone calls or emails to any employee of Menlo Security outside of the Talent organization. Menlo Security’s policy is to only accept resumes from agencies via Ashby (ATS). Agencies must have a valid services agreement executed and must have been assigned by the Talent team to a specific requisition. Any resume submitted outside of this process will be deemed the sole property of Menlo Security. In the event a candidate submitted outside of this policy is hired, no fee or payment will be paid.
