Find your next role
Discover roles across ambitious teams
17,614 jobs
- More jobs
Ramp
Research Engineer, Benchmarks & Evaluation
New York, NY · Hybrid · Full time · Mid level
$204k–$352k / year
About the role
Own Ramp's benchmarking effort for AI models in finance workflows. Build benchmarks, graders, and RL environments to improve model decisions.
What you’ll do
• Build benchmarks from real finance workflows. • Own benchmarking effort and report results. • Design graders with rubrics and programmatic checks. • Turn evals into reward functions and RL environments. • Publish benchmarks and write-ups on model performance.
What you’ll bring
• Experience building LLM benchmarks or evals. • Strong Python and software engineering skills. • Statistical judgment and variance analysis. • Knowledge of post-training and RL with rewards. • Comfortable with SQL and large datasets. • Work authorization for the United States.
Life at Ramp
Flexible PTO, health and wellness stipend, 100% medical coverage, parental leave, pet insurance, relocation coverage, and more.
About Ramp
Ramp builds smart infrastructure for finance teams automating spend flows for over 85,000 companies managing $275B annually.