Prompt Engineer, Handshake AI

Handshake • Full-time • Remote (New York, New York State, United States) • $120k - $165k / year • 3w ago

Your impact

Handshake is building the future of human data for AI.

We partner directly with top AI labs to power large language model (LLM) training and evaluation with high-quality, expert-generated data. As AI models become more sophisticated, the demand for specialized human input continues to grow—and Handshake is uniquely positioned to meet it. We power career platforms at 92% of the top U.S. universities, giving us direct access to verified expert talent across a wide range of domains.

Our AI team is rapidly building a new generation of human data products—from expert annotation platforms to AI interviewers and seamless payout infrastructure—all designed to accelerate research and improve model performance.

We’ve assembled a world-class team from YC, Notion, Scale, Coinbase, Palantir, and more, and we’re working directly with many of the world’s leading AI research labs. This is a unique opportunity to join a fast-growing team shaping the future of AI through better data, better tools, and better systems—for experts, by experts.

We’re looking for a creative and technically strong Prompt Engineer to design, experiment with, and optimize high-performing prompts that maximize the effectiveness of LLMs. You’ll collaborate closely with cross-functional teams, including product managers, ML researchers, and software engineers, to build prompt-based systems and integrate them into our core products.

Location: San Francisco or New York City

Your role

Design, test, and iterate on prompts for a range of LLMs to deliver robust and high-quality outputs across different domains and tasks.
Develop automated evaluation pipelines and metrics to quantify prompt performance and model outputs.
Collaborate with engineers and researchers to fine-tune models or train domain-specific systems when necessary.
Maintain deep knowledge of LLM behavior, updates, and capabilities, identifying opportunities for prompt optimization.
Build internal tooling and libraries to support scalable prompt experimentation and deployment.
Write detailed documentation and conduct prompt evaluations across varying use cases (e.g., summarization, Q&A, extraction).
Stay on the cutting edge of LLM advancements, contributing insights to internal knowledge bases and model selection strategies.

Your experience

Bachelor’s or Master’s degree in Computer Science, Linguistics, AI, Applied Math, or a related field.
Proficiency with Python or Typescript
Demonstrated ability to prompt and evaluate LLMs across various NLP tasks.
Analytical mindset with a strong understanding of language structures, semantics, and grammar.
Familiarity with LLM evaluation metrics (BLUE, ROUGE, MAUVE, etc.) and statistics
Experience designing experiments and interpreting evaluation metrics.
Clear written and verbal communication skills.

Bonus areas of expertise

Previous experience working directly with foundation models like GPT-4, Claude, or PaLM.
Background in linguistics, creative writing, or logic-driven fields.
Familiarity with vector databases, retrieval-augmented generation (RAG), or custom embedding models.
Experience with prompt chaining, agent orchestration, or LLMOps tooling.
Contributions to prompt engineering toolkits or open-source libraries.

What we offer

At Handshake, we'll give you the tools to feel healthy, happy and secure.

Benefits below apply to US employees in full-time positions.

Equity and ownership in a fast-growing company.
16 Weeks of paid parental leave for birth giving parents & 10 weeks of paid parental leave for non-birth giving parents.
Comprehensive medical, dental, and vision policies including LGTBQ+ Coverage. We also provide resources for Mental Health Assistance, Employee Assistance Programs and counseling support.
Generous learning & development opportunities and an annual $2,000 stipend for you to grow your skills and career.
Financial coaching through Origin to help you through your financial journey.
Monthly internet stipend and a brand new MacBook to allow you to do your best work.
Monthly commuter stipend for you to expense your travel to the office (for office-based employees).
Free lunch provided 5x a week across all offices.
Free gym access in San Francisco office building.
Referral bonus to reward you when you bring great talent to Handshake.
Team outings throughout the year to stay connected to each other.

401k Match: Handshake offers a dollar-for-dollar match on 1% of deferred salary, up to a maximum of $1,200 per year.
All full-time US-based Handshakers are eligible for our flexible time off policy to get out and see the world. In addition, we offer 13 standardized holidays, and 2 additional days of flexible holiday time off. Lastly, we have a Winter #ShakeBreak, a one-week period of Collective Time Off.
Handshake offers $500 home office stipend for you to spend during your first 3 months to create a productive and comfortable workspace at home.
Family support: Parental leave coaching and support provided by Parentaly. We partner with Maven Clinic to provide a lifetime coverage up to $15K for expenses related to fertility and family forming!
Lifestyle Savings Account: We offer you an annual stipend of $500 to use for purchases such as fitness classes, gym memberships, work-from-home setup, and more.

Looking for more? Explore our mission, values and comprehensive US benefits at joinhandshake.com/careers.

Handshake is committed to providing reasonable accommodations in our recruitment processes for candidates with disabilities, sincerely held religious beliefs or other reasons protected by applicable laws. If you need assistance or reasonable accommodation, please let your recruiter know during initial communications.

Compensation Range: $120K - $165K