Prompt Engineer

Checkmate • Full-time • New York, NY, US • $125k - $160k / year • 3w ago

In this hybrid role, you will blend traditional software engineering with natural language processing expertise to develop and maintain systems that leverage LLMs to solve real-world challenges in the restaurant tech domain. You'll collaborate with cross-functional teams to implement AI-driven features that support everything from customer service automation to order management, reporting, and staff training.

Essential Job Functions:

Prompt Development & Optimization: Design, build, test, and refine prompts for AI tasks like Q&A, classification, extraction, and synthetic data creation.
Evaluation & Testing Build robust test suites using tools like CoVal, LangFuse, and Galileo to measure prompt accuracy, compliance, and performance at scale.
Performance Analysis & Improvement: Analyze prompt outputs using metrics, product feedback, and business goals. Iterate to maximize accuracy and reliability.
Collaboration: Work closely with operations, product, data science, and engineering teams to ensure prompts meet quality standards
Model Adaptation: By understanding their unique behaviors, create effective prompts across different models (GPT, LLaMA, Gemini, and internally fine-tuned LLMs).
End-to-End Ownership Drive prompts projects from planning to deployment, including monitoring and continuous improvement.
Team Leadership & Mentorship Lead a team of analysts, guiding their evaluations and data work to support prompt development. Mentor and collaborate with ops teams to ensure seamless integration of prompts into production.

100% Remote

Minimum of three years of experience in the areas of AI, ML, NLP, and prompt engineering.

$125,000 to $160,000

Requirements

Requirements:

Strong Python programming skills
Solid data and computer science fundamentals
Experience with relational databases (e.g., MySQL, PostgreSQL, Oracle, MS SQL)
Hands-on development experience, preferably full-stack
Good data processing and testing skills
Ability to handle ambiguous problems and drive projects from start to finish
Strong problem-solving mindset with a drive to improve and optimize systems
Ability to use scientific and analytical approaches to measure and evaluate designs
Excellent communication skills
Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field
Flexible to work US hours until at least 6 pm ET with a strong remote setup

Preferred Qualifications

Experience with multiple LLMs and tools like LangFuse and Galileo
Knowledge of cloud platforms (AWS, GCP, Azure, OpenStack) and tools like Docker and Kubernetes
Focus on scalability, performance, and cost optimization in cloud environments
Broad technical expertise across front-end and back-end systems
Familiarity with machine learning concepts and workflows
Master's or PhD in Computer Science, Engineering, Math, Physics, or related field

Benefits