We are seeking a Prompt Engineer/Data Analyst to join our team. This hybrid role blends strong data analysis expertise with emerging AI capabilities to optimize Generative AI systems and ensure high data quality for AI-driven initiatives. The ideal candidate is analytical, detail-oriented, and passionate about improving LLM-based solutions to deliver measurable business value.
What You’ll Do
- Design, test, and optimize prompts for various LLM applications (GPT-4, Claude, Llama).
- Develop reusable prompt templates and implement advanced techniques (few-shot, chain-of-thought, structured prompting).
- Conduct A/B testing to evaluate and improve prompt performance across different models.
- Prepare, clean, and validate datasets for fine-tuning and RAG systems.
- Build and maintain data pipelines to feed structured context into AI systems.
- Define, track, and report KPIs such as accuracy, latency, and cost efficiency.
- Develop evaluation frameworks to assess LLM outputs (relevance, factuality, consistency).
- Create dashboards and reports to communicate AI performance and impact to stakeholders.
- Maintain prompt libraries, knowledge bases, and documentation for AI systems.
- Collaborate with cross-functional teams including Engineering, Product, and QA in Agile environments.
What You Know
- 4+ years of experience as a Data Analyst, Business Analyst, or similar role.
- Strong proficiency in SQL for data extraction and analysis.
- Hands-on experience with Python (pandas, numpy) for data manipulation.
- Experience working with LLM APIs (OpenAI, Anthropic, or open-source models).
- Solid understanding of prompt engineering techniques and LLM behavior.
- Experience with RAG systems and vector databases.
- Experience with data visualization tools (Tableau, Power BI, or similar).
- Basic knowledge of APIs, JSON/CSV formats, and Git version control.
- Comfortable working in cloud environments (AWS, Azure, or GCP).
Secondary Skills (Nice To Have)
- Knowledge of vector embeddings and semantic search.
- Familiarity with LangChain, LlamaIndex, or similar frameworks.
- Experience working in Agile environments using Jira/Confluence.
- Background in software testing, QA, or SDLC processes.
- Experience in enterprise domains (financial services, insurance, retail).
Education
- Bachelor’s degree in computer science, Data Science, Information Systems, or a related field.