Job Description:
• Design, develop, and optimize prompts for Large Language Models (LLMs) to support banking use cases such as customer service, compliance, risk analysis, and internal copilots.
• Build structured prompt templates to ensure consistent, accurate, and compliant AI-generated outputs across enterprise applications.
• Collaborate with Data Engineers and Data Scientists to integrate prompt logic into production AI systems and RAG-based architectures.
• Translate banking policies (KYC, AML, credit risk, product guidelines) into clear, executable prompt instructions.
• Monitor AI responses in production, troubleshoot inaccuracies or hallucinations, and perform root-cause analysis to improve reliability.
• Implement guardrails and output validation mechanisms to reduce regulatory and reputational risk.
• Conduct systematic prompt testing, including edge-case handling and scenario-based validation for high-risk financial queries.
• Version control and manage prompt libraries using GitHub or Azure DevOps to ensure traceability and safe deployment.
• Develop evaluation metrics (accuracy, compliance alignment, hallucination rate, user satisfaction) and continuously optimize performance.
• Document prompt logic, testing results, and governance controls to ensure auditability and regulatory compliance.
• Integrate AI systems with internal knowledge bases, policy documents, and structured data sources to enhance contextual accuracy.
• Stay up to date with advancements in LLMs, prompt engineering techniques, and responsible AI frameworks.
Preferred Skills:
• Proven experience working with Large Language Models (LLMs) and prompt engineering techniques (few-shot, chain-of-thought, role prompting, structured output control).
• Experience in banking or financial services with understanding of compliance, AML/KYC, credit risk, or regulatory frameworks.
• Familiarity with Microsoft Fabric, Azure OpenAI, or other enterprise AI platforms.
• Proficiency in Python for prompt testing, evaluation scripting, and API integration.
• Strong SQL skills (PostgreSQL, Oracle, MSSQL) for retrieving and validating contextual data.
• Experience implementing Retrieval-Augmented Generation (RAG) architectures.
• Understanding of AI governance, model risk management, and responsible AI principles.
• Strong analytical mindset with ability to identify weaknesses in AI logic and refine prompts systematically.
• Ability to break down complex business policies into structured logical instructions for AI systems.
• Strong collaboration skills to work effectively with Data Scientists, Engineers, Compliance, and Business Stakeholders.
• Excellent documentation and communication skills to support transparency and audit requirements.
• Growth mindset with ability to adapt quickly to evolving AI technologies and enterprise standards.