Job Description:

• Design, develop, and optimize prompts for Large Language Models (LLMs) to support banking use cases such as customer service, compliance, risk analysis, and internal copilots.

• Build structured prompt templates to ensure consistent, accurate, and compliant AI-generated outputs across enterprise applications.

• Collaborate with Data Engineers and Data Scientists to integrate prompt logic into production AI systems and RAG-based architectures.

• Translate banking policies (KYC, AML, credit risk, product guidelines) into clear, executable prompt instructions.

• Monitor AI responses in production, troubleshoot inaccuracies or hallucinations, and perform root-cause analysis to improve reliability.

• Implement guardrails and output validation mechanisms to reduce regulatory and reputational risk.

• Conduct systematic prompt testing, including edge-case handling and scenario-based validation for high-risk financial queries.

• Version control and manage prompt libraries using GitHub or Azure DevOps to ensure traceability and safe deployment.

• Develop evaluation metrics (accuracy, compliance alignment, hallucination rate, user satisfaction) and continuously optimize performance.

• Document prompt logic, testing results, and governance controls to ensure auditability and regulatory compliance.

• Integrate AI systems with internal knowledge bases, policy documents, and structured data sources to enhance contextual accuracy.

• Stay up to date with advancements in LLMs, prompt engineering techniques, and responsible AI frameworks.

Preferred Skills:

• Proven experience working with Large Language Models (LLMs) and prompt engineering techniques (few-shot, chain-of-thought, role prompting, structured output control).

• Experience in banking or financial services with understanding of compliance, AML/KYC, credit risk, or regulatory frameworks.

• Familiarity with Microsoft Fabric, Azure OpenAI, or other enterprise AI platforms.

• Proficiency in Python for prompt testing, evaluation scripting, and API integration.

• Strong SQL skills (PostgreSQL, Oracle, MSSQL) for retrieving and validating contextual data.

• Experience implementing Retrieval-Augmented Generation (RAG) architectures.

• Understanding of AI governance, model risk management, and responsible AI principles.

• Strong analytical mindset with ability to identify weaknesses in AI logic and refine prompts systematically.

• Ability to break down complex business policies into structured logical instructions for AI systems.

• Strong collaboration skills to work effectively with Data Scientists, Engineers, Compliance, and Business Stakeholders.

• Excellent documentation and communication skills to support transparency and audit requirements.

• Growth mindset with ability to adapt quickly to evolving AI technologies and enterprise standards.

CB Bank PCL jobs

Prompt Engineer