Sr. Manager AI Innovation, Pakistan Remote

Salary: Rs 9 Lakh/Month

Company Description Pinnacle Systems Inc is a privately held company that helps enterprises identify and develop their “AI edge,” delivering AI-driven solutions and expertise to improve business performance and spark innovation. Operating from a single location, the company partners closely with organizations to design and implement practical artificial intelligence applications that create measurable competitive advantage. Pinnacle Systems Inc focuses on connecting strategic business goals with modern AI capabilities, ensuring solutions are both technically sound and commercially relevant. Team members collaborate with clients across industries, working on initiatives that modernize operations and unlock new growth opportunities.

Key Responsibilities

Hands-On Data Science & Model Development

• Build, test, validate, and improve machine learning models for scoring, prediction, prioritization, risk detection, engagement, intervention targeting, and decision support.
• Perform exploratory data analysis, statistical analysis, hypothesis testing, data quality assessment, feature engineering, model training, model selection, and performance evaluation.
• Develop practical models that balance predictive performance, explainability, stability, maintainability, and business usefulness.
• Work with structured, semi-structured, and unstructured data (including text, voice, and event streams) to create model-ready datasets and reusable features.
• Use tools such as Python, SQL, Spark, Databricks, MLflow, scikit-learn, XGBoost, PyTorch, or similar platforms and libraries.
• Move quickly from data exploration to prototype to validated model to production-ready capability.

Predictive Scoring, Decisioning & Transparent Models

• Design and implement predictive scores, risk tiers, score bands, thresholds, cut points, and intervention logic.
• Build transparent and interpretable models where explainability is important, including logistic regression, GLMs, GAMs, decision trees, monotonic models, calibrated models, scorecard-style models, and explainable boosting approaches.
• Evaluate models for accuracy, calibration, stability, drift, fairness, interpretability, and operational usefulness.
• Help stakeholders understand what a score represents, how it should be used, how it should not be used, and how changes in the score should be interpreted.
• Document model logic, features, assumptions, limitations, validation results, and recommended usage for both business and technical audiences.

Generative AI & LLM-Powered Solutions

• Design, build, and evaluate GenAI solutions including LLM-powered workflows, retrieval-augmented generation (RAG), summarization, conversational assistants, document intelligence, and decision-support tools.
• Develop and maintain prompt engineering standards, evaluation harnesses, golden datasets, and LLM-as-judge or human-review evaluation loops.
• Fine-tune, adapt, or orchestrate foundation models where appropriate; evaluate when to use hosted LLM APIs, open-weight models, traditional ML, rules, or transparent scoring instead.
• Combine predictive models with GenAI workflows — for example, using risk scores to trigger outreach, summarize member context, recommend next actions, or support human decision-making.
• Apply guardrails, grounding, privacy controls, hallucination mitigation, and human-in-the-loop patterns appropriate for a healthcare and safety environment.

Agentic AI & AI Automation

• Design, prototype, and productionize AI agents and multi-agent systems that plan, use tools, retrieve context, and take actions within business workflows (e.g., member outreach, triage, care coordination support, operations automation).
• Work with modern agent frameworks and standards such as LangGraph, Claude Agent SDK, OpenAI Agents SDK, CrewAI, and Model Context Protocol (MCP) for tool and data integration.
• Define agent boundaries, permissions, escalation paths, and human oversight models so agents act safely, predictably, and within policy.
• Build evaluation and observability for agentic systems: task success rates, trajectory analysis, cost and latency tracking, failure-mode detection, and regression testing.
• Determine where agentic automation creates real value versus where deterministic workflows, traditional ML, or simple LLM calls are the better fit.

Production AI & MLOps / LLMOps

• Partner with data engineering, platform engineering, and application engineering teams to move models, GenAI solutions, and agents from experimentation into reliable production workflows.
• Support deployment, batch and real-time inference, model and prompt versioning, monitoring, retraining, and performance tracking.
• Help define data pipelines, feature pipelines, retrieval pipelines, inference flows, feedback loops, and monitoring requirements.
• Ensure AI systems are observable, supportable, secure, scalable, cost-aware, and aligned with enterprise architecture and governance expectations.
• Establish practical monitoring and feedback loops to determine whether models and AI systems continue to perform and create value over time.

Required Qualifications

• 8+ years of professional experience in data science, machine learning, applied AI, or related technical fields.
• 5+ years of hands-on model development experience, including feature engineering, model training, validation, evaluation, and iteration.
• 3+ years of experience deploying, operationalizing, or supporting models or AI systems in production or business-critical environments.
• 1+ years of hands-on experience building GenAI/LLM solutions (RAG, summarization, conversational AI, document intelligence, or AI-enabled workflow automation), including evaluation and guardrails.
• Strong hands-on experience with Python and SQL.
• Experience with modern ML and data platforms such as Databricks, Spark, MLflow, Snowflake, Azure, AWS, or similar technologies.
• Experience with LLM ecosystems and frameworks (e.g., Anthropic/OpenAI APIs, LangChain/LangGraph, vector databases, embedding models) and familiarity with agentic patterns and tool use.
• Strong understanding of model evaluation, calibration, thresholding, score interpretation, monitoring, drift, retraining, and production AI lifecycle management.
• Experience translating ambiguous business problems into concrete DS/ML/AI designs, requirements, validation plans, and measurable outcomes.
• Ability to explain model and AI system behavior, performance, assumptions, limitations, and tradeoffs to both technical and non-technical stakeholders.
• Strong engineering discipline, including clean code, reproducibility, versioning, testing, documentation, and maintainability.
• Ability to work independently as a senior hands-on contributor while providing technical leadership and modeling judgment.

Interview Guidelines:

• Your interview link: https://cogniter.ai/apply/71fe92d1-bd91-4958-84b7-53bb9d07698e

What to know:

• • Length: about 60 minutes
• • Format: A voice conversation with our AI interviewer. You'll speak your answers out loud — no coding editor or written test.
• • You pick your skills: At the start, you'll choose the skills you're most confident in and give yourself a quick confidence rating for each.
• • Adaptive questions: The interview adjusts to you. It starts at a fair baseline and goes deeper as you answer well, so expect it to get more challenging if you're doing great. That's by design.
• • Recording: Your camera and microphone are recorded so our team can review your interview. Please make sure you're comfortable being on camera.