Senior AI Engineer with Spark, AWS Services

We are seeking a Senior AI Engineer with Spark and AWS Services expertise to join the RBQM Production Pod within the program. You will build and maintain data pipelines that power AI/GenAI applications for Risk-Based Quality Management in clinical trials. This role focuses on RAG document ingestion, vector indexing, and building data APIs for AI applications.

Responsibilities

• Design and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality data
• Build and manage vector databases (AWS OpenSearch) for RAG-powered AI workflows
• Develop batch and streaming ETL/ELT pipelines from scratch for unstructured clinical data (PDF, DOCX, clinical reports)
• Build and expose data APIs for AI application consumption
• Optimize chunking strategies, embedding generation, and retrieval performance for RAG architectures
• Manage data quality, lineage, and governance for AI/ML data pipelines
• Deploy and maintain AWS data infrastructure (S3, Lambda, Glue, Athena, Step Functions, DynamoDB)
• Collaborate with Data Scientists and Backend Developers in an integrated pod team

Requirements

• 5+ years of hands-on data engineering experience at scale
• Expertise in RAG document ingestion pipelines (chunking, embedding, vector indexing)
• Proficiency in AWS OpenSearch as a vector database for RAG workflows
• Advanced proficiency in Python, including SQL and Spark SQL
• Skills in unstructured data transformation (PDF, DOCX) for RAG/LLM applications
• Familiarity with AWS Services: S3, Lambda, Glue, Athena, Bedrock, Step Functions, API Gateway, CloudWatch, DynamoDB
• Knowledge of containerization with Docker
• Capability to build custom pipelines from scratch, beyond configuring out-of-the-box services
• Proficiency in English at a B2+ level

Nice to have

• Background in pharmaceutical or life sciences domain
• Familiarity with Snowflake, Pinecone (vector DB alternative)
• Knowledge of SageMaker processing jobs
• Skills in CI/CD tools (Jenkins, Git/Bitbucket) and infrastructure tools (CDK or Terraform)
• Understanding of clinical data standards (CDISC, ADaM, SDTM)

We offer

CONTINUOUS UPSKILLING, LEARNING & DEVELOPMENT

• Diversity of tasks and projects
• Assessment center for objective review of competency level
• Personal development plan
• Mentoring programs and leadership development
• Certification and professional development support 
• Access to learning platforms including more than 2,500 internal courses
• English courses taught by certified teachers

CORPORATE BENEFITS

• Extra leave days
• Referral bonuses

COMPENSATION PACKAGE

• Competitive compensation paid in USD
• Regular salary and performance reviews

MEDICAL & HEALTHCARE

• Private health insurance
• Well-being events

WORKING ENVIRONMENT

• Recreation areas and kitchens
• Tea, coffee and snacks
• Sports equipment and game consoles
• IT Equipment
• Microsoft’s Software Assurance Home Use Program (HUP)

Please note that our Talent Attraction Team reviews applications and CVs submitted in English.

EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.

Remote in Türkiye