Generative AI Engineer II - LLM Application Engineering

Job title: Generative AI Engineer II - LLM Application Engineering

About the Role The Platform Engineer II will design, build, and optimize scalable platforms that support Large Language Model (LLM) application development, deployment, and operations. This role enables rapid experimentation and reliable productionization of Generative AI solutions by creating robust infrastructure, automation pipelines, and integration frameworks. The engineer collaborates closely with AI/ML, application development, and DevOps teams to ensure high system reliability, security, and performance across the LLM lifecycle.

What You'll Do

• Architect, implement, and enhance platform components supporting LLM-based application engineering, including model serving, vector databases, feature stores, and data pipelines.
• Develop and maintain CI/CD automation for model deployment, evaluation, monitoring, and rollback.
• Optimize infrastructure for performance, scalability, cost efficiency, and enterprise-grade security.
• Integrate LLM services with internal tools, APIs, microservices, and cloud-native systems.
• Build observability frameworks including logging, tracing, model performance metrics, and guardrail monitoring.
• Support fine-tuning and RAG workflows by enabling scalable data processing and model hosting.
• Troubleshoot production issues, conduct root-cause analysis, and drive continuous platform improvements.
• Collaborate with cross-functional teams to establish best practices, governance, and reusable components for LLM application delivery.
• Contribute to documentation, knowledge sharing, and internal enablement around platform architecture and usage.
• Work with stakeholders to identify key business problems and determine how AI can address them.
• Stay up-to-date with the latest AI trends and technologies, Responsible AI to continually enhance your skills and contributions.

What We're Looking For

• Strong Python programming skills.
• Understanding of LLM frameworks, RAG systems, GenAI, Agentic concepts.
• Strong experience in cloud platforms (AWS/Azure/GCP) including compute, networking, storage, and IAM.
• Expertise in containerization and orchestration (Docker, Kubernetes, Helm).
• Proficiency in building CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins, ArgoCD).
• Solid understanding of LLMs, model serving frameworks (OpenAI, Hugging Face, TensorRT-LLM, LangChain), and RAG architectures.
• Experience with infrastructure-as-code (Terraform, CloudFormation, ARM/Bicep).
• Hands-on experience with monitoring/observability tools (Prometheus, Grafana, ELK, OpenTelemetry).
• Familiarity with distributed systems, microservice architecture, and API design.
• Experience with vector databases (FAISS, Pinecone, Milvus, Weaviate) and model hosting patterns.
• Behavioral: Excellent Communication skills; Critical and Analytical Thinking; Strong problem-solving mindset; Excellent collaboration and communication across cross-functional teams; Adaptability and continuous learning orientation.

Nice to Have

• Experience with MLOps platforms (SageMaker, Vertex AI, Azure ML).
• Model optimization techniques (quantization, distillation, GPU tuning).
• Understanding of data engineering tools (Spark, Databricks, Kafka).
• Security best practices for AI/ML systems (zero trust, data protection, compliance).
• Experience contributing to internal developer platforms or platform engineering frameworks.