Large Language Model

Oracle Cloud Infrastructure (OCI) provides fully managed infrastructure to work with Large Language Model (LLM).

Train and Deploy LLM

You can train LLM at scale with multi-node and multi-GPU using Data Science Jobs (Jobs), and deploy it with Data Science Model Deployment (Model Deployments). The following blog posts show examples training and deploying Llama2 models:

• Multi-GPU multinode fine-tuning Llama2 on OCI Data Science

• Deploy Llama 2 in OCI Data Science

• Quantize and deploy Llama 2 70B on cost-effective NVIDIA A10 Tensor Core GPUs in OCI Data Science

Integration with LangChain

ADS is designed to work with LangChain, enabling developers to incorporate various LangChain components and models deployed on OCI seamlessly into their applications. Additionally, ADS can package LangChain applications and deploy it as a REST API endpoint using OCI Data Science Model Deployment.

• Bridging cloud and conversational AI: LangChain and OCI Data Science platform

• Deploy LangChain applications as OCI model deployments

Installation

Install ADS and other dependencies for LLM integrations.

$ python3 -m pip install "oracle-ads[llm]"

• Training Large Language Model
  • Access the Pre-Trained Model
  • Fine-Tuning the Model
• LangChain Integration
  • Integration with Generative AI
  • Integration with Model Deployment
  • Authentication
• Deploy LangChain Application
  • Configuration
  • LangChain Application
  • Initialize the ChainDeployment
  • Prepare the Model Artifacts
  • Verify the Serialized Application
  • Save Artifacts to OCI Model Catalog
  • Deploy the Model
  • Invoke the Deployed Model
• Integration with OCI Generative AI and OpenSearch
  • OCI Generative Embedding
  • Retrieval QA with OpenSearch
  • Retrieval QA with FAISS
  • Deployment of Retrieval QA
• Guardrails
  • HuggingFace Measurement as Guardrail
  • Using Guardrail with LangChain
  • Guardrail Sequence