Model Deployment and Serving

Process of making trained deployment model available to endusers

Model Development :

Problem definition → Data collection → Data cleaning → Feature engineering → model selection → model training → model evaluation → Hyper-P Tuning → package→deploy model

(All these steps are performed on local environment)

Model Deployment:

• Packaging the model

• Store in registry

• Deploying the model

Model Serving:

MLOps engineers should implement model serving to make the model available to the endusers

• Setting up runtime environment for the model

• Resources (CPU, memory, etc.)

• Expose API

• Setting up Ingress, Load Balancer, autoscaling, CDN

Ways to Model deployment and Serving:

• VMs

  • Develop API using Flask/FastAPI

  • Artifacts stored on VM

  • Configure WSGI, LB, Autoscaling, VPC

  • (Cloud knowledge is required)

• Kubernetes

  • Create container

  • Pod creation

  • Service (SVC) creation

  • Ingress and API gateway configuration

  • (Kubernetes knowledge is required)

• Amazon SageMaker

  • Drag and drop configuration in AWS

  • AWS-based

• Kserve

  • YAML → Kubernetes-based Custom Resource Definitions (CRDs)

  • Deployment + serving model

  • Kubernetes based with less manual configuration steps