Any Framework

You could also use your own frameworks to deploy your models. You could follow the instructions:

• Dockerize your model server.
• Upload your Docker image to a public Docker registry (e.g. Docker Hub).
• Create the deployment with the custom image, choose Other framework, and set the command and port in the advanced features.

Autoscaling

You could enable autoscaling for your deployment. The autoscaling feature will automatically scale up and down the number of replicas based on the request load. You could set the minimum and maximum number of replicas.