Tools to Deploy ML Apps logopeterindia.net
Knowledge Hub

Kubernetes Tools for Machine and Deep Learning Models

Tools that train, schedule and serve machine learning models on Kubernetes, from ML toolkits and model servers to batch schedulers and GPU enablement.

16 topics•4 categories•Updated October 2026

ML Platforms & Model Serving

5 entries

Toolkits that run the ML life cycle on Kubernetes.

01KubeflowThe Machine Learning Toolkit for Kubernetes. 02Seldon CoreConverts ML models (TensorFlow, PyTorch, H2O) or language wrappers (Python, Java) into production REST/gRPC microservices. 03KServeStandard model inference platform on Kubernetes. 04MLflowOpen-source platform for the machine learning life cycle. 05Ray on KubernetesRun Ray clusters on Kubernetes to scale Python and AI workloads.

Scheduling & GPUs

4 entries

Batch scheduling and accelerator support.

06VolcanoProvides a suite of mechanisms required by many classes of batch & elastic workloads. 07KueueKubernetes-native job queueing for batch and ML workloads. 08NVIDIA Container ToolkitAllows users to build and run GPU-accelerated Docker containers (successor to nvidia-docker). 09NVIDIA GPU OperatorAutomates management of NVIDIA GPU software on Kubernetes.

Build & Cluster Management

2 entries

10repo2dockerFetches a git repository and builds a container image based on the configuration files found in it. 11Kubermatic Kubernetes PlatformA cluster-management tool offering automation to integrate with ML/DL workflows.

Related Pages

5 entries

Related directories on this site.

12MLOps PlatformsMLOps platforms. 13Cluster Management ToolsManaging clusters. 14Tools for Automated K8s ManagementAutomating cluster management. 15Lightweight Kubernetes DistributionsSmall-footprint distributions. 16ContainerizationContainerization hub.

Frequently Asked Questions

Why run machine learning on Kubernetes?

Kubernetes gives ML teams portable infrastructure, autoscaling, GPU scheduling and the same deployment tooling as the rest of the organisation.

Kubeflow or KServe?

Kubeflow is a broader ML toolkit with pipelines and notebooks. KServe focuses on serving trained models at scale and can be used on its own.

How are GPUs handled?

The NVIDIA Container Toolkit and GPU Operator expose GPUs to containers, and schedulers such as Volcano and Kueue manage queues of GPU jobs.

What changed on this page?

All 6 original tools are kept. KServe, MLflow, Ray, Kueue and the NVIDIA GPU Operator were added.

Are the links checked?

Links were checked when this page was updated and all of them loaded.

Has Kubeflow graduated?

Yes. CNCF announced Kubeflow's graduation on 17 August 2026, describing it as a mature platform for AI and machine learning operations on Kubernetes. KServe was accepted into the CNCF on 29 September 2025. The Certified Kubernetes AI Conformance program certified 31 platforms by March 2026.