KubeRay CRD API reference

This page is the generated field reference for the ray.io/v1 custom resource definitions. It covers RayCluster, RayCronJob, RayJob, and RayService, along with the supporting types their fields refer to.

The fields are those of KubeRay v1.6.2. Fields added to KubeRay after that release don’t appear here.

Resource Types

• RayCluster

• RayCronJob

• RayJob

• RayService

AuthMode

Underlying type: string

AuthMode describes the authentication mode for the Ray cluster.

Appears in:

• AuthOptions

Field	Description
disabled	AuthModeDisabled disables authentication.
token	AuthModeToken enables token-based authentication.

AuthOptions

AuthOptions defines the authentication options for a RayCluster.

Appears in:

• RayClusterSpec

Field	Description	Default	Validation
enableK8sTokenAuth boolean	EnableK8sTokenAuth enables Kubernetes-delegated token authentication. When true, the RAY_ENABLE_K8S_TOKEN_AUTH environment variable is set to “true” across all Ray Pods, and Ray will delegate authentication to the K8s API server. NOTE: The Kubernetes ServiceAccount token mounted to Raylets must be granted the ray:write custom verb via RBAC for this to function correctly. WARNING: This feature is intended for standalone RayCluster objects and is currently unsupported for RayJob or RayService resources.
secretName string	SecretName is the name of the Secret that contains the authentication token. If set, KubeRay will skip generating a Secret object per RayCluster containing a token. The Secret must have a data key auth_token that contains the value of the token.
mode AuthMode	Mode specifies the authentication mode. Supported values are “disabled” and “token”. Defaults to “token”.		Enum: [disabled token]

AutoscalerOptions

AutoscalerOptions specifies optional configuration for the Ray autoscaler.

Appears in:

• RayClusterSpec

Field	Description	Default	Validation
resources ResourceRequirements	Resources specifies optional resource request and limit overrides for the autoscaler container. Default values: 500m CPU request and limit. 512Mi memory request and limit.
image string	Image optionally overrides the autoscaler’s container image. This override is provided for autoscaler testing and development.
imagePullPolicy PullPolicy	ImagePullPolicy optionally overrides the autoscaler container’s image pull policy. This override is provided for autoscaler testing and development.
securityContext SecurityContext	SecurityContext defines the security options the container should be run with. If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext. More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
idleTimeoutSeconds integer	IdleTimeoutSeconds is the number of seconds to wait before scaling down a worker pod which is not using Ray resources. Defaults to 60 (one minute). It is not read by the KubeRay operator but by the Ray autoscaler.
upscalingMode UpscalingMode	UpscalingMode is “Conservative”, “Default”, or “Aggressive.” Conservative: Upscaling is rate-limited; the number of pending worker pods is at most the size of the Ray cluster. Default: Upscaling is not rate-limited. Aggressive: An alias for Default; upscaling is not rate-limited. It is not read by the KubeRay operator but by the Ray autoscaler.		Enum: [Default Aggressive Conservative]
version AutoscalerVersion	Version is the version of the Ray autoscaler. Setting this to v1 will explicitly use autoscaler v1. Setting this to v2 will explicitly use autoscaler v2. If this isn’t set, the Ray version determines the autoscaler version. In Ray 2.47.0 and later, the default autoscaler version is v2. It’s v1 before that.		Enum: [v1 v2]
env EnvVar array	Optional list of environment variables to set in the autoscaler container.
envFrom EnvFromSource array	Optional list of sources to populate environment variables in the autoscaler container.
volumeMounts VolumeMount array	Optional list of volumeMounts. This is needed for enabling TLS for the autoscaler container.

AutoscalerVersion

Underlying type: string

Validation:

• Enum: [v1 v2]

Appears in:

• AutoscalerOptions

Field	Description
v1
v2

ClusterUpgradeOptions

These options are currently only supported for the IncrementalUpgrade type.

Appears in:

• RayServiceUpgradeStrategy

Field	Description	Default	Validation
maxSurgePercent integer	The capacity of serve requests the upgraded cluster should scale to handle each interval. Defaults to 100%.	100
stepSizePercent integer	The percentage of traffic to switch to the upgraded RayCluster at a set interval after scaling by MaxSurgePercent.
intervalSeconds integer	The interval in seconds between transferring StepSize traffic from the old to new RayCluster.
gatewayClassName string	The name of the Gateway Class installed by the Kubernetes Cluster admin.

DeletionCondition

DeletionCondition specifies the trigger conditions for a deletion action. Exactly one of JobStatus or JobDeploymentStatus must be specified:

• JobStatus (application-level): Match the Ray job execution status.

• JobDeploymentStatus (infrastructure-level): Match the RayJob deployment lifecycle status. This is particularly useful for cleaning up resources when Ray jobs fail to be submitted.

Appears in:

• DeletionRule

Field	Description	Default	Validation
ttlSeconds integer	TTLSeconds is the time in seconds from when the JobStatus or JobDeploymentStatus reaches the specified terminal state to when this deletion action should be triggered. The value must be a non-negative integer.	0	Minimum: 0

DeletionPolicy

DeletionPolicy is the legacy single-stage deletion policy. Deprecated: This struct is part of the legacy API. Use DeletionRule for new configurations.

Appears in:

• DeletionStrategy

Field	Description	Default	Validation
policy DeletionPolicyType	Policy is the action to take when the condition is met. This field is logically required when using the legacy OnSuccess/OnFailure policies. It is marked as ‘+optional’ at the API level to allow the ‘deletionRules’ field to be used instead.		Enum: [DeleteCluster DeleteWorkers DeleteSelf DeleteNone]

DeletionPolicyType

Underlying type: string

Appears in:

• DeletionPolicy

• DeletionRule

Field	Description
DeleteCluster
DeleteWorkers
DeleteSelf
DeleteNone

DeletionRule

DeletionRule defines a single deletion action and its trigger condition. This is the new, recommended way to define deletion behavior.

Appears in:

• DeletionStrategy

Field	Description	Default	Validation
policy DeletionPolicyType	Policy is the action to take when the condition is met. This field is required.		Enum: [DeleteCluster DeleteWorkers DeleteSelf DeleteNone]
condition DeletionCondition	The condition under which this deletion rule is triggered. This field is required.

DeletionStrategy

DeletionStrategy configures automated cleanup after the RayJob reaches a terminal state. Two mutually exclusive styles are supported:

• Legacy: provide both onSuccess and onFailure (deprecated; removal planned for 1.6.0). May be combined with shutdownAfterJobFinishes and (optionally) global TTLSecondsAfterFinished.

• Rules: provide deletionRules (non-empty list). Rules mode is incompatible with shutdownAfterJobFinishes, legacy fields, and the global TTLSecondsAfterFinished (use per-rule condition.ttlSeconds instead).

Semantics:

• A non-empty deletionRules selects rules mode; empty lists are treated as unset.

• Legacy requires both onSuccess and onFailure; specifying only one is invalid.

• Global TTLSecondsAfterFinished > 0 requires shutdownAfterJobFinishes=true; therefore it cannot be used with rules mode or with legacy alone (no shutdown).

• Feature gate RayJobDeletionPolicy must be enabled when this block is present.

Validation:

• CRD XValidations prevent mixing legacy fields with deletionRules and enforce legacy completeness.

• Controller logic enforces rules vs shutdown exclusivity and TTL constraints.

• onSuccess/onFailure are deprecated; migration to deletionRules is encouraged.

Appears in:

• RayJobSpec

Field	Description	Default	Validation
onSuccess DeletionPolicy	OnSuccess is the deletion policy for a successful RayJob. Deprecated: Use deletionRules instead for more flexible, multi-stage deletion strategies. This field will be removed in release 1.6.0.
onFailure DeletionPolicy	OnFailure is the deletion policy for a failed RayJob. Deprecated: Use deletionRules instead for more flexible, multi-stage deletion strategies. This field will be removed in release 1.6.0.
deletionRules DeletionRule array	DeletionRules is a list of deletion rules, processed based on their trigger conditions. While the rules can be used to define a sequence, if multiple rules are overdue (e.g., due to controller downtime), the most impactful rule (e.g., DeleteSelf) will be executed first to prioritize resource cleanup.		MinItems: 1

GcsFaultToleranceOptions

GcsFaultToleranceOptions contains configs for GCS FT

Appears in:

• RayClusterSpec

Field	Description	Default	Validation
redisUsername RedisCredential
redisPassword RedisCredential
externalStorageNamespace string
redisAddress string

HeadGroupSpec

HeadGroupSpec are the spec for the head pod

Appears in:

• RayClusterSpec

Field	Description	Default	Validation
template PodTemplateSpec	Template is the exact pod template used in K8s deployments, statefulsets, etc.
headService Service	HeadService is the Kubernetes service of the head pod.
enableIngress boolean	EnableIngress indicates whether operator should create ingress object for head service or not.
resources object (keys:string, values:string)	Resources specifies the resource quantities for the head group. These values override the resources passed to rayStartParams for the group, but have no effect on the resources set at the K8s Pod container level.
labels object (keys:string, values:string)	Labels specifies the Ray node labels for the head group. These labels will also be added to the Pods of this head group and override the --labels argument passed to rayStartParams.
rayStartParams object (keys:string, values:string)	RayStartParams are the params of the start command: node-manager-port, object-store-memory, …
serviceType ServiceType	ServiceType is Kubernetes service type of the head service. it will be used by the workers to connect to the head pod

JobSubmissionMode

Underlying type: string

Appears in:

• RayJobSpec

Field	Description
K8sJobMode
HTTPMode
InteractiveMode
SidecarMode

RayCluster

RayCluster is the Schema for the RayClusters API

Field	Description	Default	Validation
apiVersion string	ray.io/v1
kind string	RayCluster
metadata ObjectMeta	Refer to Kubernetes API documentation for fields of metadata.
spec RayClusterSpec	Specification of the desired behavior of the RayCluster.

RayClusterSpec

RayClusterSpec defines the desired state of RayCluster

Appears in:

• RayCluster

• RayJobSpec

• RayServiceSpec

Field	Description	Default	Validation
upgradeStrategy RayClusterUpgradeStrategy	UpgradeStrategy defines the scaling policy used when upgrading the RayCluster
authOptions AuthOptions	AuthOptions specifies the authentication options for the RayCluster.
suspend boolean	Suspend indicates whether a RayCluster should be suspended. A suspended RayCluster will have head pods and worker pods deleted.
managedBy string	ManagedBy is an optional configuration for the controller or entity that manages a RayCluster. The value must be either ‘ray.io/kuberay-operator’ or ‘kueue.x-k8s.io/multikueue’. The kuberay-operator reconciles a RayCluster which doesn’t have this field at all or the field value is the reserved string ‘ray.io/kuberay-operator’, but delegates reconciling the RayCluster with ‘kueue.x-k8s.io/multikueue’ to the Kueue. The field is immutable.
autoscalerOptions AutoscalerOptions	AutoscalerOptions specifies optional configuration for the Ray autoscaler.
headServiceAnnotations object (keys:string, values:string)
enableInTreeAutoscaling boolean	EnableInTreeAutoscaling indicates whether operator should create in tree autoscaling configs
gcsFaultToleranceOptions GcsFaultToleranceOptions	GcsFaultToleranceOptions for enabling GCS FT
headGroupSpec HeadGroupSpec	HeadGroupSpec is the spec for the head pod
rayVersion string	RayVersion is used to determine the command for the Kubernetes Job managed by RayJob
workerGroupSpecs WorkerGroupSpec array	WorkerGroupSpecs are the specs for the worker pods

RayClusterUpgradeStrategy

Appears in:

• RayClusterSpec

Field	Description	Default	Validation
type RayClusterUpgradeType	Type represents the strategy used when upgrading the RayCluster Pods. Currently supports Recreate and None.		Enum: [Recreate None]

RayClusterUpgradeType

Underlying type: string

Validation:

• Enum: [Recreate None]

Appears in:

• RayClusterUpgradeStrategy

Field	Description
Recreate	During upgrade, Recreate strategy will delete all existing pods before creating new ones
None	No new pod will be created while the strategy is set to None

RayCronJob

RayCronJob is the Schema for the raycronjobs API

Field	Description	Default	Validation
apiVersion string	ray.io/v1
kind string	RayCronJob
metadata ObjectMeta	Refer to Kubernetes API documentation for fields of metadata.
spec RayCronJobSpec

RayCronJobSpec

Appears in:

• RayCronJob

Field	Description	Default	Validation
jobTemplate RayJobSpec	JobTemplate defines the job spec that will be created by cron scheduling
schedule string	Schedule is the cron schedule string
suspend boolean	Suspend tells the controller to suspend the scheduling, it does not apply to scheduled RayJob.

RayJob

RayJob is the Schema for the rayjobs API

Field	Description	Default	Validation
apiVersion string	ray.io/v1
kind string	RayJob
metadata ObjectMeta	Refer to Kubernetes API documentation for fields of metadata.
spec RayJobSpec

RayJobSpec

RayJobSpec defines the desired state of RayJob

Appears in:

• RayCronJobSpec

• RayJob

Field	Description	Default	Validation
activeDeadlineSeconds integer	ActiveDeadlineSeconds is the duration in seconds that the RayJob may be active before KubeRay actively tries to terminate the RayJob; value must be positive integer.
backoffLimit integer	Specifies the number of retries before marking this job failed. Each retry creates a new RayCluster.	0
rayClusterSpec RayClusterSpec	RayClusterSpec is the cluster template to run the job
submitterPodTemplate PodTemplateSpec	SubmitterPodTemplate is the template for the pod that will run ray job submit.
metadata object (keys:string, values:string)	Refer to Kubernetes API documentation for fields of metadata.
clusterSelector object (keys:string, values:string)	clusterSelector is used to select running rayclusters by labels
submitterConfig SubmitterConfig	Configurations of submitter k8s job.
managedBy string	ManagedBy is an optional configuration for the controller or entity that manages a RayJob. The value must be either ‘ray.io/kuberay-operator’ or ‘kueue.x-k8s.io/multikueue’. The kuberay-operator reconciles a RayJob which doesn’t have this field at all or the field value is the reserved string ‘ray.io/kuberay-operator’, but delegates reconciling the RayJob with ‘kueue.x-k8s.io/multikueue’ to the Kueue. The field is immutable.
deletionStrategy DeletionStrategy	DeletionStrategy automates post-completion cleanup. Choose one style or omit: - Legacy: both onSuccess & onFailure (deprecated; may combine with shutdownAfterJobFinishes and TTLSecondsAfterFinished). - Rules: deletionRules (non-empty) — incompatible with shutdownAfterJobFinishes, legacy fields, and global TTLSecondsAfterFinished (use per-rule condition.ttlSeconds). Global TTLSecondsAfterFinished > 0 requires shutdownAfterJobFinishes=true. Feature gate RayJobDeletionPolicy must be enabled when this field is set.
entrypoint string	Entrypoint represents the command to start execution.
runtimeEnvYAML string	RuntimeEnvYAML represents the runtime environment configuration provided as a multi-line YAML string.
jobId string	If jobId is not set, a new jobId will be auto-generated.
submissionMode JobSubmissionMode	SubmissionMode specifies how RayJob submits the Ray job to the RayCluster. In “K8sJobMode”, the KubeRay operator creates a submitter Kubernetes Job to submit the Ray job. In “HTTPMode”, the KubeRay operator sends a request to the RayCluster to create a Ray job. In “InteractiveMode”, the KubeRay operator waits for a user to submit a job to the Ray cluster. In “SidecarMode”, the KubeRay operator injects a container into the Ray head Pod that acts as the job submitter to submit the Ray job.	K8sJobMode
entrypointResources string	EntrypointResources specifies the custom resources and quantities to reserve for the entrypoint command.
entrypointNumCpus float	EntrypointNumCpus specifies the number of cpus to reserve for the entrypoint command.
entrypointNumGpus float	EntrypointNumGpus specifies the number of gpus to reserve for the entrypoint command.
ttlSecondsAfterFinished integer	TTLSecondsAfterFinished is the TTL to clean up RayCluster. It’s only working when ShutdownAfterJobFinishes set to true.	0
preRunningDeadlineSeconds integer	PreRunningDeadlineSeconds is the deadline in seconds for a RayJob to reach the Running state from when it is first initialized (StartTime). If the RayJob does not transition to Running within this time, it will be marked as Failed. This is useful for cleaning up jobs stuck in Initializing or Waiting states. If not set, there is no deadline. Value must be a positive integer.		Minimum: 1
shutdownAfterJobFinishes boolean	ShutdownAfterJobFinishes will determine whether to delete the ray cluster once rayJob succeed or failed.
suspend boolean	suspend specifies whether the RayJob controller should create a RayCluster instance If a job is applied with the suspend field set to true, the RayCluster will not be created and will wait for the transition to false. If the RayCluster is already created, it will be deleted. In case of transition to false a new RayCluster will be created.

RayService

RayService is the Schema for the rayservices API

Field	Description	Default	Validation
apiVersion string	ray.io/v1
kind string	RayService
metadata ObjectMeta	Refer to Kubernetes API documentation for fields of metadata.
spec RayServiceSpec

RayServiceSpec

RayServiceSpec defines the desired state of RayService

Appears in:

• RayService

Field	Description	Default	Validation
rayClusterDeletionDelaySeconds integer	RayClusterDeletionDelaySeconds specifies the delay, in seconds, before deleting old RayClusters. The default value is 60 seconds.		Minimum: 0
serviceUnhealthySecondThreshold integer	Deprecated: This field is not used anymore. ref: https://github.com/ray-project/kuberay/issues/1685
deploymentUnhealthySecondThreshold integer	Deprecated: This field is not used anymore. ref: https://github.com/ray-project/kuberay/issues/1685
serveService Service	ServeService is the Kubernetes service for head node and worker nodes who have healthy http proxy to serve traffics.
upgradeStrategy RayServiceUpgradeStrategy	UpgradeStrategy defines the scaling policy used when upgrading the RayService.
managedBy string	ManagedBy is an optional configuration for the controller or entity that manages a RayService. The value must be either ‘ray.io/kuberay-operator’ or ‘kueue.x-k8s.io/multikueue’. The kuberay-operator reconciles a RayService which doesn’t have this field at all or the field value is the reserved string ‘ray.io/kuberay-operator’, but delegates reconciling the RayService with ‘kueue.x-k8s.io/multikueue’ to the Kueue. The field is immutable.
serveConfigV2 string	Important: Run “make” to regenerate code after modifying this file Defines the applications and deployments to deploy, should be a YAML multi-line scalar string.
rayClusterConfig RayClusterSpec
excludeHeadPodFromServeSvc boolean	If the field is set to true, the value of the label ray.io/serve on the head Pod should always be false. Therefore, the head Pod’s endpoint will not be added to the Kubernetes Serve service.

RayServiceUpgradeStrategy

Appears in:

• RayServiceSpec

Field	Description	Default	Validation
type RayServiceUpgradeType	Type represents the strategy used when upgrading the RayService. Currently supports NewCluster, NewClusterWithIncrementalUpgrade and None.
clusterUpgradeOptions ClusterUpgradeOptions	ClusterUpgradeOptions defines the behavior of a NewClusterWithIncrementalUpgrade type. RayServiceIncrementalUpgrade feature gate must be enabled to set ClusterUpgradeOptions.

RayServiceUpgradeType

Underlying type: string

Appears in:

• RayServiceUpgradeStrategy

Field	Description
NewClusterWithIncrementalUpgrade	During upgrade, NewClusterWithIncrementalUpgrade strategy will create an upgraded cluster to gradually scale and migrate traffic to using Gateway API.
NewCluster	During upgrade, NewCluster strategy will create new upgraded cluster and switch to it when it becomes ready
None	No new cluster will be created while the strategy is set to None

RedisCredential

RedisCredential is the redis username/password or a reference to the source containing the username/password

Appears in:

• GcsFaultToleranceOptions

Field	Description	Default	Validation
valueFrom EnvVarSource
value string

ScaleStrategy

ScaleStrategy to remove workers

Appears in:

• WorkerGroupSpec

Field	Description	Default	Validation
workersToDelete string array	WorkersToDelete workers to be deleted

SubmitterConfig

Appears in:

• RayJobSpec

Field	Description	Default	Validation
backoffLimit integer	BackoffLimit of the submitter k8s job.

UpscalingMode

Underlying type: string

Validation:

• Enum: [Default Aggressive Conservative]

Appears in:

• AutoscalerOptions

WorkerGroupSpec

WorkerGroupSpec are the specs for the worker pods

Appears in:

• RayClusterSpec

Field	Description	Default	Validation
suspend boolean	Suspend indicates whether a worker group should be suspended. A suspended worker group will have all pods deleted. This is not a user-facing API and is only used by RayJob DeletionStrategy.
groupName string	we can have multiple worker groups, we distinguish them by name
replicas integer	Replicas is the number of desired Pods for this worker group. See https://github.com/ray-project/kuberay/pull/1443 for more details about the reason for making this field optional.	0
minReplicas integer	MinReplicas denotes the minimum number of desired Pods for this worker group.	0
maxReplicas integer	MaxReplicas denotes the maximum number of desired Pods for this worker group, and the default value is maxInt32.	2147483647
idleTimeoutSeconds integer	IdleTimeoutSeconds denotes the number of seconds to wait before the v2 autoscaler terminates an idle worker pod of this type. This value is only used with the Ray Autoscaler enabled and defaults to the value set by the AutoscalingConfig if not specified for this worker group.
resources object (keys:string, values:string)	Resources specifies the resource quantities for this worker group. These values override the resources passed to rayStartParams for the group, but have no effect on the resources set at the K8s Pod container level.
labels object (keys:string, values:string)	Labels specifies the Ray node labels for this worker group. These labels will also be added to the Pods of this worker group and override the --labels argument passed to rayStartParams.
rayStartParams object (keys:string, values:string)	RayStartParams are the params of the start command: address, object-store-memory, …
template PodTemplateSpec	Template is a pod template for the worker
scaleStrategy ScaleStrategy	ScaleStrategy defines which pods to remove
numOfHosts integer	NumOfHosts denotes the number of hosts to create per replica. The default value is 1.	1