Experimental Java API¶


Java API support is an experimental feature and subject to change.

Java is a mainstream programming language for production services. Ray Serve offers a native Java API for creating, updating, and managing deployments. You can create Ray Serve deployments using Java and call them via Python, or vice versa.

This section helps you to:

  • create, query, and update Java deployments

  • configure Java deployment resources

  • manage Python deployments using the Java API

Creating a Deployment¶

By specifying the full name of the class as an argument to the Serve.deployment() method, as shown in the code below, you can create and deploy a deployment of the class.

  public static class Counter {

    private AtomicInteger value;

    public Counter(Integer value) {
      this.value = new AtomicInteger(value);

    public String call(String delta) {
      return String.valueOf(value.addAndGet(Integer.valueOf(delta)));

  public void create() {
        .setInitArgs(new Object[] {1})

Accessing a Deployment¶

Once a deployment is deployed, you can fetch its instance by name.

  public void query() {
    Deployment deployment = Serve.getDeployment("counter");

Updating a Deployment¶

You can update a deployment’s code and configuration and then redeploy it. The following example updates the "counter" deployment’s initial value to 2.

  public void update() {
        .setInitArgs(new Object[] {2})

Configuring a Deployment¶

Ray Serve lets you configure your deployments to:

The next two sections describe how to configure your deployments.

Scaling Out¶

By specifying the numReplicas parameter, you can change the number of deployment replicas:

  public void scaleOut() {
    Deployment deployment = Serve.getDeployment("counter");

    // Scale up to 10 replicas.

    // Scale down to 1 replica.

Resource Management (CPUs, GPUs)¶

Through the rayActorOptions parameter, you can reserve resources for each deployment replica, such as one GPU:

  public void manageResource() {
    Map<String, Object> rayActorOptions = new HashMap<>();
    rayActorOptions.put("num_gpus", 1);

Managing a Python Deployment¶

A Python deployment can also be managed and called by the Java API. Suppose you have a Python file counter.py in the /path/to/code/ directory:

from ray import serve

class Counter(object):
    def __init__(self, value):
        self.value = int(value)

    def increase(self, delta):
        self.value += int(delta)
        return str(self.value)

You can deploy it through the Java API and call it through a RayServeHandle:

import io.ray.api.Ray;
import io.ray.serve.api.Serve;
import io.ray.serve.deployment.Deployment;
import io.ray.serve.generated.DeploymentLanguage;
import java.io.File;

public class ManagePythonDeployment {

  public static void main(String[] args) {

        System.getProperty("java.class.path") + File.pathSeparator + "/path/to/code/");

    Serve.start(true, false, null);

    Deployment deployment =
            .setInitArgs(new Object[] {"1"})



Before Ray.init or Serve.start, you need to specify a directory to find the Python code. For details, please refer to Cross-Language Programming.

Future Roadmap¶

In the future, Ray Serve plans to provide more Java features, such as:

  • an improved Java API that matches the Python version

  • HTTP ingress support

  • bring-your-own Java Spring project as a deployment