Short Example: Streaming OpenAI Chat Completions in Java Using WebClient

 

Here is a short example of how to stream OpenAI chat completions using Spring's WebClient and Flux in Java.

First, set up your WebClient:

 

private final WebClient webClient;

public YourClassName(String openAiBaseUrl, String openAiApiKey) {
    this.webClient = WebClient.builder()
        .baseUrl(openAiBaseUrl)
        .defaultHeader("Authorization", "Bearer " + openAiApiKey)
        .build();
}

Then, send a request to stream chat completions:

public Flux<ChatCompletionResponseChunk> streamChatCompletion(ChatCompletionRequest request) {
    return webClient.post()
        .uri("/chat/completions")
        .header("Accept", "text/event-stream")
        .contentType(MediaType.APPLICATION_JSON)
        .bodyValue(request)
        .retrieve()
        .bodyToFlux(ChatCompletionResponseChunk.class)
        .onErrorResume(error -< {
            // See https://hilla.dev/blog/ai-chatbot-in-java/calling-chatgpt-and-openai-apis-in-spring-boot/
            // The stream terminates with a `[DONE]` message, which causes a serialization error.
            // Ignore this error and return an empty stream instead.
            if (error.getMessage().contains("JsonToken.START_ARRAY")) {
                return Flux.empty();
            } else {
                // If the error is not caused by the `[DONE]` message, propagate the error.
                return Flux.error(error);
            }
        });
}