Spring AI with Azure OpenAI — Enterprise GPT-4 Deployment in Spring Boot

Spring AI with Azure OpenAI — Enterprise GPT-4 Deployment in Spring Boot

Azure OpenAI provides access to GPT-4o and other OpenAI models through Microsoft's enterprise cloud, offering data residency, compliance certifications (SOC 2, HIPAA, ISO 27001), private networking, and SLA guarantees. If your organization's data cannot leave a specific region or requires enterprise security, Azure OpenAI is the path to using GPT models. Spring AI has full Azure OpenAI support with the same API as the standard OpenAI integration.

Azure OpenAI vs Standard OpenAI

Standard OpenAI (api.openai.com):
  ✔ Faster to set up (API key only)
  ✔ Access to latest models immediately
  ✘ No data residency guarantee
  ✘ No private networking

Azure OpenAI (your-resource.openai.azure.com):
  ✔ Data residency (EU, US, etc.)
  ✔ HIPAA, SOC 2, ISO 27001 compliance
  ✔ Private endpoints (VNet integration)
  ✔ Azure Active Directory authentication
  ✔ Enterprise SLA
  ✘ Models available with delay (behind OpenAI by weeks)
  ✘ Requires Azure subscription and deployment setup

Azure Setup (One-Time)

# 1. Create Azure OpenAI resource in Azure Portal
# 2. Deploy a model (e.g., gpt-4o deployment named "gpt4o-prod")
# 3. Get endpoint and API key from "Keys and Endpoint" tab

# Your values will look like:
# Endpoint: https://my-company.openai.azure.com/
# API Key:  abc123def456...
# Deployment: gpt4o-prod
# API Version: 2024-08-01-preview

Maven Dependency

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId>
</dependency>

application.properties

spring.ai.azure.openai.api-key=${AZURE_OPENAI_API_KEY}
spring.ai.azure.openai.endpoint=${AZURE_OPENAI_ENDPOINT}
spring.ai.azure.openai.chat.options.deployment-name=gpt4o-prod
spring.ai.azure.openai.chat.options.temperature=0.7
spring.ai.azure.openai.chat.options.max-tokens=4096

# For embeddings:
spring.ai.azure.openai.embedding.options.deployment-name=text-embedding-3-small-deployment

Service — Identical Code to Standard OpenAI

@Service
public class AzureAiService {

    private final ChatClient chatClient;

    public AzureAiService(ChatClient.Builder builder) {
        this.chatClient = builder
                .defaultSystem("""
                    You are an enterprise Java assistant.
                    Provide production-ready, security-conscious answers.
                    """)
                .build();
    }

    public String ask(String question) {
        // Identical to standard OpenAI — Azure handled by Spring AI starter
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }

    public String analyzeCode(String code) {
        return chatClient.prompt()
                .user("Review this Java code for enterprise readiness: " + code)
                .call()
                .content();
    }
}

Azure Active Directory Authentication (No API Key)

<!-- pom.xml — add for AAD auth -->
<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-identity</artifactId>
</dependency>

@Configuration
public class AzureAiConfig {

    @Bean
    public OpenAIClient openAiClient(
            @Value("${spring.ai.azure.openai.endpoint}") String endpoint) {

        // Use managed identity or workload identity — no API key in config!
        TokenCredential credential = new DefaultAzureCredentialBuilder().build();

        return new OpenAIClientBuilder()
                .endpoint(endpoint)
                .credential(credential)
                .buildClient();
    }
}

# application.properties with AAD (no api-key needed)
spring.ai.azure.openai.endpoint=https://my-company.openai.azure.com/
spring.ai.azure.openai.chat.options.deployment-name=gpt4o-prod
# spring.ai.azure.openai.api-key is NOT set — uses managed identity

Private Endpoint Configuration

# When Azure OpenAI is on a private endpoint in your VNet:
spring.ai.azure.openai.endpoint=https://my-private.openai.azure.com/
spring.ai.azure.openai.api-key=${AZURE_OPENAI_KEY}

# Disable SSL verification if using self-signed cert on private endpoint:
# (Not recommended for production — use proper certificates)
# spring.ai.azure.openai.http.proxy.host=...

Content Filtering (Azure-Specific)

// Azure OpenAI applies content filters — handle ContentFilteredException
@Service
public class AzureAiServiceWithFiltering {

    private final ChatClient chatClient;

    public AzureAiServiceWithFiltering(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String ask(String question) {
        try {
            return chatClient.prompt().user(question).call().content();
        } catch (HttpClientErrorException e) {
            if (e.getStatusCode().value() == 400 &&
                    e.getResponseBodyAsString().contains("content_filter")) {
                return "Your request was blocked by content safety filters. " +
                       "Please rephrase your question.";
            }
            throw e;
        }
    }
}

Key Points

• Your service code is identical for Azure OpenAI and standard OpenAI — only the starter and properties change
• Use Managed Identity authentication in production — no API keys to rotate or accidentally expose
• Azure applies content filtering on top of OpenAI's filters — catch HttpClientErrorException with content_filter in body
• Choose the Azure region closest to your application to minimize latency (azure openai latency adds ~50-100ms vs direct openai.com)
• Azure OpenAI deployments have separate capacity — ensure your deployment quota matches your expected load before going live