Demystifying AI Integration with Cloud Foundry: A Spring AI Deep Dive

The rise of AI has created new opportunities for enterprises to enhance their applications with natural language interfaces and smart features. However, many organizations struggle with the practical aspects of integrating AI capabilities into their existing systems. This is where Spring AI and Cloud Foundry come together to offer a powerful solution.

In a recent Cloud Foundry Weekly podcast (Season 1, Episode 26: Spring AI Zero to Hero), Distinguished Engineer Adib Saikali demonstrated how enterprises can practically integrate AI capabilities into their existing systems. This blog post explores the key insights from that discussion and provides a roadmap for organizations looking to enhance their applications with AI.

{{ embed video here – https://www.youtube.com/live/vyZSmePUiCE?si=7tyX9NKITmzNhU9B }}

The Power of Spring AI

Spring AI brings the same style and simplicity to AI integration that Spring Data brought to database connectivity. It provides a unified interface for working with various AI large language models and services, allowing developers to write code once and switch between different AI providers as needed.

The framework supports:

• Chat models from providers like OpenAI, Anthropic, HuggingFace and Ollama
• Image generation and analysis
• Audio processing
• Vector databases for efficient similarity search
• Function calling for API integration

Model Portability: Write Once, Run Anywhere

One of Spring AI’s most powerful features is its ability to switch between different LLM providers without changing application code. Here’s a practical example:

@RestController
public class ChatController {
    private final ChatClient chatClient;

    @GetMapping("/chat/joke")
    public String getJoke() {
        // This same code works with any supported LLM
        return chatClient.call("Tell me a joke");
    }
}

To switch from OpenAI to Ollama, simply update your application.properties:

# Using OpenAI
spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.provider=openai

# Switch to Ollama
spring.ai.ollama.base-url=http://localhost:11434
spring.ai.provider=ollama

This flexibility allows organizations to:

• Experiment with different models during development
• Switch to more cost-effective providers
• Meet compliance requirements by changing to approved models
• Take advantage of new AI models as they become available

Bridging AI and Enterprise API

Spring AI’s function calling lets AI naturally work with your existing business systems. Think of it as a translator between AI conversations and your APIs – when a user asks a question, the AI knows when to tap into your services for real data or actions. Here’s how simple it looks in practice:

@RestController
public class WeatherController {
    private final ChatClient chatClient;
    private final WeatherService weatherService;

    public ResponseEntity<String> getWeatherAdvice(String city) {
        // AI determines when to call the weather service
        // and formats the response naturally
        return chatClient.call(prompt, weatherFunction);
    }
}

The AI model can decide when to call your enterprise APIs based on user questions, bridging the gap between natural language and your business logic.

Cloud Foundry Integration

The Tanzu Application Service (TAS) makes deploying AI-enabled applications straightforward through:

1. The Gen AI tile, providing managed access to various AI models
2. Built-in support for vector databases like Postgres with pgvector
3. Simple deployment of Spring Boot applications with AI capabilities

Best Practices for Implementation

The most important principle for AI integration is simple: just start building. Innovation comes through experimentation, and the Cloud Foundry platform combined with Spring AI provides an ideal environment for rapid prototyping and iterative development.

Here are key practices to guide your journey:

1. Start Small and Iterate
   • Begin with a simple use case like adding a chatbot to an existing application
     • Clone the Spring AI Zero to Hero Github for samples
   • Test different prompts and system messages to optimize responses
   • Gather feedback from real users early and often
2. Use Function Calling Strategically
   • Map existing APIs to natural language interfaces
   • Keep functions focused and single-purpose
   • Test with various prompts to ensure consistent function execution
3. Test Multiple Models
   • Take advantage of Spring AI’s model portability
   • Compare response quality across different providers
   • Consider factors like cost, latency, and accuracy for your use case
4. Implement Data Controls
   • Use function calling to limit data exposure to AI models
   • Keep sensitive information within your existing security boundaries
   • Monitor and log AI interactions for compliance
5. Monitor and Scale
   • Start with development-grade models like Ollama
   • Track usage patterns and costs
   • Scale to production models once patterns are established

Remember: perfect is the enemy of good. Your first AI implementation doesn’t need to solve every use case or handle every edge case – it just needs to address a real business need. The technology and tools are ready today, and the best path forward is to start building, learning, and improving through hands-on experience

Looking Forward

The combination of Spring AI and Cloud Foundry fundamentally changes how enterprises can approach AI integration. Organizations are discovering they can deliver AI features significantly faster, while maintaining the security and reliability their customers expect. By standardizing AI integration patterns and protecting existing investments in enterprise APIs and data, teams can focus on solving real business problems instead of wrestling with infrastructure.

The path forward is clear and accessible. Start by cloning the Spring AI Zero to Hero repository and running the samples locally. Connect with others building AI applications by joining the conversation at Spring Stack Overflow , where you’ll find a growing community of developers sharing experiences and best practices.

We want to hear your story. What business challenges could AI help solve in your organization? Reach out to your Tanzu team to schedule a hands-on workshop, or join Dan Vega and DaShaun Carter for Spring Office Hours for direct learning from the experts.Don’t miss our weekly discussions on the Cloud Foundry Weekly podcast, where we regularly feature technical talks and practical implementations like this one. Be sure to subscribe to our channel to stay updated on the latest in Cloud Foundry, Spring AI, and enterprise application development. Together, we’re unleashing PaaS Potential One Episode at a Time.