Van Gogh VR Experience

Visiodrom Wuppertal Virtual Reality Experience

"How can art be brought to life through AI?"

The Van Gogh VR Experience at Visiodrom Wuppertal merges art and technology into an immersive encounter. Using virtual reality, visitors step inside the mind of Vincent van Gogh and experience him in an entirely new, interactive way.

The project covered the UX concept, interface design, coordination with the technical requirements of the VR platform, and using AI as an interactive chatbot.

Result — in brief

— A working VR dialogue flow with voice & visual feedback.
— Stable voice interaction (push-to-talk) with latency UI.
— A persona-derived tone guide for consistent microcopy.

Project Details

Team	Tom Ziegler, Joost Haschen, Nicklas Spiering
My role	UX concept, prompting for ChatGPT and Whisper, AI workflows
Technologies	ChatGPT , Whisper , Figma , Unity , Notion , Blender
Keywords	Exhibition, Voice Design, Persona Design, Spatial UX, AI

The Experience

Immersive virtual reality meets AI-powered interaction

Software Demo

AI Integration

How artificial intelligence brings Van Gogh to life

Why AI?

Many immersive Van Gogh formats rely on projection rather than genuine interaction. We wanted dialogue-based knowledge transfer — low-barrier, via voice in VR.

AI Architecture

Push-to-talk triggers speech-to-text; the response comes from an LLM persona (system prompt), is delivered via text-to-speech, and Unity visualises a waiting indicator for transparency.

Implementation: first Python prototype with OpenAI API, then a full C# port for audio I/O, controller integration, and model integration.

Persona & Prompting

I designed a persona based on Van Gogh's letters and implemented it as a system prompt with stable defaults, avoiding Markdown pitfalls. The guiding framework was CAPITAL (e.g. amicable, transparent, accessible vocabulary).

Our narrative pivot: instead of an avatar with lip sync, users hold a conversation "inside the mind" — consistent with both the persona and the tech. It also sidestepped the lip-sync problem.

Quality & Safety

Negative testing with provocation and insults: the persona remained calm and respectful. Team ratings and feedback from lecturers and fellow students served as quality assurance.

Latency & Robustness

The bottleneck was network latency on the exhibition Wi-Fi. We made it visible via a "Waiting…" state and mitigated it using a private hotspot.

Sound & Immersion

The voice output was tuned to meet historical expectations (e.g. accent, vocal tone). Reverb in Unity creates the "inside the mind" effect.

Future & Improvements

Next up: integrating Van Gogh's letters as a local knowledge base via RAG. The persona would respond exclusively based on letter excerpts and can cite letter ID/date if needed.

The prompt would also receive more context about the environment (room, station, visible works) — so responses reference the visitor's location ("You are standing in the room from Arles; on the wall hang the Sunflowers from 1888").

In a "letters-only" setup, the installation could run fully offline — making it internet-independent. Realistic in the near future thanks to open-source LLMs.

Technical Details

Framework	CAPITAL (Confidence, Amicability, Professionalism, Interactivity, Transparency, Adaptability, Lexicography)
Persona & Sources	Style & attitude derived from Van Gogh's letters; system prompt as guardrail
Interaction	Push-to-talk, spoken response (TTS), latency feedback in UI
Quality assurance	Team review, negative tests (provocation/insults), prompt versioning

Back to projects