Project Overview
Tanker inspection is a safety-critical process in the energy industry — errors can have severe consequences. Training inspectors traditionally requires access to real tankers, qualified supervisors, and significant time investment. Our VR simulation replaced this with an immersive, AI-guided training environment where inspectors can learn, practice, and be assessed against regulatory standards without any of those constraints.
What made this project particularly interesting was the integration of an AI-driven guidance system — real-time assistance that monitored user actions and provided contextual feedback and compliance validation throughout the inspection workflow.
The AI Guidance System
This was the most technically novel aspect of the project. The AI guidance system monitored the user's inspection actions in real time and provided:
- Contextual hints — when a user hesitated or approached an area incorrectly, the AI system provided audio and visual guidance without simply giving away the answer
- Compliance validation — each inspection step was validated against regulatory standards, with the AI system flagging deviations and explaining why they matter
- Dynamic feedback — the system adapted its guidance based on user performance history, providing more support to struggling users and stepping back for confident ones
- Assessment reporting — automated generation of compliance training records at the end of each session
A key differentiator in this project was the integration of Convai — an AI-powered conversational platform designed specifically for real-time voice interactions in XR applications. Convai powered the virtual safety instructor character, enabling trainees to ask questions naturally during the inspection workflow and receive contextual, voice-based responses. This went beyond simple branching dialogue — the Convai integration allowed genuinely dynamic conversation, where inspectors could ask "why is this step important?" or "what happens if I skip this?" and receive meaningful, contextually appropriate answers.
Simulation Structure
The simulation was built around three core components — personalisation settings, a video-based training library, and a hands-on VR practical experience — designed to work together as a complete learning journey ending in a formal assessment.
One of the most important decisions in this project was building for 5 languages from the ground up — English, Hindi, Arabic, Urdu, and Pashto. The training videos, the UI, and the Convai AI assistant all operate in the trainee's chosen language. A Pashto-speaking driver receives the same quality of training as an English-speaking one — same content, same AI responses, same compliance outcome. The language selection also controls the Convai AI assistant's voice and response language — so the entire interaction loop, including generative Q&A and agentic triggers, is fully localised per trainee.
The video-based training library covered all relevant safety procedures and regulatory requirements in all five languages — ensuring every trainee entered the VR experience with the same baseline knowledge regardless of language background or prior experience.
The hands-on VR simulation covered the complete tanker workflow across multiple distinct phases. Each phase had its own set of objectives that had to be completed in the correct sequence — the system validated not just whether steps were done, but whether they were done in the right order and with the right technique, matching the precision required by the regulatory standard.
The Convai AI assistant was active throughout the entire simulation. At any point, a trainee could ask a domain-specific question in their own language and receive a real-time generated response grounded in the training domain. Agentic voice triggers — phrases like "Let's keep going" or "What's next?" — advanced the workflow without any UI interaction, keeping the trainee fully immersed in the VR environment.
Open-ended questions answered in real time. Domain knowledge grounded to prevent inaccurate safety information being generated.
Spoken intents fire Unity-side actions — advancing tasks and phases through natural speech without touching any UI button.
On completing all practical phases, trainees entered a formal assessment — the full workflow without AI guidance or contextual hints. Performance was scored against the regulatory standard and a compliance training record generated automatically on completion.
Technical Challenges
Interaction Accuracy for Safety-Critical Steps
When training for regulatory compliance, interaction accuracy matters enormously. A user must not be able to "accidentally" complete a step — each action needed to be intentional and correctly executed. We implemented precision interaction zones with confirmation gestures for safety-critical steps, preventing accidental completion.
AI Response Latency
Any delay between a user action and the AI guidance response breaks immersion and disrupts the training flow. We optimised the guidance system to respond within 200ms of detecting a relevant user action, keeping the experience feeling responsive and natural.
Tech Stack
Outcomes & ADIPEC 2025 Reception
The solution was showcased at ADIPEC Abu Dhabi 2025 — the world's most impactful energy event. The AI guidance integration received particular attention from energy industry stakeholders who recognised its potential for delivering consistent, scalable compliance training across large workforces without requiring supervisor presence for every session.
Key Lessons
- AI guidance design is a UX discipline — getting the balance right between too much AI help (which removes learning) and too little (which frustrates users) required extensive iteration and user testing.
- Regulatory requirements must drive technical decisions — the compliance engine's architecture was entirely shaped by the specific regulatory standards we needed to validate against. Understanding the domain deeply is essential before writing a line of code.
- Generative AI and agentic AI are meaningfully different — Convai in this project did both: answering open-ended questions generatively, and executing simulation actions through triggers agentically. Understanding the distinction matters for design — the conversational layer handles the unexpected, the agentic layer handles the structured workflow. Both are needed for a genuinely intelligent training system.
Frequently Asked Questions
How does AI guidance work in VR training?
In this project, the AI guidance system was Convai — a generative AI conversational platform integrated directly into the Unity VR environment. It operated in two modes: a generative conversational mode where trainees could ask any inspection-related question and receive real-time voice responses, and an agentic mode where specific spoken intents triggered Unity-side actions to advance the simulation workflow. This went well beyond rule-based guidance — the Convai character could handle questions that were never anticipated during development, grounded in the inspection domain's knowledge base.
Can VR training meet regulatory compliance requirements?
Yes — when properly designed, VR training systems can generate compliance records that meet regulatory training requirements. The key is accurate tracking of all required steps, tamper-proof session logging, and validation against the specific regulatory standard. This project was designed with those requirements at its core.