From the classroom to the skies: How NYP students developed a VR training system for the RSAF

SINGAPORE: Staring out at the open expanse of ocean through the fluffy clouds, in the distance there are shapes on the water below.

Despite the slight queasiness in my stomach, this reporter squints to try to identify them. As we hover above, there are some familiar silhouettes which are immediately recognisable.  

There is a cruise-liner, a ferry, and a fishing vessel, each with their distinct shape and characteristics.

Wanting to get a different view, I switch my seating position.

Through binoculars pointing through the turboprop plane's passenger side window, I scan the horizon and locate other vessels further away such as a tanker.

But rather than doing all of this in the air, this is happening on solid ground, via the Maritime Air Surveillance Virtual Reality (VR) Training System.

Removing my headset and goggles, I am immediately transported back into the real world.

STRIVING FOR A REALISTIC EXPERIENCE

Part of a collaboration between Nanyang Polytechnic's School of Design & Media and the Republic of Singapore Air Force (RSAF), the VR system was developed for training for the RSAF’s Fokker 50 Maritime Patrol Aircraft. 

The system is used to help newly recruited surveillance officers to identify and classify ships while using EO/IR (Electro-Optical/Infra-Red) surveillance systems or binoculars.

It also helps new crew recognise different vessels and enables them to distinguish between various types of ships even before they embark on actual flights for further training.

"What this does is reconcile the gap between what we do on the ground and what we actually do when we fly," said CPT Huang Jianhui. CPT Huang is a Fokker-50 Tactical Coordinator for the RSAF's 121 Squadron.

The system was developed by 11 students - programmers and artists - who worked on the project over a span of six months. 

With the wide scope of the project, two batches of students worked on it, one group after the other. While the programmers in each group focused more on making the experience work, the art team designed close to 90 various ships.

"We catered to the environment ... such as the water and the clouds. So we tried to make the simulation as accurate as possible," said Lim Hon Wai, one of the team's programmers. 

Part of the artists' role was to update ships that had been designed previously by those in the first group, said 18-year-old Marcus Lim, who was on the artists' team.

"Once we were done with the assets, we would usually pass to them (the programmers) for them to integrate (into the system). Some of the challenges (included) assets not looking like what we intended it to look like in the simulation. For example, some of the textures would warp," he said.

"So from there, the art team is in charge of fixing the visual defects in the ships and from there the programmers will communicate with us on which ships are looking right ... We would fix it and then resend it to them again."

And after their hard work, the training system is now fully functional and has been used by the RSAF for several months.

"We use it as a supplementary teaching aid. So before they (trainees) go and fly, we will go through with them (that) these are the various types of ships they can expect. This is sort of a familiarisation, so once they are familiar we can fly, and ship recognition is one of the tasks we have to accomplish," said CPT Huang.

Part of the benefits of the training system is that it allows for the simulation of different weather conditions as well as variable visibility.

"We are very happy with the level of customisation that meets our requirements. For example, things like the weather and the detail of models that we can choose, it's something very helpful to us ... And we're glad to be able to tap their expertise in doing this. This is something that we won't be able to do on our own so we are glad to have them on board with us," said CPT Huang.

And while the system comes with its limitations, it provides a different dimension to training for crew members, he added.

If they weren't able to fly, crew members would previously have studied a compilation of photographs of various ship types from different angles instead.

"What it does for us is that instead of just looking at still photos when we are on the ground, what the new guy can do is that he puts on his headset and is put through this experience, and it looks realistic enough that it is able to help him in his training when he actually goes to fly," he added.

Marcus Lim noted that there is a "great sense of pride" among the team, with the training system's completion.

"It really felt satisfying seeing the simulation actually work, and the RSAF actually using it in their training," added Lim Hon Wai.