P0-MS2: Part 149: AI: Speech to Text, Text to Speech, Morph 3D Character System

The last major objective in Milestone 2 is to integrate the Morph 3D character creation system. Since the art tools have recently been released, it it now possible to create custom content for the system. I was waiting for this update before integrating it.

Before I do this, I wanted to implement voice recognition into the project.

The reason why I think this is important at this early stage is that firstly, the game will support virtual reality, and typing in VR is not comfortable or, really, practical. Secondly, I think that voice recognition will improve immersion and provide more realism to the gameplay.

After some research, I have implemented the Google Cloud Speech API into the project. This is not a free solution, but you do get 60 minutes free per month, so it should be essentially free as long as you don’t go over that. The accuracy of this system is exceptional, provided you speak carefully, it is almost 100% accurate. The problem is that it is web-based, it sends data to a server and then sends the response back to the client, this make it slow, although it is still technically “realtime”.

I could not implement “Dictation Recognizer” into my project (this is the standard Windows speech API) because apparently the Windows 10 anniversary update broke the Dictation Recognizer in 64-bit builds of unity (the editor and game exe). However, the upcoming windows 10 “Creators Update” is supposed to fix this, so I can evaluate “Dictation Recognizer” then. I suspect this system won’t be as accurate as Google’s system, but it is free, and client side, so it should be faster.

There are a few other speech api’s out there, but they either require a monthly subscription, or they do not support real-time streaming speech recognition.

I have combined the Voice recognition system with the AIML chat bot system, so I can actually “talk” to the bot verbally now instead of having to type. I intend to add Text to Speech functionality in the next week or so, so that the bot can “talk” back.