[REL] SH3VoiceCommand v1.0 Official Release

[shadow_wxh]

====================== SH3VoiceCommand v1.0 Official Release ======================


After nearly six months development and much time preparation in advance. I am proudly announcing that SH3VoiceCommand v1.0 is officially released.

SH3VoiceCommand is a speech to command interface dedicated to support Silent Hunter game series. It uses state of the art AI speech recognition engine and offers a much greater complexity and functionality which surpass its' predecessor and peers.

It offers these following features:

• 🎈 Lightweight: Only adds about 20MB RAM in the 32bit section.
• 📣Responsive: The command delay can be as low as 500MS with proper setup & optimization.
• 👶Independent: No dependency on any third-party software or MOD (except the OS and graphic card packages).
• 🎮Integrated: Completely merged with sh3.exe running on parallel thread which has no interference with users' keyboard & mouse inputs
• 🔠Multilingual: Supports 99 languages (current only configured for English and German, finetuned for German).
• 💡 Awareness: Aware of players' current coordinates heading speed and depth etc., and act accordingly.
• 🔢Numeracy: It can interpret any number from 0 to 1,999,999.99 in English and German
• 🎓Intelligence: Able to comprehend basic arithmetic symbols and operators like + - * / sin() cos() tan() pi and e in the configuration.
• 🤨Tolerable: Allows small deviation from standard command phrase to achieve higher catch rate
• 📎 Flexible: User can add, remove, change any voice command as they like in the config file
• 🤗Trainable: The language models can be finetuned to improve accuracy and adjust to personal accent

Disclaimer:

• This MOD is under CC BY-NC-SA license which it under the following terms:
• 👨‍💻Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. You may publish a modified version of this MOD however it must bear a different name e.g. you shall not publish "SH#VoiceCommand v2.0". (note)
• ⛔NonCommercial — You may not use the material for commercial purposes . You may not list this material on any paid to download website.
• 👨🏻‍🤝‍👨🏻ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. Any application derived from or incorporating this material must also be free.
• (note) # stands for any number between 0~9 currently targeting to support 3, 4 and 5

Minimum System Requirements:

• Operation System: Windows10 64bit
• RAM: 12GB
• Graphic Card: Nvidia Graphic Card with CUDA 6.1
• VRAM: 4GB
• Sound Card: Any sound card with microphone input

To install the MOD to your SH3 simply download the RARs in the download links in the bottom section of this post, read through user manual section one to seven and follow the instructions.

Some of the In-Game Screenshots



Backend Service and Logs



System Architecture


Known Issues:

• Everyone has unique voice characteristics. It is likely each individual player needs to construct their own version config and language models. This may require a lot of time and dedication.
• Due to decimal rounding and ocean current flow the confirmed command from the crew and actual command from user may have ±1 difference.
• Depending on the user's graphic card performance and 3D workload the respond from transcriber can be slow. If this happens, switching on a live streaming application or a screen recording application can greatly improves the processing speed.
• Upon exiting the game you might encounter invalid memory access error, e.g. exception on 0x5D9104F9 (C0000005, EXCEPTION_ACCESS_VIOLATION), this is caused by SDL2.dll not being unloaded in the correct sequence. Simply ignore this warning because the game process has been terminated anyway.
• (Personally, I am not very satisfied with the workaround fix above but in current conditions it is the easiest way without changing sh3.exe itself. If you have any recommendations, please contact me through subsim messages, any help is appreciated)
• If game crashed while using SH3VoiceCommand the share memory might be leaked and can't be retrieved again. The transcriber will indicate "Share Memory NOT Ready" and exit after several attempts. You might need to restart windows in order to load SH3VoiceCommand again.
• Some of the command phrases may not be very accurate, have grammar mistakes or even missing. If you have any recommendations, please feel free to leave a comment under the publication page of SH3VoiceCommand on subsim. Also state your reference regarding where is your source of information.

References & Credits

1. https://openai.com/index/whisper/
   Main web page of OpenAI Whisper
2. https://github.com/ggerganov/whisper.cpp/
   Open-source C++ version of Whisper
3. https://cdn.openai.com/papers/whisper.pdf
   Research paper of Whisper
4. https://huggingface.co/blog/fine-tune-whisper
   Detailed guide on how to fine tune Whisper by Sanchit Gandhi
5. https://huggingface.co/datasets/mozilla-foundation/common_voice_11_0
   Mozilla Common Voice 11.0 dataset
6. https://huggingface.co/bofenghuang/whisper-medium-cv11-german
   Bofenghuang's model after training with Mozilla Common Voice 11.0
7. https://huggingface.co/datasets/shadow-wxh/VoiceCommandAudio
   shadow_wxh's dataset on fine tuning SH#VoiceCommand
8. https://huggingface.co/shadow-wxh/voicecommand-german-medium
   shadow_wxh's model after fine tuning with VoiceCommandAudio
9. https://maritime.org/doc/subphrase/
   Some of the English command phrases are referenced from this document
10. https://translate.google.com/?sl=en&tl=de&op=translate
    Some of the German command phrases are translated using google translate
11. https://www.subsim.com/radioroom/showthread.php?t=215433
    The menu config file is derived from Ahnenerbe WideGui
12. https://www.subsim.com/radioroom/showthread.php?t=132228&highlight=sh3speech
    Some of the command phrases are inherited from SH3Speech
13. Das Boot -Filmed by Wolfgang Petersen 1981
    Some of German command phrases and taken from this film
14. https://lostsouthpacificadventures.blogspot.com/2014/09/15000-miles-from-past-3.html
    Photo of U-Boat speaking tubes
15. https://www.flickr.com/photos/lensjoy/21528336803/
    Speaking tubes onboard ORP Błyskawica taken by Marcin Bieszczanin. The voice pipe icon is edited from this.
16. Photos of speaking tubes on board U-995 taken by Toyotagt86.
    This is used as publication front page photo
17. https://store.ubisoft.com/us/silent-hunter-3/5704fac788a7e32b078b4692.html?lang=en_US
    Last but not least the original Silent Hunter III developed/published by Ubisoft January 2005

🙏Special thanks to Marcin Bieszczanin and Toyotagt86 for permission to use their excellent photographs on board ORP Błyskawica and U955🙏

OneDrive Download Links:
SH3VoiceCommand.part1.rar https://1drv.ms/u/s!AsHzobklG8RTjVom...JL8jI?e=qiswuQ
SH3VoiceCommand.part2.rar https://1drv.ms/u/s!AsHzobklG8RTjVwa...x8LC3?e=nFJgrZ
SH3VoiceCommand.part3.rar https://1drv.ms/u/s!AsHzobklG8RTjVvf...3FAiW?e=bBJYWc
SH3VoiceCommand.part4.rar https://1drv.ms/u/s!AsHzobklG8RTjVkK...IJYzu?e=Xdumcm
SH3VoiceCommandHotFix.rar https://1drv.ms/u/s!AsHzobklG8RTjV_b...3v-5r?e=USCX3M

FTP Download Links: (Limited Time & Connection Only, IP Address Subjected to Change)
SH3VoiceCommand.part1.rar ftp://Anonymous@123.191.118.202/SH3V...mand.part1.rar
SH3VoiceCommand.part2.rar ftp://Anonymous@123.191.118.202/SH3V...mand.part2.rar
SH3VoiceCommand.part3.rar ftp://Anonymous@123.191.118.202/SH3V...mand.part3.rar
SH3VoiceCommand.part4.rar ftp://Anonymous@123.191.118.202/SH3V...mand.part4.rar
SH3VoiceCommand.part5.rar ftp://Anonymous@123.191.118.202/SH3V...mand.part5.rar

[Aktungbby]

shadow_wxh!

[Anvar1061]

[rik007]

I'm a big fan of voice command so I will give it a try. The ftp links give a blanc screen but the OneDrive downloads worked perfectly!

Nice documentation! Beautiful that numbers are supported.

Thank you very much for your huge effort



Do we really need the full CUDA Toolkit? It weighs a whopping 3 GB.... I see it look for:

Cudart64_12.dll
Cublas64_12.dll

Not sure it needs more of course....

[shadow_wxh]

Apologize guys missing files are now added, the downloads has became 5 parts. The CUDA DLLs are now included. The alternative is to install NVIDIA GPU Computing Toolkit but that is to large and only needed by developers.

[rik007]

Thank you very much! Great you included the VoicePipe tga's as well. Now I see the VoicePipes and once I activated them (gold color) I voiced a German command - default I see - it sometimes turns red and sometimes nothing happens. What could that be? The log file does not show any error.

[shadow_wxh]

Originally Posted by rik007

Thank you very much! Great you included the VoicePipe tga's as well. Now I see the VoicePipes and once I activated them (gold color) I voiced a German command - default I see - it sometimes turns red and sometimes nothing happens. What could that be? The log file does not show any error.

Did the transcriber start properly. What is the specs of your graphic card?

[rik007]

It's a NVIDIA Geforce RTX 3060. I'm running W10 64bit.

It's a good point you mention because I notice that when sh3.exe has started the transcriber is started in the background. Once I start a mission it vanishes. I cannot see it in the process manager of W10 anymore.

If I then start it manually it give "Unknown exception share memory"

If I start it manually as administrator it starts and waits for input. If I switch back to sh3 and voice a command I see an error in the transcriber. Unfortunately it closes so quickly that I cannot read it

[John Pancoast]

Impressive work. Fwiw voice commands can also be done via Voice Attack. Much easier to use, setup, and install.
I have no connection but use it for various games.

[shadow_wxh]

Originally Posted by rik007

It's a NVIDIA Geforce RTX 3060. I'm running W10 64bit.

It's a good point you mention because I notice that when sh3.exe has started the transcriber is started in the background. Once I start a mission it vanishes. I cannot see it in the process manager of W10 anymore.

If I then start it manually it give "Unknown exception share memory"

If I start it manually as administrator it starts and waits for input. If I switch back to sh3 and voice a command I see an error in the transcriber. Unfortunately it closes so quickly that I cannot read it

Check the known issue section in the user manual see if that helps.

[rik007]

I did... So what...? Can you be more specific?

[shadow_wxh]

Originally Posted by rik007

I did... So what...? Can you be more specific?

1. You need at least 4GB graphic memory.
2. Set VoiceTranscriber.exe to high priority in the windows graphic settings.
3. Set SavAudio = TURE see if your voice can be captured with good quality.
4. See if VoiceTranscriber.exe has been started successfully if showing "Share Memory NOT Ready" reboot your computer. (real power off hard reboot to prevent memory not being cleared totally)
5. Turn on GeForce Experience screen recorder while in-game default is Alt+F9.
6. Try some of the command phrases while in game, if not catching check the log and console see what message displays.

[rik007]

Thank you for providing the extra info

[Mister_M]

Hello.

This work looks interesting. Voice command is probably a great addition to the game.

It would be amazing to be able to make the TDC calibration only by voice, as it was done in reality, so you can keep your eye on the periscope. But I guess it's not possible unfortunately?

[shadow_wxh]

Originally Posted by Mister_M

Hello.

This work looks interesting. Voice command is probably a great addition to the game.

It would be amazing to be able to make the TDC calibration only by voice, as it was done in reality, so you can keep your eye on the periscope. But I guess it's not possible unfortunately?

It is possible now.........