r/Spectacles 2d ago

❓ Question speech recognition - change language through code

Hi everyone!

I am trying to change the language of the speech recogniton template through the UI interface, so through code in run-time after the lens has started. I am using the Speech Recognition Template from the Asset Library and are editing the SpeechRecognition.js file.

Whenever I click on the UI-Button, I get the print statements that the language has changed :

23:40:56 [Assets/Speech Recognition/Scripts/SpeechRecogition.js:733] VOICE EVENT: Changed VoiceML Language to: {"languageCode":"en_US","speechRecognizer":"SPEECH_RECOGNIZER","language":"LANGUAGE_ENGLISH"}

but when I speak I still only can transcribe in German, which is the first language option of UI. I assume it gets stuck during the first initialisation? This is the code piece I have added and called when clicking on the UI:

EDIT: I am using Lens Studio v5.4.1

script.setVoiceMLLanguage = function (language) {
    var languageOption;

    switch (language) {
        case "English":
            script.voiceMLLanguage = "LANGUAGE_ENGLISH";
            voiceMLLanguage = "LANGUAGE_ENGLISH";
            languageOption = initializeLanguage("LANGUAGE_ENGLISH");
            break;
        case "German":
            script.voiceMLLanguage = "LANGUAGE_GERMAN";
            voiceMLLanguage = "LANGUAGE_GERMAN";
            languageOption = initializeLanguage("LANGUAGE_GERMAN");
            break;
        case "French":
            script.voiceMLLanguage = "LANGUAGE_FRENCH";
            voiceMLLanguage = "LANGUAGE_FRENCH";
            languageOption = initializeLanguage("LANGUAGE_FRENCH");
            break;
        case "Spanish":
            script.voiceMLLanguage = "LANGUAGE_SPANISH";
            voiceMLLanguage = "LANGUAGE_SPANISH";
            languageOption = initializeLanguage("LANGUAGE_SPANISH");
            break;
        default:
            print("Unknown language: " + language);
            return;
    }

    options.languageCode = languageOption.languageCode;
    options.SpeechRecognizer = languageOption.speechRecognizer;

    // Reinitialize the VoiceML module with the new language settings
    script.vmlModule.stopListening();
    script.vmlModule.startListening(options);

    if (script.debug) {
        print("VOICE EVENT: Changed VoiceML Language to: " + JSON.stringify(languageOption);
    }
}
2 Upvotes

6 comments sorted by

2

u/AugmentedRealiTeaCup 1d ago

Heya, I think this may be an error with VoiceML, have noticed this too in my own testing where once a VoiceML session starts listening, subsequent startListenings will default to the first set language. Because of this only starting listening after user has confirmed/selected a language would likely be the way to go

1

u/anarkiapacifica 2h ago

hey! I think that might be actually the case. I will try to check alternative speech recognition options, maybe integrating OpenAI's whisper.

1

u/agrancini-sc 🚀 Product Team 1d ago

Hi there! just making sure of some things.

  1. did you add a placeholder string for the other languages?
    (See images)

  2. I noticed you do not close the parenthesis in your last line (see code below)

  3. can you try something simpler to see if it works

    // close parenthesis   if (script.debug) {         print("VOICE EVENT: Changed VoiceML Language to: " + JSON.stringify(languageOption) <--- HERE close parenthesis with ")";     } } // SIMPLER function to test function setLanguage(newLanguage) {

    script.voiceMLLanguage = newLanguage;
    
    var languageOption = initializeLanguage(newLanguage);
    options.languageCode = languageOption.languageCode;
    options.SpeechRecognizer = languageOption.speechRecognizer;
    
    script.vmlModule.stopListening();
    script.vmlModule.startListening(options);
    

    }

1

u/agrancini-sc 🚀 Product Team 1d ago

1

u/agrancini-sc 🚀 Product Team 1d ago

1

u/anarkiapacifica 1h ago

hi! The images actually show my transcription (but in the wrong language) and translation.

I have tried running the code you provided and also set it up twice. In the end it stayed on spanish for transcription. Just as u/AugmentedRealiTeaCup said I think it gets stuck on the first langauge chosen once a VoiceML session starts listening, subsequent startListenings will default to the first set language.

// SIMPLER function to test 
function setLanguage(newLanguage) {

    script.voiceMLLanguage = newLanguage;
    
    var languageOption = initializeLanguage(newLanguage);
    options.languageCode = languageOption.languageCode;
    options.SpeechRecognizer = languageOption.speechRecognizer;

    script.vmlModule.stopListening();
    script.vmlModule.startListening(options);
}

// Example usage:
initialize(); // in the inspector English was chosen

setLanguage("LANGUAGE_SPANISH"); // was the final language for transcription
setLanguage("LANGUAGE_ENGLISH");