Speech Recognition Not Working

[Rick Bonari]

I connected a microphone to hometroller S6 PRO and just have the HOMETROLLEREFAULT speaker client enabled with speech recognition set to listening. When I say the wake up word (computer) there is no response from homeseer (yes sire). Hometroller uses Windows 7 and I went through the complete speech recognition training session. Everything went well with the microphone and speech recognition sessions. Any thoughts on what the problem is and how to resolve it are much appreciated. Also what command do you say to execute one of your events. Should you start by saying computer and name the event. Would it be something like.....computer..... den lamp on event? I read the homeseer documentation but kind of sketchy on speech recognition. Is there any training that needs to be done to recognize specific phrases inside homeseer? Tried creating an event to recognize a phrase but that didn't work either. I assume if it won't recognize the wake up word (computer) then it won't recognize anything? Does the microphone need to be turned on in windows control panel/speech recognition? I wouldn't think so as it is set to listen in the homeseer speaker client? Thanks much for the help !!!!! Rick

[Krumpy]

I myself have not done this in a long time. Think this through logically.

If you went through the training exercise then one would think that the hardware works.

Have you looked at the Speaker APP and confirmed that the VR settings are enabled?

Second, I would recommend a longer attention phrase than "Computer".

Third, you will want to enable "voice command" each for the devices or events that you want to control.

You will want to look at the Speaker APP event log for clues to what it recognized or didn't recognize.

Other than that, you'll want to review the documentation for more information, or look at the old HS2 threads back when this was the only option in terms of voice recognition.

[Rick Bonari]

I have done all that you have said. I'm not getting anything in the main log nor am I getting anything in the speaker app log? I will try changing the wake up phrase but not sure why I'm not getting anything in either log. Doesn't seem to respond to anything and voice recognition is set to listening? Any further suggestions are much appreciated

[jhaney143]

Does anyone have a clue as to what this problem is? I tested the voice options a while back and had no issues with it. Now when I do a simple test of just one button with an action type of "homeseer: start listening for a command" the microphone comes up on android device and does listen but then nothing. And nothing shows up in logs as making it to the homeseer server. Terribly frustrating to say the least and the documentation is really non-existent. So basically worked one day and today nothing. Any help would be greatly appreciated!

[jhaney143]

Anyone maybe know where to begin to troubleshoot the problem?

[Pete]

Once you have done the VR Microsoft SAPI control panel stuff then you have to make sure that your speaker dot exe program microphone is enabled and working.

The speaker dot exe program controls the Microsoft SAPI VR and TTS.

Before configuring the speaker program you need to validate that MS SAPI is working via the Window control panel applet. Will post pictures from Windows 10 for this.

Most important part of the Microsoft SAPI configuration applet is testing your microphone for VR. If this doesn't work then speaker will not work.

Note too that you will have to configure a 32 bit speech SAPI applet if running 64 bit Windows and speaker dot exe is a 32 bit program that uses 32 bit SAPI.

Create an icon for the 32 bit SAPI control panel applet using the following. Search for the sapi.cpl; it should be there.

%windir%\syswow64\speech\speechux\sapi.cpl

Utilize this applet to test your TTS and VR (configure your microphone and test VR)

Here is the 32bit Windows 10 speech applet in control panel which is the same for Windows 7 32 bit.

[ATTACH]68543[/ATTACH]


I have tested speaker dot exe to work fine in Windows 10 way long long time ago when testing the Kinect plugin.

MS SAPI / Homeseer Speaker TTS / VR works fine in XP, Vista, W7, W8 and W10.

Step 1 - speaker options / general tab - start listening.

[ATTACH]68540[/ATTACH]

Step 2 - speaker options / Audio output / Text-to-speech tab (here never utilize the show speak phrases in tray ballon check box - you might want to leave this on to validate your VR).

[ATTACH]68541[/ATTACH]

Step 3 - options / Voice recognition tab

[ATTACH]68542[/ATTACH]

Here running 4 instances of speaker dot exe in an Oracle Windows Virtual box on the Homeseer Pro Linux box. It is from these instances that I get Homeseer text to speech.

[jhaney143]

That is one awesomely detailed response. Thank you very much. If I dumb it down a bit the key here is that the Speaker client must be functioning properly as it is also the 'listening' client for lack of a better term. Is that correct? Makes sense if it is because I moved to a little NUC box and have no audio input and when I start the Speaker client it has a red light and 'Not Listening' label.

[jhaney143]

Well. I sort of have it working. The Speaker client is set up correctly per your instructions. The actual computer will listen but it is no easy task to get the Android client to work. It doesn't seem to be a Homeseer issue but rather an issue with the server it is running on. For whatever reason all looks good at start up but the microphone keeps wanting to not be on by default which in turn seems to affect the way the client works. I'm still not entirely sure what is wrong. So if I start the computer and make sure local microphone is working then the client microphone will work.

[Pete]

thank you John....

Will copy / move post to the help DIY section here....

If I dumb it down a bit the key here is that the Speaker client must be functioning properly as it is also the 'listening' client for lack of a better term. Is that correct?

Yes.

The Homeseer speaker client is a custom Homeseer Windows SAPI application. (TTS and VR).

For whatever reason all looks good at start up but the microphone keeps wanting to not be on by default which in turn seems to affect the way the client works.

Saw similar issues here when testing on a mini Windows 10 box (PipoX7) using the Kinect plugin (remote to the mothership) and the speaker dot exe client (and HSTouch) and a multitouch wide screen open frame test monitor. And you need to leave the microphone plugged in for this to work at all times.

Try not starting speaker when rebooting and just checking the default speaker and microphone in the sapi control panel applet. It should stick on reboot. You can also connect a USB digital audio box to your NUC computer.

[jhaney143]

I still am unable to get it to work reliably. The Speaker client will talk to my android client all day long (double speak every time but that is another issue in another thread) and my microphone can talk to the Speaker client all day long but I'll be darned if I can get the Android client to speak to the Speaker client reliably. It is amazingly frustrating because all pieces seem to work independent of each other but not all together all the time. And the fact that it did work at times tells me it is possible but now I can't get it to do it again so I can say for sure how it worked. Argh. I so want this to work.

[jhaney143]

Still desperately trying to get this to work or at least understand why it isn't. I have done the following in no particular order:

* All items listed above from Pete (Thank you!)
* Disabled the built in speakers and plugged in an external speaker
* Validated the settings per the above suggestions
* Validated that Speaker client can hear sound from microphone
* Validated that Speaker client responds to wakeup command
* Validated that HS server can execute a command it hears
* Validated the Android client can connect to HS server and hear TTS commands
* Checked all logs including Windows logs for errors
* Checked different versions of Android
* Validated microphone on Android devices were working
* Disabled firewalls
* Made sure all drivers were updated and functioning properly
* Probably a dozen other things I can't remember now but I do put everything back before trying another path so I know I haven't hosed the system up completely

Not sure what else I can do. It simply will not work consistently. I did get it to work once this morning but it hasn't worked in last 6 hours or so of tinkering with every conceivable combinations of settings and countless reboots. I know it has to be some really bizarre little setting or combination of settings.

Still plugging at it since my OCD won't let me stop although my better half may insist I give up soon.

[jhaney143]

I turned on debugging option for HSTouch and can see the Android client is connected. I activate the microphone option on Android client but never see it contact the Homeseer server in logs. Should I see that connection attempt if the Android client is functioning properly?

May-04 8:58:10 AM

Speaker
Speaker host added, Name: zotac-homeseer Instance: JTH IP address: 192.168.2.110
May-04 8:58:10 AM

HSTouch Server
Connecting speaker client to HomeSeer IP 192.168.2.39
May-04 8:58:10 AM

HSTouch Server
Client Android (192.168.2.110/42672) named JTH has CONNECTED.

[Pete]

Yes over the years the Homeseer speech changed for multiple device OS access; specifically cellular phone connectivity and then tablet connectivity.

Microsoft first mobile phones (from the 1990's) used Microsoft CE which included MS SAPI. Here played with (tinkered for home and work) with first palm based mobile phones using modem speed internet access then initial 2G/3G cellular phones (back then it was only Palm and Microsoft).

Homeseer's first in wall touchscreen used Microsoft CE. My current Omnitouch screens use embedded Linux (first generation) and embedded Microsoft CE second generation Leviton HAI Omni Touchscreens.

I have used and tested Android in house touchscreens with Homeseer Touch. They work fine. I customized the first HS Android touchscreen replacing the original OS, rooted it and hacked the kernel but never did get rid of some basics on Android that I didn't want to run so went back to using Microsoft for my in house touchscreens. Today do not like android running on my tablets so started to switch them over to Linux. Much faster surfing with a Linux kernel versus an Android kernel.

The speech using the Homeseer Touch application is just a playback of a WAV file recorded speech piece.

The status in the Homeseer 3 GUI calls the HSTouch and or Speaker dot exe instances all the same.

IE: setup / network / Connected Speaker Clients (these are the same whether using Homeseer Touch or Homeseer Speaker).

The text to speech and voice recognition only works using the speaker dot exe application which only runs in Windows and is true Microsoft SAPI

The text to speech from the Homeseer Touch client (whatever OS) is not SAPI rather just a recorded wav file playback of the speech generated by the mothership. Have a look at the Android via SSH and you will see a copy of the WAV file there which is the same copy as present on the mothership.

The methodology used by Rich is:

- speaker client on Homeseer 3 writes a wav file of the TTS and copies it over to the client that is running Homeseer touch and the client plays back the TTS.

Note here for the cellular / tablet HSTouch clients it is only TTS and not VR.

You can only get Microsoft SAPI VR / TTS using the speaker client (not the HSTouch client)

IE: only related to TTS and VR

Homeseer 3 ==> Speaker client (local or remote) ==> local or remote Microsoft SAPI TTS / VR running locally or remote only on Windows.

Homeseer 3 ==> Homeseer Touch client ==> plays back a wav file copy of the TTS and there is no VR available.

Windows uses SAPI and Linux uses whatever TTS is configured on the Linux box. IE: so talking with your RPi or Linux box you will get a robotic sounding voice. Talking with a base HS3 running on Windows you will get a windows sapi text to speech font. Here run Homeseer 3 on Ubuntu and speaker dot exe on an Oracle Virtual box running a lite version of Windows Server 32bit with Microsoft SAPI installed and using 4 instances of speaker dot exe for my speech. IE: All four instances go to my Russound amps for broadcast and I give them different speech fonts to know which HS is talking.

So for generic voice recognition with Homeseer 3 you need to utilize the Cloud and the Amazon Alexa, Google or iOS and soon Microsoft Cortana.

For non cloud in house only speech / voice recognition you would utilize Microsoft SAPI or for non cloud in house use Linux text to speech / VR.

For in house or out of house Windows tablets or computers for HSTouch you can run speaker dot exe and do true MS SAPI VR / TTS on the clients.

Android / iOS tablets in the house running Homeseer Touch will only do text to speech via a recording rather than Microsoft SAPI.

[jhaney143]

I'm sorry but that is a lot of info that I'm not quite understanding how to interpret for what I'm having an issue with.

I have no issues with the Android client using HSTouch playing the .wav file. It does it ever time it is supposed to without fail. In fact it speaks everything twice.

The issue I have is that the Android client doesn't seem to be sending to the HS server. When the microphone object is activated the Google microphone listens but then that's it. Nothing in debug log as showing the HS server received anything from the client. It has worked in the past. It worked early yesterday but I haven't been able to get it to work since.

[Pete]

Yes; Amazon cloud applications which are integrated to Homeseer.

Google cloud applications are getting integrated to Homeseer.

Relating to the use of the cloud for VR there are sub sections here:

1 - Amazon
HomeSeer Amazon Echo Skill
[URL="https://forums.homeseer.com/forumdisplay.php?f=1495"]HomeSeer Amazon Echo
2 - Google(newer)
HomeSeer and Google Home
3 - Android Voice recognition
Guessing that is under the google home stuff.

Here relating to automation I try not to have dependencies on the internet or cellular phones. But that is me. I tinker with cloud stuff all of the time.

The issue I have is that the Android client doesn't seem to be sending to the HS server. When the microphone object is activated the Google microphone listens but then that's it. Nothing in debug log as showing the HS server received anything from the client. It has worked in the past. It worked early yesterday but I haven't been able to get it to work since.

Homeseer touch on Android does not have built in VR nor microphone connectivity.

VR and use of a microphone on Android would be a separate application talking to Homeseer which I have no clue of.

You would need both sides of the Android cloud VR application running which would be a voice client on Android and Homeseer talking to Google cloud running.

Today you have cloud TTS/VR applications which are Alexa, Google, Siri and Cortana.

All of these applications are running in the cloud and require you to configure Homeseer to utilize said cloud services to be able to utilize same said services on remote control Homeseer devices whether tablets or smart phones.

Everything I wrote above is only Windows to Windows using Microsoft SAPI VR/TTS only inhouse / on LAN with no Internet dependencies.

Homeseer windows stuff that I run on Linux is actually running on a windows virtual box on the Linux server.

Linux/Windows plugins typically run fine on Linux.

Windows plugins which only run in Windows here utilize a Virtual box running windows and run the plugins such that they connect to Homeseer running on Linux (same box).