Announcement

Collapse
No announcement yet.

HS3 Speaker Client code? I want to make a Pi3 client using Win10 IoT.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    Oo, so I would have to migrate HS3 to a Windows system to get it working??


    Sent from my iPhone using Tapatalk

    Comment


      #17
      I decided to build a prototype Pi Speaker client (TTS only, no VR) on the Pi Zero, intention was to have something to go in the garden or be portable and cheap and able to speak TTS (I have no use for VR). Now this may look like two hacked up lunch containers because that is exactly what they are. For a mix between cheapness and ability to withstand a bit of rain I use them a bit in prototype projects - I might look at mounting it in a little waterproof box in the future.

      I used the Adafruit I2S MAX98357 (https://learn.adafruit.com/adafruit-...o-amp/overview
      Code:
      import RPi.GPIO as GPIO
      import time
      import socket
      import os
      
      GPIO.setmode(GPIO.BCM)
      GPIO.setwarnings(False)
      GPIO.setup(16, GPIO.OUT)
      
      speakstr = ""
      
      port = 12345
      s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
      s.bind(("", port))
      
      print ">> UDP Listener Script Starting <<"
      GPIO.output(16, GPIO.HIGH)
      
      try:
                      data, server = s.recvfrom(1024)
                      print("-> Received Data: " + data)
                      if data.find("!TTS-") <> -1:
                              GPIO.output(16, GPIO.LOW)
                              print("--> TTS To Speak")
                              speakstr = data.replace("!TTS-","")
                              os.system("pico2wave -w tmp.wav '" + speakstr + "'")
                              os.system("aplay tmp.wav")
                              print("---> TTS Spoken")
                              GPIO.output(16, GPIO.HIGH)
      
      except KeyboardInterrupt:
              GPIO.output(16, GPIO.LOW)
              s.close()
              pass
      
      except:
              GPIO.output(16, GPIO.LOW)
              s.close()
              print("> ~ Exception:")
              pass




      Comment


        #18
        Awesome keep up the good work! I would love to get TTS,music, and TTS wav file streaming working too. Also.. If your planning on commercializing this as a plugin, perhaps you can bake into your speech proxy the zoning functions and perhaps solve the remote speaker/whole house audio challenge.

        Here is something I was looking into as well but am waiting on the windows server to be ready.

        https://github.com/badaix/snapcast

        Comment


          #19
          So what's the latest on this project? I'm sure lot of people would love to start building them, I know I would especially if it work for command input too. The only thing that would be better is the MATRIX Voice Wi-Fi/BT/MCU module communicating back to the server.
          HomeSeer Version: HS3 Standard Edition 3.0.0.548
          Linux version: Linux auto 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
          Number of Devices: 484 | Number of Events: 776

          Enabled Plug-Ins: 3.0.0.13: AirplaySpeak | 2.0.61.0: BLBackup
          3.0.0.70: EasyTrigger | 1.3.7006.42100: LiftMaster MyQ
          4.2.3.0: mcsMQTT | 3.0.0.53: PHLocation2 | 0.0.0.47: Pushover 3P
          3.0.0.16: RaspberryIO | 3.0.1.262: Z-Wave

          Z-Net version: 1.0.23 for Inclusion Nodes
          SmartStick+: 6.04 (ZDK 6.81.3) on Server

          Comment


            #20
            Originally posted by Timon View Post
            So what's the latest on this project? I'm sure lot of people would love to start building them, I know I would especially if it work for command input too. The only thing that would be better is the MATRIX Voice Wi-Fi/BT/MCU module communicating back to the server.
            I have a Pi waiting for a project and so would give this a go.

            Comment


              #21
              Originally posted by Timon View Post
              So what's the latest on this project? I'm sure lot of people would love to start building them, I know I would especially if it work for command input too. The only thing that would be better is the MATRIX Voice Wi-Fi/BT/MCU module communicating back to the server.
              I had a look at Wireshark and the communications exchange between HS and the speaker client and it looked to be that it now uses the HS Communications Framework to talk to the clients. There is virtually no documentation about this so it would be some work to figure out how it was connecting to mimick that connection. You can see the text sent and login details so it is how it is working, the ISpeechAPI is available in .net but would need some figuring out as to how it works.

              I don't know if the OP had a similar idea to me to re-broadcast the TTS on a UDP broadcast which appears the easiest method as I'm hoping that I can also receive these broadcasts on my ESP8266 modules that I have hooked up to a matrix sign - no Pi required

              Comment


                #22
                Building one of these with a RaspBerry Pi is great but how about a totally standalone version using the MATRIX Voice WiFi/BT/MC module.

                This could have both a Speaker client, Amazon Echo client and even a Gogle Home Client in one device. Since each would be triggered with it's on key word there would be no interference between them. It also solves the problem for those that worry that big brother is listening since you could configure it with the Echo or Google home disabled or even removed.

                The voice side is basically done but Homeseer really needs to release the Speaker Client information to developers so it doesn't have to be reverse engineered. Then the rest can be quickly engineered.

                Take this one step further and add a Wave chip and it can be a Z-Net as well. Now that would be one KickAss device.

                So come on HomeSeer, open it up.

                Last edited by Timon; June 1, 2017, 05:16 AM.
                HomeSeer Version: HS3 Standard Edition 3.0.0.548
                Linux version: Linux auto 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
                Number of Devices: 484 | Number of Events: 776

                Enabled Plug-Ins: 3.0.0.13: AirplaySpeak | 2.0.61.0: BLBackup
                3.0.0.70: EasyTrigger | 1.3.7006.42100: LiftMaster MyQ
                4.2.3.0: mcsMQTT | 3.0.0.53: PHLocation2 | 0.0.0.47: Pushover 3P
                3.0.0.16: RaspberryIO | 3.0.1.262: Z-Wave

                Z-Net version: 1.0.23 for Inclusion Nodes
                SmartStick+: 6.04 (ZDK 6.81.3) on Server

                Comment


                  #23
                  Originally posted by Timon View Post
                  Building one of these with a RaspBerry Pi is great but how about a totally standalone version using the MATRIX Voice WiFi/BT/MC module.

                  This could have both a Speaker client, Amazon Echo client and even a Gogle Home Client in one device. Since each would be triggered with it's on key word there would be no interference between them. It also solves the problem for those that worry that big brother is listening since you could configure it with the Echo or Google home disabled or even removed.

                  The voice side is basically done but Homeseer really needs to release the Speaker Client information to developers so it doesn't have to be reverse engineered. Then the rest can be quickly engineered.

                  Take this one step further and add a Wave chip and it can be a Z-Net as well. Now that would be one KickAss device.

                  So come on HomeSeer, open it up.

                  Completely unrelated, but love the name and avator .

                  Comment


                    #24
                    Originally posted by waynehead99 View Post
                    Completely unrelated, but love the name and avator .
                    Not totally because the basic MATRIX Voice module makes an excellent voice analyzer for the Pi version, if you could somehow add it to the existing app, but if one could take it to the next level using the MATRIX Voice WiFi/BT/MC standalone module would be the best and most economical way to go.

                    The point still remains that, IMHO, none of this can be easily and quickly done without Homeseer releasing the communications protocol, and possibly even the entire Speaker Client source, so developers could create Speaker Clients across multiple platforms easily. Forcing the Speaker Client to be stuck on Windows is not the best interest for the on going development of VR for HS3.

                    Note to developers, I highly recommend that any Speaker Client be open source so that users can be assured that there is no way it cab be hacked so the microphones can be accesses from the outside.

                    All of that said, if a Windows IoT version on Raspberry Pi running the existing Speaker Client can get up soon I'll seriously consider using it.
                    HomeSeer Version: HS3 Standard Edition 3.0.0.548
                    Linux version: Linux auto 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
                    Number of Devices: 484 | Number of Events: 776

                    Enabled Plug-Ins: 3.0.0.13: AirplaySpeak | 2.0.61.0: BLBackup
                    3.0.0.70: EasyTrigger | 1.3.7006.42100: LiftMaster MyQ
                    4.2.3.0: mcsMQTT | 3.0.0.53: PHLocation2 | 0.0.0.47: Pushover 3P
                    3.0.0.16: RaspberryIO | 3.0.1.262: Z-Wave

                    Z-Net version: 1.0.23 for Inclusion Nodes
                    SmartStick+: 6.04 (ZDK 6.81.3) on Server

                    Comment


                      #25
                      Originally posted by Timon View Post
                      Not totally because the basic MATRIX Voice module makes an excellent voice analyzer for the Pi version, if you could somehow add it to the existing app, but if one could take it to the next level using the MATRIX Voice WiFi/BT/MC standalone module would be the best and most economical way to go.

                      The point still remains that, IMHO, none of this can be easily and quickly done without Homeseer releasing the communications protocol, and possibly even the entire Speaker Client source, so developers could create Speaker Clients across multiple platforms easily. Forcing the Speaker Client to be stuck on Windows is not the best interest for the on going development of VR for HS3.

                      Note to developers, I highly recommend that any Speaker Client be open source so that users can be assured that there is no way it cab be hacked so the microphones can be accesses from the outside.

                      All of that said, if a Windows IoT version on Raspberry Pi running the existing Speaker Client can get up soon I'll seriously consider using it.
                      Actually it probably isn't beyond the realm of possibility, there is an (unpublished in the SDK) JSON call to put speech text into HS via a HTTP call that I forgot about. It probably would be a little bit of code either on a Pi or inside the ESP32 to capture the captured voice and send it to HS to interpret and do whatever is appropriate. Thinking about it more I am not sure there is an absolute need for the inner workings of the speaker client to become known, you probably would not get the connection like the speaker clients but I'm sure you could end up with some solution that gets the majority of the functionality.

                      However forgive me having no great knowledge of this but why wouldn't you just use an Echo Dot?

                      Comment


                        #26
                        I'm also definitely interested in this. I've been looking for a way to make use of the few pi's that I have to somehow get VR in place like I currently have for the kinect. Much easier to have a bunch of these little pi units sitting around with a mic than it is to have a full blown windows machine available everywhere for the kinect.

                        I wouldn't mind the echo dots either if we could bypass the amazon junk and just send speech right to homeseer with our own trigger.

                        Comment


                          #27
                          Originally posted by mrhappy View Post
                          However forgive me having no great knowledge of this but why wouldn't you just use an Echo Dot?
                          Because I don't like the Big Brother way of doing it. Since Echo isn't open source we have no idea if it could be used to listen in to our homes. That's why I've said these devices should be open source so the code can be vetted.
                          HomeSeer Version: HS3 Standard Edition 3.0.0.548
                          Linux version: Linux auto 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
                          Number of Devices: 484 | Number of Events: 776

                          Enabled Plug-Ins: 3.0.0.13: AirplaySpeak | 2.0.61.0: BLBackup
                          3.0.0.70: EasyTrigger | 1.3.7006.42100: LiftMaster MyQ
                          4.2.3.0: mcsMQTT | 3.0.0.53: PHLocation2 | 0.0.0.47: Pushover 3P
                          3.0.0.16: RaspberryIO | 3.0.1.262: Z-Wave

                          Z-Net version: 1.0.23 for Inclusion Nodes
                          SmartStick+: 6.04 (ZDK 6.81.3) on Server

                          Comment

                          Working...
                          X