www.homeseer.com    
 

Go Back   HomeSeer Message Board > Gaming Related Plug-ins > Gaming Related Discussion > Kinect (3P)

Kinect (3P) Discussions related to the Kinect plugin for HS3.

Closed Thread
 
Thread Tools Display Modes
  #1  
Old October 8th, 2013, 08:26 PM
spud's Avatar
spud spud is offline
Moderator
 
Join Date: Jan 2012
Location: Canada
Posts: 4,641
Kinect Plug-in - User Guide

Installation / Configuration


System Requirements:

This plugin is for HS3 or HS3PRO.
It supports both the Kinect for Windows sensor and the Kinect for Xbox 360 sensor. Kinect v2 (i.e. Kinect for XBox One) is not yet supported.
The Kinect needs Windows 7 or later

Prerequisites:
You need to have the Kinect drivers installed, if you don't have them installed yet, you can download them from the folowing links:
Kinect for Windows drivers (Runtime)
Kinect for Xbox 360 drivers (SDK)

for Speech Recognition features you can install additional language packs to enable speech recognition for different languages or accents:
http://www.microsoft.com/en-US/downl....aspx?id=34809

Plugin Installation:
Download and install from the updater, or download package from the Beta Testing thread and follow instructions to install.

Last edited by spud; August 16th, 2015 at 07:52 PM.
  #2  
Old October 9th, 2013, 08:54 AM
spud's Avatar
spud spud is offline
Moderator
 
Join Date: Jan 2012
Location: Canada
Posts: 4,641
FEATURES

Voice Recognition

Go to PLUG-INS > Kinect > Config, and click on the Voice Recognition tab
  • set the Speech Recognizer to the language you want to use, you can install some additional languages as explained above in the prerequisites section
  • if you get too much false positive try to increase the Confidence Threshold, if the system miss too much commands try to decrease it. It must be a value between 0 and 1.
  • the Attention Phrase is the phrase that will prefix all your commands (except for sub grammars see below): "Kinect, turn the lights on". You can set this to anything you want.
Name:  vr_settings.jpg
Views: 3004
Size:  57.2 KB

EVENT VOICE COMMANDS

You can use the Kinect Trigger "A Phrase Was Recognized" to trigger any event using a voice command. Different variations of a same command can be configured by using a special syntax (see syntax guide for a full description) For example the phrase diagram on the screenshot below will recognize 4 different phrases: "play some music", "play music", "put some music" and "put music". Do not forget to say your Attention Phrase before each command. For example if your command is "Play some music" and your Attention Phrase is "Kinect", you need to say "Kinect, play some music"

Name:  vrevent2.jpg
Views: 3943
Size:  43.6 KB

DEVICE VOICE COMMANDS

You can also add voice command to any HS device without using any event or script.

To do so, from the "Device management" page, click on the device you want to control then click on the Kinect tab. Tick the "Voice Command Enabled" checkbox. Then for each control enter a custom command of your choice. The only requirement is for controls that are ranges, for those type of controls the command need to include the "(value)" string. This string will be replaced by all possible values by the plugin.

Name:  device_cmd2.jpg
Views: 2995
Size:  48.5 KB

If you don't want to use any voice command for one control, simply set the command to an empty string. This is good practice to do so for all the controls you don't use in order to keep the grammar as small as possible.

You need to replace any number or special character with actual words, for example "Zone four" instead of "Zone 4", "degrees" instead of "°C", etc..

Known problems:
(1) There is currently a bug with device configuration and multi instance plugins. https://www.homeseer.com/bugzilla//show_bug.cgi?id=1673
So, if you use more than one kinect instance, this feature will only work with the first one (usually the instance without a name)

(2) For range controls the plugin replaces the (value) string with all possible values in words, for example something like <one|two|three|...|hundred>, so right now only english is supported.

SUB GRAMMARS

Sub grammars are grammars (i.e set of voice commands) that can be loaded temporarily. This allow to create context specific commands without having to say the attention phrase before each command. See this thread for more details

Last edited by spud; August 7th, 2014 at 01:39 PM.
  #3  
Old October 9th, 2013, 08:54 AM
spud's Avatar
spud spud is offline
Moderator
 
Join Date: Jan 2012
Location: Canada
Posts: 4,641
FEATURES

People Detector

The Kinect is capable of detecting people (up to 6) and tell where they are. From the config page you can define zones and the plugin will automatically create a corresponding device which will report how many people have been detected in this zone. An overall zone is a global zone which detect people no matter where they are. Zones can overlap or not.

Name:  zonessettings.jpg
Views: 4807
Size:  38.0 KB

Name:  zonedevices.jpg
Views: 4780
Size:  46.9 KB

Posture And Gesture Recognition

The plugin is capable of detecting some basic gestures and postures that can be used as triggers for events.

Name:  posture.jpg
Views: 3002
Size:  60.2 KB

A posture is a static position that needs to be maintained for 1 or 2 seconds to be detected. It can't detect multiple postures at the same time.
Current list of Postures are:
  • HandsJoined
  • LeftHandOverHead
  • RightHandOverHead
  • LeftHello (= Left hand near head)
  • RightHello (= Right hand near head)

Current list of Gestures are:
  • RightHandSwipeLeft (= Right hand moving from right to left)
  • RightHandSwipeRight (= Right hand moving from left to right)
  • RightHandSwipeDown(= Right hand moving top down)
  • RightHandSwipeUp (= Right hand moving bottom up)
  • LeftHandSwipeLeft (= Left hand moving from right to left)
  • LeftHandSwipeRight (= Left hand moving from left to right)
  • LeftHandSwipeDown(= Left hand moving top down)
  • LeftHandSwipeUp (= Left hand moving bottom up)

IP CAMERA

To use the kinect as an IP Camera, go to the config page, Camera tab and tick the checkbox called "Use Kinect Camera as NetCam"
This will start a snapshot server on the machine where the plugin is running, then you can use the snapshot url (http://<plugin_ip>:8078/kinectsnapshot/) in a browser, in HSTouch, or any other software that takes a snapshot url as input.

Warning you need to run the plugin as Administrator for this to work.

Name:  netcam.jpg
Views: 2257
Size:  38.5 KB

You can use the same snapshot url with the tilt=up or tilt=down parameter to use the motor to move the kinect sensor up or down
For example in HSTouch configuration set the "Pan Up Start" and "Pan Down Start" as in the screenshot below. It will allow you to tilt the kinect sensor up or down when you tap the up or down arrows from the HSTouch interface.
Name:  tilturl.jpg
Views: 2266
Size:  64.7 KB

The step parameter is optional and let you specify the number of degrees you want the sensor to move for one tap.

Last edited by spud; March 3rd, 2015 at 04:23 PM.
  #4  
Old December 21st, 2013, 12:10 PM
spud's Avatar
spud spud is offline
Moderator
 
Join Date: Jan 2012
Location: Canada
Posts: 4,641
Event Actions

Event Actions

Here is the current list of kinect actions that can be done from events:
  • "Load Grammar"
  • "Unload Grammar"
  • "Rebuild Grammars"
  • "Enable Voice Recognition"
  • "Disable Voice Recognition"
  • "Enable Visual Recognition"
  • "Disable Visual Recognition"
  • "Enable Camera"
  • "Disable Camera"
  • "Take Picture"
  • "Tilt Sensor Down"
  • "Tilt Sensor Up"

and here is an example of how to take a picture using the kinect camera:

Name:  take_picture_event.jpg
Views: 2996
Size:  40.9 KB

Last edited by spud; May 25th, 2015 at 05:27 PM.
  #5  
Old December 21st, 2013, 12:10 PM
spud's Avatar
spud spud is offline
Moderator
 
Join Date: Jan 2012
Location: Canada
Posts: 4,641
Multiple instances and Remote instances

Multiple Instances

You can control multiple kinect by starting new instances of the plugin. If the kinect is plugged to a remote computer, see the Remote instance section below, if you have multiple kinects plugged into your HS machine, start a new instance of the kinect plugin by clicking the + button from the PLUG-INS > Manage page.

Name:  multiple_instances.jpg
Views: 3024
Size:  45.3 KB

Enter a name for your instance, submit, and then enable it.
This will create a new Config link in the PLUG-INS > Kinect menu, allowing you to choose which sensor this instance is supposed to control and to change the settings for this instance.

Name:  multiple_instances2.jpg
Views: 3059
Size:  84.9 KB

Remote Instances

To run the plugin remotely, you need first to install it on your HS machine, even if you don't have a Kinect connected to your HS machine you need to have it installed there (but you don't have to enable it) so that HS doesn't complain about licensing issue when you start your remote instances.
Then
  • copy HSPI_KINECT.exe, bin/Kinect/Kinect.Toolbox.dll to a directory on your remote computer
  • copy HomeSeerAPI.dll, HSCF.dll and Scheduler.dll from your HS3 directory to the same remote directory
  • open a command prompt on your remote computer
  • cd to the directory, then type HSPI_KINECT.exe server=<ip_of_your_hs_machine> instance=<instance_name> autoreconnect=<seconds_to_wait> console=<false|true>
where <instance_name> can be anything you want but only alpha numeric characters.

autoreconnect is an optional parameter which makes the plugin automatically reconnect when the connection with HS is lost. It can be useful for example when you want to restart HS without having to restart all the remote plugins manually.
<seconds_to_wait> is the number of seconds to wait before the plugin tries to reconnect.

console is an optional parameter that hides the console window when set to false

Last edited by spud; January 28th, 2015 at 12:10 PM.
Closed Thread

Bookmarks

Tags
configuration, features, kinect, user guide

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Kodi/XBMC Plug-in - User Guide spud Kodi/XBMC Plug-In (3P) 3 April 15th, 2015 03:37 PM
RFXCOM plug-in User & Programmers Guide b_weijenberg RFXCOM Plug-In (3P) 1 February 9th, 2014 07:13 AM
iTunes Speakers plug-in - User Guide spud iTunes Speakers 7 November 6th, 2013 11:21 AM
DirecTV Plug-in - User Guide spud DirecTV for HS3 (3P) 1 October 1st, 2013 08:09 AM
BC4 Plug-In User's Guide BitWise_Mark Bitwise Controls BC4 2 October 16th, 2010 06:23 PM


All times are GMT -4. The time now is 03:41 PM.


Copyright HomeSeer Technologies, LLC