Kinect Plug-in - User Guide

[spud]

Multiple instances and Remote instances

Multiple Instances

You can control multiple kinect by starting new instances of the plugin. If the kinect is plugged to a remote computer, see the Remote instance section below, if you have multiple kinects plugged into your HS machine, start a new instance of the kinect plugin by clicking the + button from the PLUG-INS > Manage page.



Enter a name for your instance, submit, and then enable it.
This will create a new Config link in the PLUG-INS > Kinect menu, allowing you to choose which sensor this instance is supposed to control and to change the settings for this instance.



Remote Instances

To run the plugin remotely, you need first to install it on your HS machine, even if you don't have a Kinect connected to your HS machine you need to have it installed there (but you don't have to enable it) so that HS doesn't complain about licensing issue when you start your remote instances.
Then

• copy HSPI_KINECT.exe, bin/Kinect/Kinect.Toolbox.dll to a directory on your remote computer
• copy HomeSeerAPI.dll, HSCF.dll and Scheduler.dll from your HS3 directory to the same remote directory
• open a command prompt on your remote computer
• cd to the directory, then type HSPI_KINECT.exe server=<ip_of_your_hs_machine> instance=<instance_name> autoreconnect=<seconds_to_wait> console=<false|true>

where <instance_name> can be anything you want but only alpha numeric characters.

autoreconnect is an optional parameter which makes the plugin automatically reconnect when the connection with HS is lost. It can be useful for example when you want to restart HS without having to restart all the remote plugins manually.
<seconds_to_wait> is the number of seconds to wait before the plugin tries to reconnect.

console is an optional parameter that hides the console window when set to false

[spud]

Event Actions

Event Actions

Here is the current list of kinect actions that can be done from events:

• "Load Grammar"
• "Unload Grammar"
• "Rebuild Grammars"
• "Enable Voice Recognition"
• "Disable Voice Recognition"
• "Enable Visual Recognition"
• "Disable Visual Recognition"
• "Enable Camera"
• "Disable Camera"
• "Take Picture"
• "Tilt Sensor Down"
• "Tilt Sensor Up"

and here is an example of how to take a picture using the kinect camera:

[spud]

FEATURES

People Detector

The Kinect is capable of detecting people (up to 6) and tell where they are. From the config page you can define zones and the plugin will automatically create a corresponding device which will report how many people have been detected in this zone. An overall zone is a global zone which detect people no matter where they are. Zones can overlap or not.





Posture And Gesture Recognition

The plugin is capable of detecting some basic gestures and postures that can be used as triggers for events.



A posture is a static position that needs to be maintained for 1 or 2 seconds to be detected. It can't detect multiple postures at the same time.
Current list of Postures are:

• HandsJoined
• LeftHandOverHead
• RightHandOverHead
• LeftHello (= Left hand near head)
• RightHello (= Right hand near head)

Current list of Gestures are:

• RightHandSwipeLeft (= Right hand moving from right to left)
• RightHandSwipeRight (= Right hand moving from left to right)
• RightHandSwipeDown(= Right hand moving top down)
• RightHandSwipeUp (= Right hand moving bottom up)
• LeftHandSwipeLeft (= Left hand moving from right to left)
• LeftHandSwipeRight (= Left hand moving from left to right)
• LeftHandSwipeDown(= Left hand moving top down)
• LeftHandSwipeUp (= Left hand moving bottom up)

IP CAMERA

To use the kinect as an IP Camera, go to the config page, Camera tab and tick the checkbox called "Use Kinect Camera as NetCam"
This will start a snapshot server on the machine where the plugin is running, then you can use the snapshot url (http://<plugin_ip>:8078/kinectsnapshot/) in a browser, in HSTouch, or any other software that takes a snapshot url as input.

Warning you need to run the plugin as Administrator for this to work.



You can use the same snapshot url with the tilt=up or tilt=down parameter to use the motor to move the kinect sensor up or down
For example in HSTouch configuration set the "Pan Up Start" and "Pan Down Start" as in the screenshot below. It will allow you to tilt the kinect sensor up or down when you tap the up or down arrows from the HSTouch interface.


The step parameter is optional and let you specify the number of degrees you want the sensor to move for one tap.

[spud]

FEATURES

Voice Recognition

Go to PLUG-INS > Kinect > Config, and click on the Voice Recognition tab

• set the Speech Recognizer to the language you want to use, you can install some additional languages as explained above in the prerequisites section
• if you get too much false positive try to increase the Confidence Threshold, if the system miss too much commands try to decrease it. It must be a value between 0 and 1.
• the Attention Phrase is the phrase that will prefix all your commands (except for sub grammars see below): "Kinect, turn the lights on". You can set this to anything you want.

EVENT VOICE COMMANDS

You can use the Kinect Trigger "A Phrase Was Recognized" to trigger any event using a voice command. Different variations of a same command can be configured by using a special syntax (see syntax guide for a full description) For example the phrase diagram on the screenshot below will recognize 4 different phrases: "play some music", "play music", "put some music" and "put music". Do not forget to say your Attention Phrase before each command. For example if your command is "Play some music" and your Attention Phrase is "Kinect", you need to say "Kinect, play some music"



DEVICE VOICE COMMANDS

You can also add voice command to any HS device without using any event or script.

To do so, from the "Device management" page, click on the device you want to control then click on the Kinect tab. Tick the "Voice Command Enabled" checkbox. Then for each control enter a custom command of your choice. The only requirement is for controls that are ranges, for those type of controls the command need to include the "(value)" string. This string will be replaced by all possible values by the plugin.

http://www.homeseer.com/bugzilla//show_bug.cgi?id=1673
So, if you use more than one kinect instance, this feature will only work with the first one (usually the instance without a name)

(2) For range controls the plugin replaces the (value) string with all possible values in words, for example something like <one|two|three|...|hundred>, so right now only english is supported.

SUB GRAMMARS

Sub grammars are grammars (i.e set of voice commands) that can be loaded temporarily. This allow to create context specific commands without having to say the attention phrase before each command. See this thread for more details

[spud]

Installation / Configuration


System Requirements:
This plugin is for HS3 or HS3PRO.
It supports both the Kinect for Windows sensor and the Kinect for Xbox 360 sensor. Kinect v2 (i.e. Kinect for XBox One) is not yet supported.
The Kinect needs Windows 7 or later

Prerequisites:
You need to have the Kinect drivers installed, if you don't have them installed yet, you can download them from the folowing links:
Kinect for Windows drivers (Runtime)
Kinect for Xbox 360 drivers (SDK)

for Speech Recognition features you can install additional language packs to enable speech recognition for different languages or accents:
http://www.microsoft.com/en-US/downl....aspx?id=34809

Plugin Installation:
Download and install from the updater, or download package from the Beta Testing thread and follow instructions to install.