Log File full of conversion errors - Encode payload template doesn't seem to work

[jrhubott]

Log is filled with the following error and other similar ones. I'm trying to log all of the data collected by the AmbientWeather plugin to MQTT and store it in a InfluxDB.

HSEvent Line 0 Conversion from string "119141 lux" to type 'Double' is not valid

The following is the values stored in the device. I have tried all of the template values and still get this error. I would have thought that $$
Reference ID 4867
Status 119141 = Unknown
Value 119141 = "119141 Lux"
String 119141 w/m2



This is another interesting thing. When I set the template to $$STATUS: this is what gets stored in the history. I don't understand why the publish say's 119141 but the subscribed say's "119141 Lux". The "119141 Lux" is what is being sent though. I just want the 119141.

2155	P	2018-09-07 14:37:01	STAT/salix/AmbientWeather%20luminance/AmbientWeather/Backyard/Luminance/AmbientWeather/4867/	119141
2156	S	2018-09-07 14:37:01	STAT/salix/AmbientWeather%20luminance/AmbientWeather/Backyard/Luminance/AmbientWeather/4867/	119141 Lux

Any help would be appreciated.

Thanks,
Jason

[Michael McSharry]

I get back Sunday PM and will look into this. I do not have environment at this time.

[Michael McSharry]

HSEvent conversion errors are from the parameters passed from HS which are defined as integer in the SDK. What I did with version 3.4.8.0 is protect from this and record in the debug log if it happens. It also means the event data will not be processed by the plugin if the string rather than the reference is passed. I will submit this version to the updater. You can also obtain the plugin exe from http://mcsSprinklers.com/mcsMQTT_3_4_8_0.zip and do a manual replace. After installed, enable debug from the General tab. The resultant .txt file will be in ...\bin\Data\mcsMQTT.

What I infer about device 4867 is that it is a number and "Lux" is the suffix specified for display. The DeviceString was also populated using w/m2 as the formatting suffix. Since DeviceString exists there is no DeviceStatus. I'm not certain what HS returns to mcsMQTT for DeviceStatus of 4867, but it looks like it is the DeviceValue without any formatting based upon what you show as the Publish topic payload in row 2155.

Some node is sending the payload reflected in row 2156 and that node includes the Lux suffix. Who is publishing this topic?

A more general question is what you are trying to accomplish with mcsMQTT? Is your objective to have it publish MQTT messages that contain AmbientWeather items and then you have another MQTT client that writes to your Influx DB?

[jrhubott]

The only publisher is mcsMQTT. That is why I don't understand why the publisher message is the number without the lux but the subscriber sees it with the Lux. I just want to send it out without the lux part.

You are correct I reviewed the event in node-red and store it to a influxdb. Only the Amibient weather plugin causes this issue. It happens for all of the devices.

Jason

[Michael McSharry]

On the General Tab for Inbound there is an Echo option. In your case you do not want to process the echo of transmitted messages. It should not make any difference for the issue, but no need to burn CPU cycles. This should no longer store subscribed entries for the ambient devices in the mcsMQTT database. If you still get them then I should be able to figure out why from the debug file.

Since you only want numbers and not text published then assure that you are not using the publish template of $$STRING: Your publish template should be blank as the default is to publish the DeviceValue and using the default will be more efficient execution.

Looking at the code again I see a more likely place where the conversion error is happening. Let me update to 3.4.8.1 before you update the plugin. Post your debug file when you have updated the version.

[Michael McSharry]

Updated to http://mcsSprinklers.com/mcsMQTT_3_4_8_1.zip and submitted to updater. I suspect this will correct the issue.

[jrhubott]

It is still occurring. The following are screenshots of the error and how I have things configured.

[simplextech]

It is still occurring. The following are screenshots of the error and how I have things configured.

jrhubott, out of curiosity what are you looking to achieve?

[Michael McSharry]

What I need to work the issue is the mcsMQTT debug file that is produced while running 3.4.8.1.

jrhubott, out of curiosity what are you looking to achieve?

What was stated above is the desire to log various device values to his Influx database. Node-red is the integration tool and MQTT is a protocol that Node-red understands. All are working except the Ambient plugin's as there is something unexpected in the HSEvent callback.

[jrhubott]

Will get it for you tomorrow. Thanks for digging into this.

The MQTT publishing is correct now. Node-Red is receiving just the raw data but these errors are still appearing in the logs. This only happens for the AmbientWeather plugin. A send a lot of other data to node red using your MQTT plugin without any issues. Love your plugin! Also love the AmbientWeather plugin as it gets local data and didn't need the cloud.

As for the use I store all of my weather and energy use data in influx for graphing with Grafana. MQTT => Node-Red => InfluxDB => Grafana.

Jason

[simplextech]

Will get it for you tomorrow. Thanks for digging into this.


As for the use I store all of my weather and energy use data in influx for graphing with Grafana. MQTT => Node-Red => InfluxDB => Grafana.

Jason

That is what I was wondering. Thanks. Thinking if there's something I can do with the AmbientWeather plugin to make this type of use easier?

[jrhubott]

Figured out part of the problem. The link above is to the V3.4.8.0. I changed it and downloaded 3.4.8.1 and the error message is now gone. I still get the unit text added at the end though for all of the ambientweather devices.



When I look at your debug log though I see it send 2 messages one with and one without if I'm reading it correctly. I have attached it maybe you can figure it out.

Thanks,
Jason

[jrhubott]

Now I really messed something up. The plugin only runs for about 30 seconds and then stops responding. HS restarts it and it runs for 30 seconds or so and crashes. This repeats over and over.

It is also send double message for everything when this occurs.

This all happened after I turned of the STAT and INFO prefixes.

Jason

[jrhubott]

Got the STAT and INFO turned back on and that at least lets the plugin work. If I turn them off it goes back to crashing every 30 seconds.

I also noticed this issue where it is sending out payloads with a topic of STAT/


I tracked this to the following device as it was the only device to have a value of 617

I checked the settings and this device should not be reporting anything.


I have several other nodes also reporting just STAT/ for some reason. It is hard to track down.

[Michael McSharry]

A Plugin can register for callbacks from HS when a DeviceValue or a DeviceString property of a device changes. The debug shows that both string and value callbacks occur for the luminance device. The Ambient plugin is changing both properties of this device.

mcsMQTT only registers a DeviceString callback when some device's association publish template contains $$STRING: This is why I previously suggested looking for an association with a template that contained this substitution. It does not need to be device 4867, but any device with this association's template.

There are four ways that I can think of to deal with this. I will leave it up to discussion as I am open to an mcsMQTT change if that will work best.
1. If publishing on DeviceString change is not needed for any device then remove $$STRING: from all publish templates. The attached debug contains output that identifies the devices with template that contains $$STRING: to make the search easier. .. jrhubott action
2. HS is oriented to not using DeviceString but using the VSP and prefix/suffix properties to render textually formatted numbers. In the Luminance device the "Lux" could be put in the format suffix and then the DeviceString would remain blank. HS shows DeviceStatus and it would render just as DeviceString does now. If DeviceString is not null then DeviceString becomes the DeviceStatus when rendering. ... simplexttechnology action
3. Add a event timeout (e.g. 1 second) in mcsMQTT and inhibit publishing any change callback that occurred within this timeout. I do this in mcsXap plugin that has similiar behavior. ... my action
4. In node-red add similar timeout logic to suppress topics received within a timeout window. Logic may also be needed to extract numeric portion of payload if that does not already exist. mcsMQTT does regular expression on subscriptions, but no provisions exist for them while publishing. ... jrhubott action

There may be others. What seems to be the best solution for you?