mcsMQTT Plugin

[petez69]

Michael

I'm getting these messages in my log every time the WeMOS sends the update.

Dec-27 09:20:42 mcsMQTT ParseJSON tele/sonoff-wemos/SENSORS18B21={"Time":"2017-12-27T00:50:43","Switch1":"ON","DS18B21":{"Temperature":39.6,}, "TempUnit":"C"} Line 0 Length cannot be less than zero. Parameter name: length

Any way to stop these messages or is there a real error ?

Thanks..Pete

[petez69]

Michael

Did a windows 7 update today, possibly co-incidental and its caused a bunch of issues with the mcsMQTT database, why I dont know.

I found that even through devices that were set to "accept" in the plugin, after restarting HS they were not updating. I had an SQLite viewer and saw for some reason the device had the accept entry in the mcs mqtt message table turned off after restart, before the restart the flag was turned on

I attempted to totally remove the plugin and was hoping to build a new mcsMQTT database, clearly I didnt find everything as after dropping the 5 files back into the correct folders, the log was full of messages about a table missing when messages were being received.

I ended up putting the old db back however I removed all of the entries in the table except for the original HS devices so all of the mqtt generated entries i deleted. I've then "accepted" the messages again and its created the devices. I've now restarted a number of times and it appears to now "stick", ie: tickbox against the mqtt device in teh plugin for "A" (accept).

I'd be keen to understand how to totally remove the plugin and start with a fresh database, as mentioned above I removed the mcsMQTT database from the data directory, the plugin on startup created a new DB but of zero size. Also I dont know how HS keeps track of the device numbers however after removing the DB entries by hand, I've lost those device numbers. Is there a way to reclaim them ?

All a learning experience....something I didnt really need :-)

[Michael McSharry]

It looks to me as if the payload is ill-formed with a missing trailing } at the end. Please confirm. Some protections exist in the parser, but I could have missed this one.

It would be helpful if more specific info is available about why the new database was not working. The database contains all info about received topics so if it gets messed up the info is lost. In xapmcsDatabase I put a lock on to prevent unintentional changes being recorded. I could do the same here if this is really a problem.

I use HS3 to remember info about transmit info. It is stored in plugin extra data and not visible like the info in the receive database. I could do the same for receive, but would prefer not to hide the info and it could be a burden for HS3 since plugin extra data probably was not designed for a large potential volume of data.

I will be away til Jan 2 so any changes I make I will not be able to test this week.

[petez69]

Hi Michael

The mqtt message from the WeMOS via the tasmota console.

09:04:54 MQT: tele/sonoff-wemos/SENSOR = {"Time":"2017-12-27T09:04:54","Switch1":"ON","DS18B21":{"Temperature":42.0,}, "TempUnit":"C"}

On the broker I am subscribed to the topic:

tele/sonoff-wemos/#

And this is the message I see from the WeMOS device on the broker.

{"Time":"2017-12-27T09:06:34","Switch1":"ON","DS18B21":{"Temperature":42.6,}, "TempUnit":"C"}

It does look like all delimters are there ?

Pete

[petez69]

Hi Michael, ok the story on the Windows upgrade.

There were 3 things I did today in order

1) WIndows upgrade. I doubt this would have done anything as sqllite is a seperate app so I'm discounting this.

2) I tried to get Jon00's charting to work again. I had it running a year ago and simply "rem'd" it out of the startup.vb script. All that needed doing was to modify his ini file and give him the device ID of the ds18b20 temp sensor. I dont believe I copied any files out of his zip file, I simply ran the app. I then tried to get updates from the temp sensor and then noted I had no updates. Simply I cannot validate if this was straight after the update or I ran Jon00s binary.

3) I removed the temp sensor manually from the device management page and added it again via the plugin. I did this because there was no "A" against the original temp sensor hence why I removed it manually. The temp sensor sprang to life and I was getting updates. I added the 2 timestamps, one from the WeMOS and the other Sonoff. All were updating. I did a reboot and then found the devices did not have the "A" ticked. This is when it all went circular and I decided to remove the plugin, including your DB and thats when it went south with no database found.

I put the database back and used a DB viewer to look at the message table. I could see that it had "accept" against the items I wanted. I rebooted and went back to the DB viewer and then saw the "accept" was missing.

I then removed ALL mqtt entries and left the original HS devices in the DB. After doing this I now have 3 devices that are staying "accepted" after a reboot.

If there is any specific debug I can provide please ask, I'm not a DB guy but I can get my head around it.

I've just added the switch within the light sonoff and it its toggling the light fine and reflecting correct status in HS (works fine after reboot)

Hope that makes sense as to what transpired.

Pete

[Michael McSharry]

I have not got into a detailed analysis, but I suspect "{"Temperature":42.0,}" is where the problem exists. The trailing comma indicated another field to follow, but none present. Whatever it may be, I will update the code to accommodate.

The database is updated on a device-by-device basis from changes made on the mcsMQTT setup page. There is no save when the plugin exits so anything that has been recorded should remain even if it was a power cycle rather than normal shutdown. Loosing all Accept does seem odd.

The other thing I will do is recognize there is redundancy with both Ref and Accept. If there is a positive Ref, then Accept should be forced to true.

[petez69]

I have not got into a detailed analysis, but I suspect "{"Temperature":42.0,}" is where the problem exists. The trailing comma indicated another field to follow, but none present. Whatever it may be, I will update the code to accommodate.

The database is updated on a device-by-device basis from changes made on the mcsMQTT setup page. There is no save when the plugin exits so anything that has been recorded should remain even if it was a power cycle rather than normal shutdown. Loosing all Accept does seem odd.

The other thing I will do is recognize there is redundancy with both Ref and Accept. If there is a positive Ref, then Accept should be forced to true.

Michael

Thanks for the feedback. Yes the 42, would suggest another field is to come. I suspect the author has coded this to accept devices that offer temp/humidity. I was going to try the AM231 sensor (temp/humid) on the WeMOS on the weekend to see what the mqtt message looks like. Also this is where I caused a problem for myself, I uncommented a define in the tasmota code that allows for multiple DS18b20 sensors, with the one sensor it removed the "," so the tmp value shown by the plugin was 42} which is clearly an issue.

As for the accept, that was definitely weird as I saw the correct status in the mcsMQTT database table.

Hope you are having a good break :-)

Pete

[Michael McSharry]

I found the parsing error. Fixed in 3.0.3.0 which is attached here.

I also may be able to explain the removal of Accept check. During initialization the plugin confirms that a device exists with the Ref property that has been associated with the topic. If not it forces the checkbox false and sets the Ref property of this topic to -1. I suspect during your cleanup a device was deleted so a device with the HS3-assigned Ref property no longer existed.

What I did do was ignore the "A"ccept value in the database and based Accept on the Ref property and existence of a device with that Ref value. I elected this more conservative approach rather than creating a new device to avoid possible user confusion with new devices being created without their explicit action.

What I also did was made a backup of the database on each plugin start. This was one can go back to a prior configuration by renaming the database from the backup name.

I also changed to Device update to only occur from MQTT Topic and not based upon a device action that commands a MQTT Topic to be commanded.

[petez69]

Thanks Michael

Thanks for the update, the parsing error has now stopped. I'm waiting for a few more sonoff and WeMOS devices and then I'll start adding things again. For now I'll just try adding the AM231 temp/humidity sensor and see the structure of the mqtt message....

Again thankyou for all your help....its working nicely...

Pete

[petez69]

Michael

Added a NodeMCU (ESP8266) board with an AM2301 temp/humidity sensor....Looks like the parsing is working fine.....

One pic is from the NodeMCU console mqtt messages and the other is the homeseer devices.

Cheers..Pete

[qwiksilver96]

Michael,

FYI, in ver 3.0.4.0, the IP address of the server seems to be hardcoded to 192.168.0.194

Best,
Frank

[petez69]

Frank, dont you go into the mcsmqtt plugin and put in the mqtt broker ?

Maybe I am misunderstanding your question ?

Pete

[qwiksilver96]

The IP I referred to is for the plugin connection to the HS3 instance upon plugin startup. Once that connection is live, then the plugin connects to the MQTT Broker.

Best,
Frank

Frank, dont you go into the mcsmqtt plugin and put in the mqtt broker ?

Maybe I am misunderstanding your question ?

Pete

[Michael McSharry]

FYI, in ver 3.0.4.0, the IP address of the server seems to be hardcoded to 192.168.0.194

My mistake. Fixed in 3.0.4.1

[qwiksilver96]

My mistake. Fixed in 3.0.4.1

All good. Thank you!


Incidentally, looks like there's an issue if there are devices with single quotes in their name. Please see the attached image for details.

Best,
Frank