Howto get HSdevice value to mscMQTT

[Cor]

The replace function does not work as expected, I had the comma replaced by period in this way:

Buiten/afdak/Temp_buiten=<<Replace($$DVR:(859):,",","\.")>>

The callback message was first:

Message arrived [Buiten/afdak/Temp_buiten] 16,2012

and after the replace:

Message arrived [Buiten/afdak/Temp_buiten]    1012(16,2

This is a mix of the temperature and another callback from the barometric pressure.

Any idea?

[Michael McSharry]

Why do you use \. rather than just . ? I suggest removing the \ as you do not want it as part of the replacement text.

[Cor]

I did test without the backslash but the I get a callback of a different sensor. I will explain hereafter.

Here are a few callbacks of the 5 sensors I use for this paticular case with a ESP8266. The callbacks with the topic "Sensor/Buiten/*" stem from ESP8266 nodes that communicate via the mcsMMQT plugin, these are just fine. The topics with "Buiten/afdak/" stem from the event actions.

Message arrived [Buiten/afdak/Temp_buiten] 14,8016
Message arrived [Buiten/afdak/RelatieveVochtigheid] 69,8016
Message arrived [Sensor/Buiten/buitentemp] 13.76
Message arrived [Sensor/Buiten/licht] 707
Message arrived [Sensor/Buiten/baro] 1016

You can see the comma in 14,8016 that I want to change in a period. The comma in 69,8016 doesnt matter because I only use the integer part.
Now, if I change the event action to:

Buiten/afdak/Temp_buiten=<<Replace($$DVR:(859):,",",".")>>

then the callback messages on the ESP8266 become:

Message arrived [Buiten/afdak/Temp_buiten] 1016
Message arrived [Buiten/afdak/RelatieveVochtigheid] 69 1016
Message arrived [Sensor/Buiten/buitentemp] 13.76
Message arrived [Sensor/Buiten/licht] 932
Message arrived [Sensor/Buiten/baro] 1016

So you can see that Temp_buiten get the value of Sensor/Buiten/baro, And the relative humudity gets two values now, the first is the correct one, the second is the barometric pressure too. Weird isnt it?

[Cor]

Sorry Michael, I made a mistake here with printing the payload 🤔
The payload was written to an array and I when printed there were still previous values stored.

If I now print the payload correct then the callback of "Buiten/afdak/Temp_buiten" yields zero characters:

Message arrived [Buiten/afdak/Temp_buiten]
Message arrived [Buiten/afdak/RelatieveVochtigheid] 62

The second callback has no replace so these are passed through.

[Michael McSharry]

The debug file contains a line starting with "ApplyExpression" followed by the input=output strings. This allows you to assess the actual input string and output string of the Replace function. After this one there is also "Handle Action Topic=Payload" topic=payload which occurs just after the message is published. The debug is enabled from the top of the General tab and the .txt file is in folder \data\mcsMQTT.

To confirm the function works with periods and decimals I did the following test with the debugger which showed the code, executing in my environment, does the replacement either way of these characters. I had previously tested end to end with zero and one values as part of event action and saw the published message with the correct replacements. The double quotes below are needed in the debugger, but are not needed from the text box in mcsMQTT action.

?ApplyExpression("Replace(""1,2"","","",""."")")
"1.2" {String}
String: "1.2"
?ApplyExpression("Replace(""1.2"",""."","","")")
"1,2" {String}
String: "1,2"

If this does not help then I can add more detailed debug to see where in the process the failure is occurring.

[Cor]

If I look in the logfile then, after I run the event, I can see:

10-6-2020 10:26:23 144188350 | Handle Action Entry
10-6-2020 10:26:23 144188391 | Handle Action Topic Buiten/afdak/Temp_buiten
10-6-2020 10:26:23 144188392 | Handle Action Topic=Payload Buiten/afdak/Temp_buiten=

So no ApplyExpression text.

If I remove the Replace function then I see:

10-6-2020 10:30:42 144447461 | Handle Action Entry
10-6-2020 10:30:42 144447484 | Handle Action Topic Buiten/afdak/Temp_buiten
10-6-2020 10:30:42 144447485 | Handle Action Topic=Payload Buiten/afdak/Temp_buiten=15,1

I also see a lot of lines from devices that are non-plugin-devices. Is it normal that I can see all these devices? One is from a timer (device 258), this generates a lot of output:

10-6-2020 10:19:45 143789689 | HSEvent VALUE_CHANGE| 1024| | 133332| 133331| 258
10-6-2020 10:19:46 143790894 | HSEvent VALUE_CHANGE| 1024| | 133333| 133332| 258
10-6-2020 10:19:47 143791698 | HSEvent VALUE_CHANGE| 1024| | 133334| 133333| 258
10-6-2020 10:19:48 143792902 | HSEvent VALUE_CHANGE| 1024| | 133335| 133334| 258
10-6-2020 10:19:49 143793710 | HSEvent VALUE_CHANGE| 1024| | 133336| 133335| 258
10-6-2020 10:19:50 143794912 | HSEvent VALUE_CHANGE| 1024| | 133337| 133336| 258
10-6-2020 10:19:51 143795716 | HSEvent VALUE_CHANGE| 1024| | 133338| 133337| 258
10-6-2020 10:19:52 143796929 | HSEvent VALUE_CHANGE| 1024| | 133339| 133338| 258

[Michael McSharry]

I added much detail in the HandleAction processing of the event action. It is attached.

Yes, the debug contains every change of a value of HS devices. This means it should also see a change in the value of device 859 and send the publish topic assigned to this device.

The HandleAction procedure with additional debug is below for your reference.

Public Function HandleActionX(ByVal TANumber As Integer, ByVal sMessage As String) As Boolean 'ActInfo As IPlugInAPI.strTrigActInfo) As Boolean Implements HomeSeerAPI.IPlugInAPI.HandleAction
Try
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Entry " & sMessage)
End If
Dim s As String = sMessage
If TANumber = 1 Then 'ActInfo.TANumber = 1 Then 'topic=payload
'$$date Replacement is the current date in long format, e.g.: April 1, 2006
'$$time Replacement is the current time in 12 hour format, e.g. 2:00 PM
'$$DVA:(address): Replacement is the VALUE of the device indicated by (address). For example, if the device at address R40 has a value of 100,then using $$DV:R40: in the text will result in 100 after the substitution.
'$$DVC:(code): Same as $$DVA but gets the device value using the device code.
'$$DVR:(ref): Same as $$DVA but gets the device value using the device reference number.
' $$DSA:(address): Replacement is the STATUS of the device indicated by (address). For example, if the device at address S39 has a status of "Disarmed", then using $$DSA:S39: in the text will result in "Disarmed" after the substitution.
' $$DSC:(code): Same as $$DSA but gets the status using a device code.
' $$DSR:(ref): Same as $$DSA but gets the status using a device reference number.
'$$DTA:(address): Replacement is the STRING of the device indicated by (address). For example, if the device at address S39 has a string of "Come listen to a story about a man named <b>Jed</b>", then using $$DTA:S39: in the text will result in "Come listen to a story about a man named Jed" after the substitution.
'$$DTC:(code): Same as $$DTA but gets the string using a device code.
'$$DTR:(ref): Same as $$DTA but gets the string using a device reference number.
Dim m As Integer
Dim k As Integer
Dim sIndex As String
Dim i As Integer = InStr(s, "=")
Dim iRef As Integer
Dim iStart As Integer = 1
Dim bSubstitutions As Boolean = True
If i > 0 Then
Dim sPayload As String = s.Substring(i)
Do While bSubstitutions
Do
k = InStr(iStart, sPayload, "$$")
If k = 0 Then
bSubstitutions = False
Exit Do
End If
iStart = k + 2
m = InStr(k + 7, sPayload, "):") + 1
If m > 1 Then
sIndex = sPayload.Substring(k + 6, m - k - 8)
Else
sIndex = ""
End If

Dim sKey As String
Dim iKeyLength As Integer = 5
If sPayload.Length > k + 5 Then
iKeyLength = 5
Else
iKeyLength = sPayload.Length - (k + 1)
End If
If iKeyLength > 3 Then 'sPayload.Length > k + 5 Then
sKey = sPayload.Substring(k + 1, iKeyLength).ToUpper()
'ElseIf sPayload.Length > k + 4 Then
'sKey = sPayload.Substring(k + 1, 4).ToUpper()
Else
Exit Do
End If
If sKey = "DATE:" Then
sPayload = sPayload.Substring(0, k - 1) & Now.ToShortDateString() & sPayload.Substring(k + 6)
ElseIf sKey = "TIME:" Then
sPayload = sPayload.Substring(0, k - 1) & Now.ToShortTimeString() & sPayload.Substring(k + 6)
Else
sKey = sKey.Substring(0, 4)
Select Case sKey
Case "DATE"
sPayload = sPayload.Substring(0, k - 1) & Now.ToString("D") & sPayload.Substring(k + 5)
Case "TIME"
sPayload = sPayload.Substring(0, k - 1) & Now.ToString("T") & sPayload.Substring(k + 5)
Case "DVC:", "DSC:", "DTC:"
If m = 1 Then Exit Do
iRef = hs.DeviceExistsCode(sIndex)
Case "DTA:", "DSA:", "DVA:"
If m = 1 Then Exit Do
iRef = hs.DeviceExistsAddress(sIndex, False)
Case "DTR:", "DSR:", "DVR:"
If m = 1 Then Exit Do
iRef = Val(sIndex)
Case Else
Exit Do
End Select
End If
Dim sProperty As String = sKey.Substring(1, 1)
If Not (sProperty = "A" OrElse sProperty = "I") Then
If hs.DeviceExistsRef(iRef) Then
If sProperty = "V" Then
Dim dv As Scheduler.Classes.DeviceClass = hs.GetDeviceByRef(iRef)
If dv IsNot Nothing Then
sIndex = dv.devValue(hs).ToString
Else
sIndex = ""
End If
ElseIf sProperty = "S" Then
Dim dv As Scheduler.Classes.DeviceClass = hs.GetDeviceByRef(iRef)
If dv IsNot Nothing Then
sIndex = hs.DeviceVSP_GetStatus(iRef, dv.devValue(hs), ePairStatusControl.Status)
Else
sIndex = ""
End If
ElseIf sProperty = "T" Then
sIndex = Replace(hs.DeviceString(iRef), q, "'") '8/5/2019 removed quotes from JSON payloads
Else
sIndex = "?"
End If
Else
sIndex = "?"
End If
sPayload = sPayload.Substring(0, k - 1) & sIndex & sPayload.Substring(m)
End If
Loop
Loop
Dim sTopic As String = s.Substring(0, i - 1)
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Topic|Payload " & sTopic & " | " & sPayload)
End If

'apply numeric expression '5/1/2020
Dim sPayloadLeft As String = ""
Dim sPayloadRight As String = sPayload
i = 1
Do
i = InStr(sPayloadRight, "<<")
If i = 0 Then
Exit Do
End If
Dim j As Integer = InStr(i + 2, sPayloadRight, ">>")
If j = 0 Then
Exit Do
End If
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Inline i,j " & i.ToString & " | " & j.ToString)
End If

Dim sExpression As String = sPayload.Substring(i + 1, j - i - 2)
Dim n As Integer = InStr(i, sExpression, "<<")
If n > 0 Then
'inner expression
sExpression = sExpression.Substring(n + 1)
i = n
End If
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Expression | n" & sExpression & " | " & n.ToString)
End If

Dim oValue As Object = ApplyExpression(sExpression)
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action oValue | n" & oValue.ToString)
End If

sPayloadLeft = sPayloadLeft & sPayloadRight.Substring(0, i - 1) & oValue.ToString
sPayloadRight = sPayloadRight.Substring(j + 1)
Loop
sPayload = sPayloadLeft & sPayloadRight
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Payload " & sPayload)
End If

'if a sign topic then remove the HTML
If sTopic <> "" Then
Dim arrSignTopic As String() = FindSignTopic()
If arrSignTopic IsNot Nothing Then
For Each sSignTopic As String In arrSignTopic
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Sign " & sTopic.Length.ToString & "=" & sTopic.Substring(0, sSignTopic.Length) & "=" & sSignTopic)
End If
If sTopic.Length > sSignTopic.Length AndAlso sTopic.Substring(0, sSignTopic.Length) = sSignTopic Then
sPayload = StripHTML(sPayload)
Exit For
End If
Next
End If
If sTopic <> "" AndAlso Not MQTTSendTopicList.Contains(sTopic) Then
MQTTSendTopicList.Add(sTopic)
End If
For iBroker As Integer = 0 To oMQTTClient.Length - 1
If oMQTTClient(iBroker) IsNot Nothing Then
oMQTTClient(iBroker).Publish(sTopic, Encoding.UTF8.GetBytes(sPayload), gDefaultQOS, gDefaultRetain)
'hsWritelog(PLUGIN_NAME, "Event Action Publishing " & sTopic & " with Payload of " & sPayload)
End If
Next
End If
If gDebugLog Then
hsWritelog(PLUGIN_DEBUG, "Handle Action Topic=Payload " & sTopic & "=" & sPayload)
End If
gPublishCount = gPublishCount + 1
gPublishCountToday = gPublishCountToday + 1
gLastPublishTopic = sTopic
gLastPublishPayload = sPayload
Dim strRefresh As New RefreshPair
With strRefresh
.Key = iRef.ToString
.LastDate = Now.ToString(DATETIMEFORMAT)
.Payload = sPayload
End With
AddToSendRefreshQueue(strRefresh)
Return True
End If
Else 'publist
Dim sFile As String = gDataPath & s & ".pub"
If File.Exists(sFile) Then
Dim sIn As String
Dim iRow As Integer = -4
Dim i As Integer
Dim arrPublistSubstitution(3) As String
Dim arrPublist() As String = Nothing
Try
For i = 0 To 3
arrPublistSubstitution(i) = ""
Next
Using r As StreamReader = New StreamReader(sFile, Encoding.UTF8)
sIn = r.ReadLine
Do While (Not String.IsNullOrEmpty(sIn))
If iRow < 0 Then
i = InStr(sIn, "=")
If i = 0 Then
i = 1
End If
arrPublistSubstitution(iRow + 4) = sIn.Substring(i)
Else
ReDim Preserve arrPublist(iRow)
arrPublist(iRow) = sIn
End If
iRow = iRow + 1
sIn = r.ReadLine
Loop
End Using
Dim sMessage2 As String = PublishPublist(arrPublistSubstitution, arrPublist)
If sMessage2 = "" Then
hsWritelog(PLUGIN_NAME, "Event Action Publishing Publist " & sFile)
Return True
Else
hsWritelog(PLUGIN_NAME, "HandleAction " & sMessage2)
Return False
End If

Catch ex As Exception
hsWritelogEx(PLUGIN_DEBUG, "BuildPublishRows " & sFile, ex.Message)
Return ex.Message & " while using " & sFile
End Try
End If
End If
'End If
Return False

Catch ex As Exception
hsWritelogEx(PLUGIN_NAME, "HandleActionX", ex.Message)
Return False
End Try
End Function

[Cor]

When I use the latest dll then I get this ouput in the log if I activate the event with the replace:

11-6-2020 11:47:43 50065 | Handle Action Entry Buiten/afdak/Temp_buiten=<<Replace($$DVR:(859):,",",".")>>
11-6-2020 11:47:43 50088 | Handle Action Topic|Payload Buiten/afdak/Temp_buiten | <<Replace(18,1,",",".")>>
11-6-2020 11:47:43 50088 | Handle Action Inline i,j 1 | 24
11-6-2020 11:47:43 50089 | Handle Action Expression | nReplace(18,1,",",".") | 0
11-6-2020 11:47:43 50139 | Handle Action oValue | n
11-6-2020 11:47:43 50139 | Handle Action Payload
11-6-2020 11:47:43 50140 | Handle Action Topic=Payload Buiten/afdak/Temp_buiten=

Furthermore, if I look in the serial monitor of the Arduino where the ESP8266 is connected to I get an exeption error:

Message arrived [Buiten/afdak/Temp_buiten]

Exception (29):
epc1=0x402014e9 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

>>>stack>>>

ctx: cont
sp: 3ffffd80 end: 3fffffc0 offset: 0190
3fffff10: 00000018 3ffef457 3ffee4c8 4020147c
3fffff20: 3ffef470 00000000 3ffee52c 40205e5a
3fffff30: 3fffff60 00000001 00000002 0000001c
3fffff40: 0001b7a5 00000018 3ffee52c 40207d14
3fffff50: 0001b7a5 00000000 3ffee52c 402034c8
3fffff60: 007a1201 2e7ec084 3ffee500 40201235
3fffff70: 00000000 00000000 80fee4e8 3ffef800
3fffff80: 3fffdad0 00000000 3ffee52c 3ffee780
3fffff90: 3fffdad0 00000000 3ffee52c 40201264
3fffffa0: feefeffe feefeffe 3ffee740 40205f3c
3fffffb0: feefeffe feefeffe 3ffe84fc 40100c11
<<<stack<<<
@⸮ *⸮⸮⸮*⸮D⸮⸮E⸮"⸮Fʁ⸮⸮⸮P⸮J⸮⸮)6ETl⸮
Connecting to sensor-net-IoT
. WiFi connected; IP address: 192.168.1.107
Hostname: ESP_Oled_1

Attempting MQTT connection....connected

I made a work around in the Arduino software to replace the comma in a period.

But I can understand that you want to know why the replace is not working so I am present to help.

[Michael McSharry]

The critical piece of information is Replace(18,1,",","."). The comma in the number is treated as a parameter delimiter. Change your expression to include quotes around the number.

<<Replace("$$DVR:(859):",",",".")>>

This will mean the debug output will then be Replace("18,1",",",".") and will evaluate properly.

Any info as to getting device 859 in HSEvent entry in the debug log?

[Cor]

If I change the replace as you mention, than the two double quotes disappear as I submit the string. Is there perhaps a HS4 problem?

I do not see more info on the event entry as I showed above.

[Michael McSharry]

I used HS4 4.0.7.0 to evaluate and had no issue on the save. The action is shown below. When it gets saved it just shows as "<>", but that is just a HS display problem with HTML open/close brackets.

You had shown device 258 above. That was the only one that was generating the HSEvent?

[Cor]

After editing I see this:



that looks good but after I push the Submit button the quotes are gone.


and after saving I see <> indeed.

You had shown device 258 above. That was the only one that was generating the HSEvent?

If I activate this event I see in the mcsMQTT log:

11-6-2020 18:48:18 24071144 | Handle Action Entry Buiten/afdak/Temp_buiten=<<Replace($$DVR:(859):,",",".")>>
11-6-2020 18:48:18 24071167 | Handle Action Topic|Payload Buiten/afdak/Temp_buiten | <<Replace(20,7,",",".")>>
11-6-2020 18:48:18 24071168 | Handle Action Inline i,j 1 | 24
11-6-2020 18:48:18 24071168 | Handle Action Expression | nReplace(20,7,",",".") | 0
11-6-2020 18:48:18 24071168 | Handle Action oValue | n
11-6-2020 18:48:18 24071169 | Handle Action Payload
11-6-2020 18:48:18 24071169 | Handle Action Topic=Payload Buiten/afdak/Temp_buiten=

[Cor]

I found the solution! I deleted the action and then I made a new mqTT message. Now the submitting is ok! In the log:

11-6-2020 18:59:20 24733773 | Handle Action Entry Buiten/afdak/Temp_buiten=<<Replace("$$DVR:(859):",",",".")>>
11-6-2020 18:59:20 24733819 | Handle Action Topic|Payload Buiten/afdak/Temp_buiten | <<Replace("20,7",",",".")>>
11-6-2020 18:59:20 24733819 | Handle Action Inline i,j 1 | 26
11-6-2020 18:59:20 24733820 | Handle Action Expression | nReplace("20,7",",",".") | 0
11-6-2020 18:59:20 24733837 | ApplyExpression Replace("20,7",",",".") Result=20.7
11-6-2020 18:59:20 24733837 | Handle Action oValue | n20.7
11-6-2020 18:59:20 24733837 | Handle Action Payload 20.7
11-6-2020 18:59:20 24733838 | Handle Action Topic=Payload Buiten/afdak/Temp_buiten=20.7

Bingo!

[Cor]

But I will look forward to use the device instead of the event route.

If I can do some testing just let me know.
Many thanks for now.
---
Cor

[Michael McSharry]

If you have an event trigger that is based upon a value change of device 859 then I expect in the mcsMQTT debug log a line with HSEvent and the end of the line being 859. It is a mystery as to why you do not see it. It is not associated with the event action. It is just normal device change operations.

Another means for mcsMQTT to be notified of a change in device 859. This is via the HS log. Does the HS log contain an indication of device 859 having a change?