Log Stops showing events

[randy]

I have run PHLocation since there has been a PHLocation and never have had an issue with HS logs, other than a corruption when my UPS failed. The HS log is a SQLite dB and can be viewed with with a dB browser. I use sqlite browser. You might be able to see the last record created.

Lately I have been running two instances of PHLocation 3.0.1.121 and a development version. Even with both running, I have not seen any issues.

I use iCloud support and GeoFency posting through MyHS.

[sooty]

As the author of PHLocation I'm very interested to establish if this issue is somehow related to the plug-in.

Not sure how long this issue has been around. The first post in this thread is dated February 2018.
I've looked over the PI code and I can't immediately see anything that would cause something like this. The plug-in has been around since late 2013 with no major changes in any code that I can see being an issue since February 2017.

From what I can establish reading this post, some users are only experiencing the issue once the log database size reaches the maximum size set by the user. This could indicate that HS3 stops writing to the log if it establishes that the max size has been exceeded which subsequently could indicate that the plug-in is somehow preventing HS3 from purging it's log database to reduce the file size.

Not having an in-depth knowledge of how the logging works in HS3, I've sent Rich an email asking for some pointers as to what, within a plug-in could cause this issue.

In the meantime, I have a few questions for you guys who are experiencing the issue.

1. Is your HS3 Linux, Windows or HomeTroller?
2. What version of HS3?
3. What version of PHLocation?
4. Do you only see the issue when the database file size reaches / exceeds the set value?
5. How long have you had PHLocation installed and running? (Has the plug-in previously been working OK without seeing this log issue)

Paul..

[Broes]

Thanks for diving into it!

1. Windows
2..3.0.0.435
3. 3.0.1.109
4. yes
5. very long time (yes)

[sooty]

Thanks for diving into it!

1. Windows
2..3.0.0.435
3. 3.0.1.109
4. yes
5. very long time (yes)

I could well be wrong here but I have a gut feeling the issue is that HS3 stops writing to the log if the max file size set by the user is reached.
If other users experiencing this issue see the same behaviour then it would reinforce my theory.

On my own system I don't see this issue. My log db never reaches the max size because the "Max number of days of log entries to keep" setting I have means that it clears log entries out well before the max size is reached.

As a test, you could try setting the max days to a low value to prevent the db reaching the max size and see if the problem goes away.

My settings are 14 days / 50 MB and my db never gets much above 30 MB although I don't have a whole lot of logging going on so you may need different settings.

Paul..

[John245]

As the author of PHLocation I'm very interested to establish if this issue is somehow related to the plug-in.

Not sure how long this issue has been around. The first post in this thread is dated February 2018.
I've looked over the PI code and I can't immediately see anything that would cause something like this. The plug-in has been around since late 2013 with no major changes in any code that I can see being an issue since February 2017.

From what I can establish reading this post, some users are only experiencing the issue once the log database size reaches the maximum size set by the user. This could indicate that HS3 stops writing to the log if it establishes that the max size has been exceeded which subsequently could indicate that the plug-in is somehow preventing HS3 from purging it's log database to reduce the file size.

Not having an in-depth knowledge of how the logging works in HS3, I've sent Rich an email asking for some pointers as to what, within a plug-in could cause this issue.

In the meantime, I have a few questions for you guys who are experiencing the issue.

1. Is your HS3 Linux, Windows or HomeTroller?
2. What version of HS3?
3. What version of PHLocation?
4. Do you only see the issue when the database file size reaches / exceeds the set value?
5. How long have you had PHLocation installed and running? (Has the plug-in previously been working OK without seeing this log issue)

Paul..

1. Windows 10 Pro
2. .435
3. 3.0.1.109
4. Can happen after 4 days, at day 1 database exceeds the set value
5. > 2 years. In the past no issues.

---
John

[randy]

1. Windows 10 Pro
2. .435
3. 3.0.1.109
4. Can happen after 4 days, at day 1 database exceeds the set value
5. > 2 years. In the past no issues.

---
John

What would happen if you set the database to a certain number of days, with the maximum size large enough to cover twice the number of days you set? I have mine set to 14 days and 200mb. After 14 days my db is about 100mb. This morning at 8:00AM I changed my settings to 21 days and 100 mb to see what happens. A few hours later it pruned the db to less than 100mb without issue.

Sep-21 10:40:08 AM Log Maintenance 1354 additional records were removed from the log database to reduce its size to 99.51MB.
Sep-21 7:15:00 AM Log Maintenance 279 records, selected by date/time, were removed from the log database.
Sep-21 4:15:00 AM Log Maintenance 65 records, selected by date/time, were removed from the log database.
Sep-21 1:15:00 AM Log Maintenance 184 records, selected by date/time, were removed from the log database.

[John245]

What would happen if you set the database to a certain number of days, with the maximum size large enough to cover twice the number of days you set? I have mine set to 14 days and 200mb. After 14 days my db is about 100mb. This morning at 8:00AM I changed my settings to 21 days and 100 mb to see what happens. A few hours later it pruned the db to less than 100mb without issue.

Sep-21 10:40:08 AM Log Maintenance 1354 additional records were removed from the log database to reduce its size to 99.51MB.
Sep-21 7:15:00 AM Log Maintenance 279 records, selected by date/time, were removed from the log database.
Sep-21 4:15:00 AM Log Maintenance 65 records, selected by date/time, were removed from the log database.
Sep-21 1:15:00 AM Log Maintenance 184 records, selected by date/time, were removed from the log database.

That is not possible. Maximum allowed size is 1000.

But I will give it a try with 1000 and 1 day,

---
John

[randy]

That is not possible. Maximum allowed size is 1000.

But I will give it a try with 1000 and 1 day,

---
John

What is not possible? The setting is in mb, so 1000 is 1000mb or 1gb. On my very busy system 1000mb would store 140 days of records. Mine generates about 7mb per day.

[mrceolla]

1. Windows 7 Ultimate
2. HS3 Standard Edition 3.0.0.435
3. 3.0.1.38
4. Not sure exactly but I noticed the DB file was at or near max size after the problem occurred last time (I have days set higher than max size will typically allow)
5. Been using PHLocation for about 3 years an haven't noticed an issue w/ HS logging until recently
6. I use Backitude on 2 android devices (Galaxy S4, S5)

Say, where are you guys getting version 3.0.1.109?

Edit:
Nevermind, I see I must download it manually from a different website instead of through the updater. I must have done that in the first place. I couldn't locate an explanation why it isn't in the updater, though.

[John245]

What is not possible? The setting is in mb, so 1000 is 1000mb or 1gb. On my very busy system 1000mb would store 140 days of records. Mine generates about 7mb per day.

I generate approx, 600 mb a day

—-
John

[sooty]

The possibility that PHLocation is the cause of this "log freezing" is bugging me somewhat so following an email I received yesterday from another plug-in user (who is currently not participating in this thread) with a different issue that could be related, I did some testing.

FYI I have 2 systems here that I used for testing:
1. HS3 v 3.0.0.451 running on Windows 7 (x86)
2. HS3 v 3.0.0.473 running in Windows 10 (x64)

Log settings on both systems are 14 days / 50 MB
Probably not relevant but HS3 (and all its plug-ins) on system No1 above has been up since 5th June 2018.

The other user was sending updates from EgiGeoZone mobile clients into the plug-in's built-in http server, NOT over MyHS.
His issue turned out to be a malformed url on one of his mobile clients which was causing 3 error messages from the plug-in every 5 minutes to be written to the HS3 log which coincided with the 5 minute update interval on his device and the fact that the client app would try 3 times per interval to send the data.

At the time he reported this issue, I wasn't aware that he was also experiencing the "log freezing" problem being discussed here.
The url issue has now been resolved and the plug-in is working normally and apparently the "log freezing" issue is now also resolved.

I can't for the life of me see why writing these errors to the log would cause it to freeze so I hit both my systems with 50 malformed requests at 10 second intervals to produce the same error message. I did this twice yesterday and twice today (without any HS3 restarts in between) and neither system suffered any issues with the log which is still working normally.

To test my theory about the max log size possibly being the cause, I reduced the max log size to 10MB on both systems which previously had log files over that size.
What HS3 appears to do in this situation is enter a loop where it keeps chipping a bit of the log file until it gets it down to the max size.
This test also went fine. The log files were eventually reduced to about 10 MB and continued to work normally.

So, given the fact that the above 2 tests did not recreate the issue on either of my systems and at least some users who are experiencing the issue have been using the same version of PHLocaion for some time before this log issue appeared, I'm having a hard time understanding the cause.

There are more than 100 registered users of PHLocation. At least 3 of them (that I know) don't have an issue with the HS log so there has to be another factor somewhere.

As it appears that HST are pointing the finger at PHLocation, I emailed Rich to ask for some ideas where to look for something that could cause HS3 to stop logging. So far I've not had a reply.

All I can suggest at this stage is that users with this issue disable PHLocation and see if the log issue goes away. If it does, then I'll try and investigate further.

Paul..

[randy]

I generate approx, 600 mb a day

—-
John

Wow! That is an amazing amount of logging. Seems excessive.

[John245]

Wow! That is an amazing amount of logging. Seems excessive.

I'm planning to reduce this in the future.

---
John

[Broes]

PHLocation disabled and with the Max log size on 10 Mb the log still stopped collecting at Max log size (see log below, after these entries no logging anymore).
I have reported this to support (29217).

sep-23 2:20:02

Log Info

The log database is currently 10,96MB in size.

sep-23 2:20:01

weatherXML

Check for timer items to run

sep-23 2:20:00

Arduino Plugin

New UDP Text Recieved = Alive 2

sep-23 2:20:00

Arduino Plugin

Keeping Board: 2 Alive.

[wadesready]

The possibility that PHLocation is the cause of this "log freezing" is bugging me somewhat so following an email I received yesterday from another plug-in user (who is currently not participating in this thread) with a different issue that could be related, I did some testing.

FYI I have 2 systems here that I used for testing:
1. HS3 v 3.0.0.451 running on Windows 7 (x86)
2. HS3 v 3.0.0.473 running in Windows 10 (x64)

Log settings on both systems are 14 days / 50 MB
Probably not relevant but HS3 (and all its plug-ins) on system No1 above has been up since 5th June 2018.

The other user was sending updates from EgiGeoZone mobile clients into the plug-in's built-in http server, NOT over MyHS.
His issue turned out to be a malformed url on one of his mobile clients which was causing 3 error messages from the plug-in every 5 minutes to be written to the HS3 log which coincided with the 5 minute update interval on his device and the fact that the client app would try 3 times per interval to send the data.

At the time he reported this issue, I wasn't aware that he was also experiencing the "log freezing" problem being discussed here.
The url issue has now been resolved and the plug-in is working normally and apparently the "log freezing" issue is now also resolved.

I can't for the life of me see why writing these errors to the log would cause it to freeze so I hit both my systems with 50 malformed requests at 10 second intervals to produce the same error message. I did this twice yesterday and twice today (without any HS3 restarts in between) and neither system suffered any issues with the log which is still working normally.

To test my theory about the max log size possibly being the cause, I reduced the max log size to 10MB on both systems which previously had log files over that size.
What HS3 appears to do in this situation is enter a loop where it keeps chipping a bit of the log file until it gets it down to the max size.
This test also went fine. The log files were eventually reduced to about 10 MB and continued to work normally.

So, given the fact that the above 2 tests did not recreate the issue on either of my systems and at least some users who are experiencing the issue have been using the same version of PHLocaion for some time before this log issue appeared, I'm having a hard time understanding the cause.

There are more than 100 registered users of PHLocation. At least 3 of them (that I know) don't have an issue with the HS log so there has to be another factor somewhere.

As it appears that HST are pointing the finger at PHLocation, I emailed Rich to ask for some ideas where to look for something that could cause HS3 to stop logging. So far I've not had a reply.

All I can suggest at this stage is that users with this issue disable PHLocation and see if the log issue goes away. If it does, then I'll try and investigate further.

Paul..

I am on windows 10, hs version .478...I do a daily reboot, and logging stops on a reboot about 1 time per week. It will start logging thru the reboot and then just stops. The only way to resolve it is to reboot it yet again. I do not nor have I ever had PHLocation installed. I've tried adjusting the log amounts, and day length to different levels to see if it would resolve itself, but it seems to have little effect.