Problem importing from data file

[frank.van.der.borg]

Problem: My MCStemperature database only fills with zero's

Installation went well, virtual devices are created (see homeseer jpg), import data file from WS3600 weather station (see history3600.dat).

The problem is that the virtual devices have no value, understandable if the database is filled with zero's. Somehow the value from the file does not go into the database.

I must admit that I have had problems understanding the import configuration, but the virtual devices are created well.

Please help, what is wrong ? what did I oversee ? In the zipfile I have added the database file, the homeseer page, the datafile from the weatherstation and the ini settings.

Thanks in advance,

Frank.

[Michael McSharry]

The data appears to be separated by spaces so the first character in the VirtualDevicesmcs.txt file should be a space. I edited it and attached it. I did not test to confirm, but it should be all you need. I suspect that the first number after the time field is your T1 device. If so, then there should only be three rather than 4 blank fields at the start of VirtualDevicesmcs.txt. If this is the case then just delete the 2nd line of the file

[frank.van.der.borg]

Thank you for the fast response, but I am afraid we are not yet there.

Correct, the separator is a space, I missed that one ! and it is correct that the 21.8 is the T1 temeparture. So I deleted the second line.

Unfortunately it does not work at all, even less...The Virtual devices are not created at all (with the first wrong file that worked...). I deleted ini files, reinstalled MCS-temperature etc. Nothing.

Somehow there is something wrong...With old file I get the correct virtualdevices..But no data. In both cases the forecast devices are made very well, but... no also no data.

I use weather XML and this works fine...

What to do ?

[frank.van.der.borg]

One step further...

I couldn't let it rest...so I tried a few things...

True, the space has to be the first character, but ..... in between comma's !! Then my virtual devices are created in Homeseer, but the Virtual devices have no value and no update time-stamp (all unknown).

So I am a little further, but still the virtual devices aren't updated in HS.

I have started logging, this is what I get, please check the log.txt, I can't really understand the logging, I have also added the database still filled with zero's unfortunately.

Please let me know what a sensible next step could be,

kind regards,

Frank

[frank.van.der.borg]

More info ...might help

Michael,

It looked good, I deleted the database file and temp.ini file and reinstalled it, just to experiment. I got the following message:

<TABLE cellSpacing=2 cellPadding=0 width="100%" border=0><TBODY><TR><TD class=LOGDateTime0 noWrap align=left>17-1-2006 22:12:32 </TD><TD class=LOGType0 align=left colSpan=3>mcsTemperature </TD><TD class=LOGEntry0 align=left colSpan=8>Created Database(s) C:\Program Files\HomeSeer\mcsTemperature.mdb</TD></TR><TR><TD class=LOGDateTime1 noWrap align=left>17-1-2006 22:12:32 </TD><TD class=LOGType1 align=left colSpan=3>mcsTemperature </TD><TD class=LOGEntry1 align=left colSpan=8>Creating database table for normalize forecast data in C:\Program Files\HomeSeer\mcsTemperature.mdb\tblTemperatureForecast</TD></TR><TR><TD class=LOGDateTime0 noWrap align=left>17-1-2006 22:12:32 </TD><TD class=LOGType0 align=left colSpan=3>mcsTemperature </TD><TD class=LOGEntry0 align=left colSpan=8>Creating database table for forecast data in C:\Program Files\HomeSeer\mcsTemperature.mdb\Forecast</TD></TR><TR><TD class=LOGDateTime1 noWrap align=left>17-1-2006 22:12:32 </TD><TD class=LOGType1 align=left colSpan=3>mcsTemperature </TD><TD class=LOGEntry1 align=left colSpan=8>Expanding C:\Program Files\HomeSeer\mcsTemperature.mdb\Forecast to support charting</TD></TR><TR><TD class=LOGDateTime0 noWrap align=left>17-1-2006 22:12:34 </TD><TD class=LOGType0 align=left colSpan=3>mcsTemperature </TD><TD class=LOGEntry0 align=left colSpan=8>CreateEnvironment on line 1080 Object variable or With block variable not set</TD></TR></TBODY></TABLE>



Result, no virtual devices in Homeseer etc.

I hope this helps,

kind regards,

Frank

[Michael McSharry]

You definition file needs to be edited to replace all the commas with spaces since the space was identified to be the delimiter that is being used in the first character. I missed this the first time.

The error you see in the last post is now showing up in recent HS2 in various plugins and locations. In essence the plugin asks for the reference to a device, then when it uses this reference to get the device, HS2 returns nothing.

Since the error is happening so often, yet intermittent, I will submit a ticket to the help desk.

[frank.van.der.borg]

Working !!!

Thanks Michael,

We have got it working ! The virtual devices are created and show correct values (I think). I'm struggling a bit with the fourth parameter (the scaling factor). Some values seem to be multiplied by 100 some by 10 and some by 1 to form the integer value.

What kind of scaling values can I use ? Is it in the manual ? I'll have to look at that.

But I'm very happy :-)

Thanks for your fast responses, as you might know we have all kinds of weather in Holland, so I will have a lot of data :-)

BTW
Again the formerly mentioned error occured (error 1080) and it even halted Homeseer on MCStemperature while shutting down, after this everything went well and the data appeared.

I can go on now,

kind regards,

Frank.

[Michael McSharry]

The 4th parameter is simply a number by which the file's data is multiplied by before storing it into the DeviceString. The integer part of the result is stored in the DeviceValue. The DeviceString result is scaled by 100 and stored in the database.

An example of scaling may be to convert data in mililiters in the file to liters in HS devices. In this case a value of 0.001 would be used.

[frank.van.der.borg]

No database entries....

I somehow keep running into problems....

When I define a trend group and draw the graph I get the message:

24-1-2006 19:01:15 - mcsTemperature - FormatTemperatureScreen on line 2120 Syntaxisfout (operator ontbreekt) in query-expressie ID=.


ontbreekt = missing in Dutch

(The rest is the same in English & Dutch:-)

What is wrong ?

The database is not filled, this probably causes this error, no graphs (back ground and grid show, no lines).

The Virtualdevices give the correct data, why is nothing stored in the database ?

Sorry for all the questions, I tried many reinstallations, but I can't get it working.
Could it be the problem that I'm only have data from a file ?

Kind regards, Frank

[frank.van.der.borg]

Other questions....

I'm sorry but I can not follow you on the scaling answer.

Going from mm's to liters..?

I have in the data file: 21.8 the virtual device shows: 218, with a scaling factor of 1. The pressure is 1027.700, this gives a 1027700 with a scaling factor of 1.

What do I use to go from 21.8 to 21 ?
What do I use to go from 1027.700 to 1027 ?

If I just change the file nothing seems to change, do I have reinstall (delete ini and data base ?) before this will have effect ?

Maybe this is all caused by the same problem ? The database does not get any bigger than the initial 108 KB.

Also forecast data is not imported, it doesn't matter what I use as source. Weather_xml works fine.

What is wrong ??

Kind regards,

Frank.

[Michael McSharry]

I took the mcsTemperature.ini file that you included above and ran it for a few hours to collect data in the database and then make trend group to show a few of the non-zero readings and all worked without a glitch.

The error you are getting reflects that nothing is in the database. Is there anything in the homeseer log every 5 minutes that would indicate that the data is not getting to the database?

You have the get weather from the XML site. For this to work you need to manually enter your Weather.com license info in the mcsForecast.ini file. The manual should describe this.

You can run with the general debug box checked for 5 minutes and post the data from the HS log to see if anything in there will shed light as to why you are not recording to the DB.

As far as the scaling...
I suspect the data in your file is recorded as English format with period used to distinguish the integer from the fraction. You regional settings on your computer are likely setup to use comma for this purpose and the period is simply a separator that is ignored.

There are two potential ways to deal with this. If your input is of fixed size such that there is always a fiexed number of digits after the period for an individual sensor, then you can use the scaling factor to account for it. If it not the case then I can make a change to use the decimal substitution that was setup for the Temp05 to be applied to file input as well.

If your pressure is 1027.700 then you need to scale it by .001 to give it a value of 1027,700 per your regional setting nomenclature. When it is stored in the database it will be storred as 102770 because all values are scaled by 100 to give 2 decimal digits of precision.

There is no provision for sensor-by-sensor rounding so you will not be able to get 1028 unless I make a change. For a type barometer the scaling for display is 2 decimal digits so in your case I think it would show as 1027,70. If the European convention is integer only for air pressure then I could make this change.

[frank.van.der.borg]

The log-file shows more...?

Every 5 minutes you see a red line in the log, it is in Dutch !

25-1-2006 20:42:39 - mcsTemperature - Change Field because Het aantal querywaarden en doelvelden komt niet overeen., error number=-2147217900, Fields=Binnentemperatuur,5,Buitentemperatuur,5,Dauwpunt,5,RV _Binnen,4,RV_Buiten,4,Windsnelheid,1,Windhoek,1,Windrichting ,1,Windchill,5,Regen_uur,1,Regen_dag,1,Regen_week,6,Regen_ma and,6,Regen_tot,6,Luchtdruk,10,Trend,1

If I translate this:

Change field because the number of queryvalues en targetfields do not match, errornumber = etc.


The names come from the file we discussed earlier. This message must come from my Dutch XP-Pro ?

Here is the complete log-part:

25-1-2006 20:41:38 - mcsTemperature - EvaluateTrigger T9
25-1-2006 20:41:38 - mcsTemperature - EvaluateTrigger T10
25-1-2006 20:41:38 - mcsTemperature - EvaluateTrigger T11
25-1-2006 20:41:39 - mcsTemperature - EvaluateTrigger T12
25-1-2006 20:41:39 - mcsTemperature - EvaluateTrigger T13
25-1-2006 20:41:39 - mcsTemperature - EvaluateTrigger T14
25-1-2006 20:41:39 - mcsTemperature - EvaluateTrigger T15
25-1-2006 20:41:39 - mcsTemperature - EvaluateTrigger T16
25-1-2006 20:42:39 - mcsTemperature - Log Temperature Sensor Count=16, INSERT INTO Temperature(SampleDate,Binnentemperatuur,Buitentemperatuur,D auwpunt,RV_Binnen,RV_Buiten,Windsnelheid,Windhoek,Windrichti ng,Windchill,Regen_uur,Regen_dag,Regen_week,Regen_maand,Rege n_tot,Luchtdruk,Trend) SELECT #2006-01-25 20:42:39# AS QT,21300 AS Q0,-2100 AS Q1,-8000 AS Q2,3800 AS Q3,6400 AS Q4,0 AS Q5,0 AS Q6,0 AS Q7,-2100 AS Q8,0 AS Q9,0 AS Q10,108700 AS Q11,134600 AS Q12,315000 AS Q13,1,0276E+08 AS Q14,0 AS Q15
25-1-2006 20:42:39 - mcsTemperature - Change Field because Het aantal querywaarden en doelvelden komt niet overeen., error number=-2147217900, Fields=Binnentemperatuur,5,Buitentemperatuur,5,Dauwpunt,5,RV _Binnen,4,RV_Buiten,4,Windsnelheid,1,Windhoek,1,Windrichting ,1,Windchill,5,Regen_uur,1,Regen_dag,1,Regen_week,6,Regen_ma and,6,Regen_tot,6,Luchtdruk,10,Trend,1
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T1
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T2
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T3
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T4
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T5
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T6
25-1-2006 20:42:40 - mcsTemperature - EvaluateTrigger T7
25-1-2006 20:42:41 - mcsTemperature - EvaluateTrigger T8

What is a good next step ?

I hope this will help to solve the issue.

I missed the ini-file part for the forecast, I had not reached the last part of the manual. Scaling is more clear now, I will try this as soon if we have this working.

With kind regards,

Frank.

[frank.van.der.borg]

More logging... more errors...

I decided to try the scaling, the effect is that no data at all is imported to the virtual devices in HS.

1. I deleted mcstemperature and the data base
2. restarted HS
3. after changes in the menu

There are a lot of different errors in the logging, this could help, I hope,

I think a give it a rest for tonight :-)

Kind regards,

Frank

[Michael McSharry]

On the first of the two posts the line to write to the database has the following for what I think is your Luchtdruk field.

1,0276E+08 AS Q14

This number is not reasonable and actually is treated by SQL/Access as two numbers because the comma is a separator. The error messages comes as the plugin tried to correct the problem by updating the database field name, but is unable.

If this is an imported number then you need to look at the source data and the field definitions to make certain you are picking up the correct stuff or scale it down by a million or so.

The second log contains the following errors
CreateEnvironment on line 1080 Object variable or With block variable not set
Setup Load on line 3750 Object variable or With block variable not set

There is a help desk ticket open with HST to correct the problem of DeviceClass objects not being returned with valid device references.

Because of this I suspect your database is not being completely created. You can post it and I can take a look as to its state.

[frank.van.der.borg]

A few tests done...

I took out that value, no idea where it came from it is not in the datafile coming from the weather station, but anyhow.

The result is the same, the same scientific number appears at the same position (3rd position befor the last). Even if I go from 16 to 15 devices....it just moves with it.

I have added everything in the logging. Also the datafile that is imported, no scientific numbers there!

I don't know it is a strange problem, maybe the other problems (you entered through the ticket), cause the errors, I cannot oversee that.

Any idea when that will be solved ? does that mean a release of HS ? Im already using the latest version.

Kind regards,

Frank