403 Forbidden

[Pnord]

First off, I'd like to commend Jeff on his fantastic WeatherXML plug-in. It does everything you could ever want and more, beautifully.
But, of course, tinkerers feel the need to re-invent the wheel. There is, after all, a certain thrill when you finally get something to work.

The 403 error is discussed at length in the WeatherXML forum, but I have 2 questions.

First, it would appear that NOAA has changed the rules for it's XML site. However,contrary to what i read, my HS2 script continues to work.
(abbreviated)

Sub main()
xmlfile = "http://w1.weather.gov/xml/current_obs/KCBG.xml"
Set objXMLDoc = CreateObject("Microsoft.XMLDOM")
objxmlDoc.async = False
objXMLDoc.load(xmlfile)

Loads just fine, it is my HS3 .vb script that returns the error

Public Sub Main(ByVal Parms as Object)
DIM NOAAUrl As String = http://w1.weather.gov/xml/current_obs/KCBG.xml"
Dim NOAAXML As XMLDocument
NOAAXml.Load(NOAAUrl)

Returns the 403 Forbidden error. (I presume it is at least trying to load in order to return the error)
Why would the one continue to work after NOAA's change & the other not?

Second, of course, is what on earth is an "HTTPRequestHeader.UserAgent"
and how do you put it in a script ?

Thanks for sharing, without the examples posted here I would be even deeper in the weeds.

Paul

[mrhappy]

When you do XMLDocument.Load and pass it a web address then behind the scenes it does some sort of method to try and grab that data from the internet. I know that the options available to configure this particular method are I believe virtually if not completely non-existent so if you come against a particular web service that needs a password or some type of additional header then you have nowhere to set the data up. I suspect that the NOAA for reasons like knowing whether you are accessing something either via a mobile device or some other means then you must include the HTTP User-Agent header. When you include this it appears to work, if not then it does fail.

This seems to work, it tricks the NOAA into thinking the request has come from a Chrome browser and then downloads the XML. In this example I don't know what you are doing with it so I just download it to the HS main directory called test.xml

imports system.net

Sub Main(ByVal Parms As Object)

try
        dim url as string = "http://w1.weather.gov/xml/current_obs/KCBG.xml"

        dim client as new WebClient

        client.headers.add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36")

        dim data as string = client.DownloadString(url)

        dim doc as new XmlDocument

        doc.LoadXml(data)

        doc.save(hs.getapppath & "\test.xml")

catch wex as webexception
        hs.writelog("", "Web Exception: " & wex.tostring)
catch ex as exception
        hs.writelog("", "Exception: " & ex.tostring)
end try

End Sub

[Pnord]

Thanks, I'll try this over the weekend.
Paul