Simple web request to pull Octopus Energy rates not working.

[ecuboss]

I'm trying to access the Octopus Energy API to grab the current rate for a given time, a working example URL is:

https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=2020-02-06T22:30:00Z&period_to=2020-02-06T22:35:00Z

Even this:
"curl https://api.octopus.energy/" from the command line eturns a formatted response if but fails in my script.

If relevant - HS is running on Windows 7 32bit.


https://developer.octopus.energy/docs/api/

I've tried the built-in hs.urlaction/geturl and the .net methods and they both fail:

Sub Main(parm as object)
      Dim Period_from As String
        Period_from = (System.DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssK"))
        Dim Period_to As String
        Dim response As String
        Period_to = (System.DateTime.UtcNow.AddMinutes(1).ToString("yyyy-MM-ddTHH:mm:ssK"))
        'Built URL based on the time now.
        'Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=" & Period_from & "&period_to=" & Period_to
        'example of a working URL:
        'Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=2020-02-06T22:30:00Z&period_to=2020-02-06T22:35:00Z"

        'very simple request that should return "not found"
        Dim requestUrl As String = "https://api.octopus.energy/"  '

        response = hs.URLAction(requestUrl, "GET", "", "Content-Type: text/plain")
        hs.WriteLog("response:", response)
End Sub

Throws this:

Feb-15 14:45:23

response:

The underlying connection was closed: An unexpected error occurred on a send.

Feb-15 14:45:23

Error

In URLAction GET: The underlying connection was closed: An unexpected error occurred on a send.

Imports System.Convert
Imports System.IO
Imports System.Net    

Public Sub Main(ByVal Parms As Object)

        Try
            Dim myWebRequest As WebRequest = WebRequest.Create("https://api.octopus.energy")
            ' Send the 'WebRequest' and wait for response.
            Dim myWebResponse As WebResponse = myWebRequest.GetResponse()
        Catch ex As Exception : hs.WriteLog("Error:  ", ex.Message.ToString)
        End Try

    End Sub

Throws:

Feb-15 14:47:42

Error:

The underlying connection was closed: An unexpected error occurred on a send.

[jon00]

See if this works:

Imports System.Net    

Public Sub Main(ByVal Parms As Object)
    System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12
    Dim response As String = ""
    Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=2020-02-06T22:30:00Z&period_to=2020-02-06T22:35:00Z"  '
    response = hs.URLAction(requestUrl, "GET", "", "Content-Type: application/json")
    hs.WriteLog("response:", response)

    End Sub

[ecuboss]

Yes it does work - been stuck on that all morning, Many thanks.

Any tips on how i can extract the "value_inc_vat" value from the response?

[jon00]

It's a json response so you would need to deserialize.

Example here:

https://forums.homeseer.com/forum/de...450#post910450

You can also use my datascraper script which you can extract the data though regex and show on a virtual device without any coding.

[ecuboss]

So i've spent loads more time and feel close but no cigar,

I can extract from the first level, such as obj.count - but

I cant extract the

value_inc_vat from the results subsection.

I've got it working with tenscripting as well now and i can see the data is at my fingertips:

Imports System.Web.Script.Serialization

 Public Sub Main(ByVal Parms As Object)
        System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12

        Dim Period_from As String
        Period_from = (System.DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssK"))
        Dim Period_to As String
        Dim response As String = ""



        Period_to = (System.DateTime.UtcNow.AddMinutes(1).ToString("yyyy-MM-ddTHH:mm:ssK"))
        Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=" & Period_from & "&period_to=" & Period_to
        'Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=2020-02-06T20:30:00Z&period_to=2020-02-06T22:35:00Z"

        hs.WriteLog("url:", requestUrl)

        response = hs.URLAction(requestUrl, "GET", "", "Content-Type: application/json")
        Dim json As New JavaScriptSerializer
        hs.WriteLog("reply:", response)

        Try
            Dim dataObj As jsonStructure = json.Deserialize(Of jsonStructure)(response)



            hs.WriteLog("response:", dataObj.results.value_exc_vat.ToString)


        Catch ex As Exception : hs.WriteLog("Test Script", "Error:  " & ex.Message.ToString)
        End Try

    End Sub


    Public Class jsonStructure
        Public Property count As Integer
        Public Property _next As Object
        Public Property previous As Object
        Public Property results() As Result
    End Class

    Public Class Result
        Public Property value_exc_vat As Single
        Public Property value_inc_vat As Single
        Public Property valid_from As Date
        Public Property valid_to As Date
    End Class

[jon00]

You can use a quick and dirty method to get JSON data:

Imports System.Web.Script.Serialization

 Public Sub Main(ByVal Parms As Object)
        System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12

        Dim Period_from As String
        Period_from = (System.DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssK"))
        Dim Period_to As String
        Dim response As String = ""



        Period_to = (System.DateTime.UtcNow.AddMinutes(1).ToString("yyyy-MM-ddTHH:mm:ssK"))
        Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=" & Period_from & "&period_to=" & Period_to
        'Dim requestUrl As String = "https://api.octopus.energy/v1/products/AGILE-18-02-21/electricity-tariffs/E-1R-AGILE-18-02-21-B/standard-unit-rates/?period_from=2020-02-06T20:30:00Z&period_to=2020-02-06T22:35:00Z"

        hs.WriteLog("url:", requestUrl)

        response = hs.URLAction(requestUrl, "GET", "", "Content-Type: application/json")
        Dim json As New JavaScriptSerializer
        hs.WriteLog("reply:", response)

        Try

           Dim dataObj As Object = json.Deserialize(Of Object)(response)

           hs.WriteLog("response:", dataObj("results")(0)("value_exc_vat").ToString)
           hs.WriteLog("response:", dataObj("results")(0)("value_inc_vat").ToString)
           hs.WriteLog("response:", dataObj("results")(0)("valid_from").ToString)
           hs.WriteLog("response:", dataObj("results")(0)("valid_to").ToString)
           hs.WriteLog("response:", dataObj("count").ToString)


        Catch ex As Exception : hs.WriteLog("Test Script", "Error:  " & ex.Message.ToString)
        End Try

    End Sub

[ecuboss]

Thanks - that seems to be pulling the rates in nicely thanks.

[ecuboss]

As a follow-up - here's what I wanted to do.
Octopus energy in the UK have a "time of day" pricing scheme, where it's generally lower than a flat rate by increases massively in the early evening to discourage peak usage.
As we cook with electricity and typically around 5pm, I wanted to see what the impact would be compared to having cheaper electricity elsewhere.

• Yellow line is live consumption in Kw
• The blue line represents how much I spent in 24 hours on flat rate electricity (13p a Kwh)
• The red line against Y2 axis is the pence per Kwh for Octopus - as you can see, it varies between 5p and 25p.
• The green line is what I would of spent if I was using the Octopus [varied] rate. You can see I save money, then it catches up in the evening but never overtakes the flat rate.

I used a TLS257 photodiode connected to a NodeMCU board the Enigmatheatre Arduino Plugin in API mode to pulse count the usage of my electricity meter
Scripts to pull in the rates and bring the pulses in and update virtual devices with consumption
and the "Device History" plugin for the graphs.

[chrisgla]

Did you manage to get Octopus Agile rate import fully working, and if so could you share script?
It's years since I've done any scripting so I'd started to import individual half hourly rates to devices using Big5- it seems slightly easier but may be just as much learning by the time I add the logic and calculations.

I want to use the rates to work out cheapest time to charge for car and possibly set charge rate as well.
Ideally, knowing battery capacity (set value) and state % (from Teslaseer) to get kWh needed, divide by 7kW and then identify half hourly charge slots.

Ambitious

I have some other load I plan to shed during the 4-8pm peak rate (also looked at battery but too expensive and bureaucratic for now)

There's an interesting example in the Octopus Agile API reference of how to get the lowest rate on day- I need to figure out how to get the "x" minimum half hourly slot rates:-

$ http $TARIFF_URL/standard-unit-rates/ \
period_from=="2018-05-16T00:00" period_to=="2018-05-17T00:00" | \ jq '.results| min_by('.value_inc_vat')' { "value_exc_vat": 7.2, "value_inc_vat": 7.56, "valid_from": "2018-05-16T04:30:00Z", "valid_to": "2018-05-16T05:00:00Z" }

Particularly with many of us being stuck at home during the day Agile rates make a significant difference.

I also have a Hildebrand Glowstick CAD + HAN on the way that should give real time data using the Smart Meter Zigbee interface via MQTT or Modbus.

I don't know how many are interested in the UK, I'd be happy to put towards a bounty to get a proper PI or script as I think a few of us in UK would benefit...

[chrisgla]

Received my Hildebrand IHD+CAD on Monday, few teething problems but I'm now getting live consumption data from Smart meter.
Hildebrand API detail here:- https://docs.glowmarkt.com/Glowmarkt...rForBright.pdf
And examples/ more docs here:- https://bitbucket.org/ijosh/brightglowmarkt/src/master/

The TOU rates aren't coming across from Octopus in the meter data as yet- just the previous set tariff. Unsure if others are getting this data in their IHD?

Going through a steep learning curve with Postman!

[manaesh]

Hi Ecuboss

I am trying to accomplish the same solution as you. Would you mind sharing your scripts/advice on how you got your solution working?

I already have my power via currentcost, and the device history plugin.

Due to migrate to Agile Octopus in 2 weeks time!

Cheers

Mike

[chrisgla]

As an option, there’s a flow to pull the data into Node-Red:- https://discourse.nodered.org/t/best...ost-array/7929

I tried Homeseer scripts and got one working, but my limited programming skills wouldn’t have been up to keeping it up to date or debugging. From Node-Red I then just use mcsMQTT to bring in the current rate or other data as needed to Homeseer. Node-Red also feeds influxdb/ Grafana directly avoiding loading up H9meseer with this task.

[manaesh]

Thanks Chris - will take a look at Node-Red.

I have got Jon00s script working and putting the data into the log....

Stupid question - but how do i get to write that to a device? Its been a while (10 years) since i dabbled in scripting, and that was in HS2!

Thanks anyone for any help!

Cheers

Mike

[jon00]

Thanks Chris - will take a look at Node-Red.

I have got Jon00s script working and putting the data into the log....

Stupid question - but how do i get to write that to a device? Its been a while (10 years) since i dabbled in scripting, and that was in HS2!

Thanks anyone for any help!

Cheers

Mike

It may be easier if you use my datascraper script which parses JSON and creates devices for you.

[manaesh]

Thanks Jon

I will take a look at that and try and get my head round it - JSON is a new area for me....

Then i need to work out how to get the cost per hour from my Currentcost for both rates....

INteresting excerise!

Cheers

Mike