Future API - Features Or Changes You Would Like To See
Hi all,
Hildebrand are potentially looking into exposing more of their API's to the public, with this in mind what features or changes would you like to see?
I hope we can use this discussion to bounce around some ideas, especially if people use the current API in different ways or have different needs.
For me at this stage I hope that they can fix some of the minor bugs / inconstancies with the current cost reporting (not a major issue at all and is easy worked around). As to new features I am all ears 😀
Comments
the mix of timestamp formats is a bit frustrating. They expect requests to be in the format "2018-04-10T00:00:00" (which i believe is ISO 8601 although if you’re gong to go for that why not also include time zone?) but return responses as unix timecode (e.g. 1523318400).
it would make it easier if they accept the from/to time in data requests in unix timecode, or accept either format and handle the conversion for the requestor?
also it'd be good if the API included the timestamp of the earliest and latest available record in the "resources" response so that it's easy to pull all the available data.
Volume data for gas meters would be fantastic.
At the moment I'm having to do a calculation but it isn't quite right because I don't know the values the meter is using internally to calculate the kWh value, so i can't accurately convert back to volume.
Awesome feedback all -- "organic" APIs are not the way to go I can tell you!
I think we are going to need to do a bit of a revamp to get everything in good shape. Some of the balance has been between single site and multisite (or multi-meter) features, how industry expects to see APIs, etc. But clearly there are many people that would benefit from something that is friendly and then lets you dive down into finer control.
Also we are interested in what other data sources are key - things like weather, weather forecasts, as above calorific values, volume, etc.
Also we are interested in what other data sources are key - things like weather, weather forecasts, as above calorific values, volume, etc.
I certainly think weather / weather forecasts would be a great addition as that would sit side by side with power planning or predictive usage.
Hi Team,
Having the Electricity meterread API call historical rather than instant would be a great help. This would help to reconcile actual power imported (particularly for those who have solar panels) - I know you are looking into the possibility of changing the readings to show Actual Power Imported as opposed to netting the data (which can sometimes show negative in the App and API on sunny days !), but as a stop gap measure, the above would be very helpful.
Thanks.
I mentioned it on another thread but I'll put it here too for visibility, I'd love to see some lower level metrics exposed via MQTT - Volts, Hertz, Power Factor etc. I'm not sure if these metrics are available to you? Obviously I understand your average consumer probably isn't interested in these metrics, but some of the geekier amongst us probably are, myself included.
Hi all - I've added the request to our requirements list. As @jcooper said above, we are looking at all of this and we are thinking about what else we might want to expose in our public API's. As I'm sure you can appreciate - current focus is on the big projects but we do try and keep improving how we support our individual customers as well. Keep the requests coming, we are happy to take them - just can't promise whether and when we will deliver :)
The meter I have (Secure Liberty 100) has a record of the Exported kWh (EXP_KWH), and it would be good if this could be available via the API (like the meter reading of imported kWh can be done currently).
Hi andrewjb, I have requested the same - I have also asked if this could be historical rather than just instant and this is something they are looking into. If you have subscribed to the MQTT service, you may be able to get access to the Import & Export read (I currently do this). The format is JSON (although if you are using PHP you will have to play about with it a bit as the arrays are numbers and PHP does not like this) and an example of the output is below (using a Raspberry Pi 3B+ as the client):-
Client mosqsub/25036-rpi1 sending CONNECT
Client mosqsub/25036-rpi1 received CONNACK
Client mosqsub/25036-rpi1 sending SUBSCRIBE (Mid: 1, Topic: SMART/HILD/<your MAC for IHD/CAD>, QoS: 0)
Client mosqsub/25036-rpi1 received SUBACK
Subscribed (mid: 1): 0
Client mosqsub/25036-rpi1 received PUBLISH (d0, q0, r0, m0, 'SMART/HILD/<your MAC for IHD/CAD>', ... (627 bytes))
SMART/HILD/<your MAC for IHD/CAD> {"elecMtr":{"0702":{"03":{"01":"00000001","04":"00","02":"000003E8","07":"<your MPAN>","03":"FB","08":"","00":"00","06":"00"},"00":{"07":"00000000","01":"0000000E6CB2","00":"000000728B4D","14":"02","02":"000000000000"},"04":{"01":"00121E","03":"07A10E","00":"FFFFFB0E","02":"00FA5F"},"02":{"00":"00"}},"0708":{"01":{"01":"E.ON"}}},"gasMtr":{"0702":{"00":{}}},"ts":"2020-05-29 09:14:18","hversion":"GLOW-IHD-01-1v4-SMETS2","time":"5ED0D26A","zbSoftVer":"1.2.5","gmtime":1590743658,"pan":{"rssi":"B9","status":"joined","nPAN":"00","join":"0","lqi":"74"},"smetsVer":"SMETS2","ets":"2000-01-01 00:00:00","gid":"<Your GID>"}
Highlighted stuff is unique to you - If you look at the row that starts: {"07": you will see that array "01" is your Export and array "00" is your Import, so for me at 09:14:18 on the 29th May 2020 my Export Meter Read was: 0000000E6CB2, which is 945330 decimal, so 945.330kWh exported...
Hope this helps.
Dan.
I've asked about reading the export register with MQTT but I think you can only do that if you're on an export tariff and meter has been programmed to provide that reading. Certainly my JSON string doesn't have a block labelled ["elecMtr"]["0702"]["01"][anything].
Rather annoying as I'm trying to monitor exports to see if it would be worth joining a SEG scheme. I've had to settle for reading the register manually once a month.
I'm now able to get my meter readings with MQTT but rather disappointed to see that rather than the 3 decimal places that Octopus seem to offer (but not till following day) I'm only seeing the first decimal place plus a 0 or 5 in second and third is always 0.
Hi KS_Eric,
Would you be able to post the full JSON string from the MQTT (the same format as mine above), make sure you remove your MAC/GID/MPAN information before posting. Also what MQTT client are you using to pull the information and what is the make/model of your meter ?
If you are not on the legacy FIT scheme and looking to join SEG, it's always worth it as you will get something rather than nothing (assuming you do export some power). If you are already on FIT, it will not likely be worthwhile to move over to SEG as you only get paid for Export and not Export AND Generation.
Dan.
Here's the latest JSON string but 'prettied up' and with MPAN & meter serial number removed :-
{
elecMtr: {
0702: {
03: {
01: "000001",
05: "43",
04: "9B",
02: "0003E8",
07: "(My MPAN)",
08: "(My Serial Number)",
03: "43",
00: "00",
06: "00"
},
00: {
14: "02",
07: "27216A9A",
01: "000000000000",
00: "00000028B644",
02: "000000001F5C"
},
04: {
01: "005BFE",
00: "00034F"
},
02: {
00: "00"
}
}
},
gasMtr: {
0702: {
00: { }
}
},
ts: "2020-10-20 09:30:05",
hversion: "EHZBWIFI0v4",
time: "5F8EAE1D",
gmtime: 1603186205,
pan: {
status: "joined",
nPAN: "00",
join: "0",
lqi: "FF"
},
ets: "2020-10-20 09:30:02",
error: {
lastCommand: "00",
errorCode: "00"
},
gid: "70B3D521E0004195"
}
I don't think the Glowstick's MAC or serial number are included there.
Meter is a 'Secure Liberty 100' and my MQTT client is whatever you get when you use this first line in a Python script :-
import paho.mqtt.client as mqtt # pip3 install paho-mqtt
My understanding is that my FIT contract is valid for the full 25years (from 2011) and the generation payment is payable for the whole period (but incremented with CLI). The 'export payment' (also index linked) has been based on a deemed 50% of exports but I should be free to opt out of that and have a new SEG contract without losing the generation payment.
Hi KS_Eric,
I was under the impression you couldn't do both. I recently had a complaint against British Gas over this as I wanted an Export MPAN, but I would have to move over to SEG to get this and loose FIT, certainly worth checking out if you believe you export more than 50% of what you generate. There does seem to be a lot of conflicting information over this and I don't think the power companies and even Ofgem fully understand it all !
As for the MQTT dump (the important bit below):-
00: {
14: "02",
07: "27216A9A",
01: "000000000000",
00: "00000028B644",
02: "000000001F5C"
01 (Export) is showing ZERO
00 (Import) is showing 28B644 (2668.100 decimal).
So it does indeed look like it is not reporting the reading (but is as you say registering it as you can view it manually) and not showing the full 3 decimal places on Import.
I did a little research and from what I understand the 'Secure Liberty 100' is a legacy SMETS 1 meter that has been adopted by the DCC (https://www.smartme.co.uk/technical.html). It maybe that export and extended data (3 decimal places) is not being sent up to the DCC from a legacy SMETS 1 meter, but that seems odd if Octopus are seeing 3 decimal places....
Also, I was under the impression the Glowstick did not support SMETS 2 and only SMETS 1 meters so it might be worth talking directly to the support guys to see if the Glowstick supports reading/sending Export and/or extended data, as this may be where the issue really is...
Dan.
Excuse my ignorance, but
Documentation on the MQTT payload would be most useful as I can't fine this anywhere.
many thanks!
Justin
Hello Justin,
We don't currently support water meters because there is such a range of communication protocols they use - we'd like to us there is so much waste but it doesn't feel as though there are standards as there are with smart meters.
On the MQTT - and sorry as we should have sent you some background and one of these days will get the time to get the fuller glowmarkt site live with all this information - I believe these are the documents you need to peruse (and they take some reading!)
The data is delivered using SEP (zigbee Smart Energy Profile specification) cluster identifiers encoded in JSON (https://www.json.org/json-en.html)
Detail on the SEP is available in this document, we suggest reviewing the Simple Metering Cluster section : https://zigbeealliance.org/wp-content/uploads/2019/12/docs-07-5356-19-0zse-zigbee-smart-energy-profile-specification.pdf
Hope that is helpful.
And Dan, we are hoping to do a short production run of the GlowStick as a SMETS2 variant....
For people with SMETS2 meters, we are well underway with supporting export - we just now need to build the front end interface and include in our APIs. We already have the Shine app for people with clamp/tx solution (which allows generation measurement) - but working out the best app for representing export and consumption is our next step, when we have time.
Hi Jane,
This is brilliant news ! :)
Dan.
Has export been included in the API / App yet?
A refresh token call, so I don't have to store my credentials in an IoT device?
There is a refresh token API at
#!/bin/bash curl -X POST \ -H "Content-Type: application/json" \ -H "applicationId: b0f1b774-a586-4f72-9edd-27ead8aa7a8d" \ -d '{"username":"example@gmail.com","password":"redacted"}' \ "https://api.glowmarkt.com/api/v0-1/auth" | jq '.token' > tokenI'm using JQ to pull the token out of the CLOB of JSON that comes back from that API end-point.
@BruceH5200 - apologies for the delay - no, we haven't had time to support export within the API yet.
And @DMoore - unfortunately the global component shortage is causing us nightmares as well as everyone else - which has delayed the launch of the SMETS2 GlowStick but it is definitely on the horizon.
@JaneatGlow the component shortage is getting an extra ten day blip in the supply chain. Something to do with "sideways", "EVER", "GIVEN" and "Suez".
Feature request: could supply voltage be added to the local mqtt data?
Mainly for interest, and also so that I can cross-check with some smart plugs that are reporting it, and possibly use the data to correlate with devices "going wrong".
I was surprised to measure 246V - I expected to see something closer to 230V
A very belated reply to @marrold and @SJMac about adding things to the local mqtt data - unfortunately voltage and power factor aren't available on the HAN and therefore aren't things we can add to that feed.
On the more positive news front, these are data streams we are looking at picking up and adding to our APIs at some point.