Help to access data using API
Hi,
I'm trying to access web-based data from glowmarkt. I'm trying to use a Node Red HTTP request node to send Bright app userid and password to https://api.glowmarkt.com/api/v0-1/auth embedding the content type and applicationId in headers. I've spent hours researching the use of the HTTP request node but keep getting statusCode: 500.
To try to find where the problem lies, I tried using the example cURL from a command prompt on my Raspberry Pi, substituting my own credentials
curl -X POST -H "Content-Type: application/json" -H "applicationId: b0f1b774-a586-4f72-9edd-27ead8aa7a8d" -d '{"username": "MYBRIGHTIDHERE", "password":"MYBRIGHTPASSWORDHERE"}' https://api.glowmarkt.com/api/v0-1/auth"
The response was a string with a token in. When I used the next cURL example:
curl -X GET -H "Content-Type: application/json" -H "token:MYVERYLONGTOKENRECEIVEDINTHELASTSTEPPASTEDINTOHERE" -H "applicationId:b0f1b774-a586-4f72-9edd-27ead8aa7a8d" “https://api.glowmarkt.com/api/v0-1/virtualentity"
I just got a /> prompt (By the way, it looks like enter a single greater than character causes the text to become quoted). After pressing enter a couple of times, I got:
curl: (3) URL using bad/illegal format or missing URL
curl: (6) Could not resolve host: application
I can't progress the Node red implementation to receive historical data until I've sorted out why the examples from the API Documentation don't seem to work from a command prompt. I have seen that there are Postman and Swagger means to access the data and that others have created Python and HA integration but I'd just like to use Node red.
Does anyone have any idea what's not right here?
Comments
Following an email to support, it all worked Ok. Node red development is now going well
@Tim
Hello there,
Would you mind sharing your node red flow, or node settings for getting token, please? I am struggling with it. Keep getting "403 forbidden".
Thanks.
@cholek3,
I have created a contrib on Github which can be found on the Node red flows by searching for "Hildebrand". You should be able to do everything you need with my codes as a starter for ten to tweak however you wish. Note that I really struggled when I first tried this and it suddenly all started working after I contacted Hildebrand by email for help. I wonder if they have to do some tweaking at their end to enable MQTT for your data, but also for the API to work properly. Their product is a Consumer Access Device, for which they need to know your MPAN to connect the CAD to. I'm sure it's difficult for them to open up both MQTT and API access without your permission as they don't need it themselves for the CAD. The CAD connects to your SMART meter locally over Zigbee and uses your wifi to send real-time data to their web servers (hence how you can subscribe to their MQTT topic for your MPAN). I suspect it's a data protection issue but they have been very responsive when I've contacted them and my system is working now OK.
@Tim
Thank you for you quick reply. At the moment I only have API access as I wanted to see what I can do with my data. I actually managed to get through authentication process and get my token, but I got stuck on 3rd step with getting values. Either way, I will check your flow and update what is necessary. Thank you for sharing it, not only with me, but with everyone.
@Tim - bit late to the party here.
Got the IHD (finally) and its all hooked up OK and providing 'real time' data to the android app.
I also started with the cURL examples, and have got the authentication token back, but like you, I just get a prompt when I send the next line (I'm using WSL for various painful reasons, by the way)
You mention above that after consulting support, 'it all worked'
Can you share if you needed to change anything with the format of the second cURL query, or how you got it to work.
I've copied your nodered template and have got that mostly to work (bit complicated as I have two properties logged, one with electricity only, one with leccy and gas)
It was a little while ago but I think the issue was that the API wasn't open to me. Once they knew I was wanting to use the API, something changed and it all worked. Whether there's something they do at their end to allow API access, I don't know but once I'd emailed to say I was using the API, it worked.
In fairness, as long as the authentication token is renewed before expiry (I renew mine via Node-RED three times a week), it seems to be quite reliable.
I'm going to ask a really newbie question.
I have got access to the API and MQTT and I'm getting back some data that thanks to the contributions on this forum I can parse.
But I haven't figured the gas data or where electric tariff data is yet.
I feel I'm missing some documentation on the basics.
Is there a link I have not found yet?
Both API and MQTT data should be JSON objects, or can be accepted by your processing system as such. If you use MQTT Explorer and subscribe to the MQTT topic, you should be able to view the expanded JSON in the right hand pane, showing the structure. If you're looking for the specification for how the data/objects are structured, there's a specification which details the format of the data in Section D.3.2 onwards. It's a heavy read and there's a Github page which will provide more targeted information on exactly which objects returned by MQTT would be of interest. For example 0702: Metering# - 00: Reading Information Set# - 00 is your current electricity meter reading for import (0702.00.01 would be for export if it's passed back). Note that the object returned includes multiplier and divisor values to scale the value returned for the units being measured.
If you can see gas data on your In Home Display connected to your Smart Meter, the data should come back with the electricity data over MQTT. To access the tariff, you need to use the /tariff API which is mentioned in the very last paragraph of the Glowmarkt API document.
Making use of the object key values can be done in Home Assistant, Node-Red, Python amongst others. It's up to you to choose which suits you best.
I've documented all of the JSON that comes back from the MQTT API in
https://github.com/DougieLawson/glowmarkt/blob/main/mosq.c
I take that JSON and decode everything in
https://github.com/DougieLawson/glowmarkt/blob/main/mosqJSON.c
I decode just the bits I want and re-publish them on my local MQTT broker in
https://github.com/DougieLawson/glowmarkt/blob/main/mosqBloop4eva.c
As far as I can see there is no tariff info in the JSON that's published on the Glowmarkt server. I haven't looked at what's available with the https API (because my current Octopus tariff (until September) has a fixed daily price and four hours (00:30 to 04:30) at a lower tariff). Everything is going to change this month or next as I'm getting solar PV installed and may be able to export when I'm not charging the car.
Edit: Never mind, I found the postman collection and it seems to work.:)