Interfacing to Domoticz using MQTT
Just thought I'd share how simple it is to interface the GlowCAD data to Domoticz HA server using Node Red and MQTT. If you're running Domoticz on a Pi, you can follow the instructions to install Node Red here https://nodered.org/docs/getting-started/raspberrypi - if you've never seen it, it's a brilliantly simple way to take data from sensors and manipulate it for import into other systems.
In Domoticz, you simply need to add hardware "MQTT Client Gateway with LAN Interface," just use "none" as the publish topic. Then import my Node Red flow into Node Red ( import is on the main menu accessed top right. ) You'll also need to create virtual sensors of type Electric ( Instant + Counter ) for Import and Export, a Gas Meter for Gas, and I also have a user variable which is a signed version of the current grid power ( -ve = export. )
You'll need to double-click on the Hilderbrand SMQTT node and change the Topic, then click on the edit button next to "Hildebrand Encrypted MQTT" and set your username/password under "security." Both of these details will be supplied when you get MQTT access from Hildebrand. Finally, change the "idx" values in the various "Update..." nodes to match your virtual devices/variable ( or just remove them if you don't use them. )
Since the data's updated every 10 seconds or so ( for electricity ) it opens up quite a few possibilities. For example, I have an event script which turns on an air source heat pump in my office if average export for the last 5 minutes is higher than 650W AND the Honeywell EvoHome zone setpoint is above 18 degrees. It also lowers the zone by one degree and sets the heat pump one degree higher ( so I use gas if there's no sun, and the heat pump when there is, but still control either source via the Honeywell wall thermostat.... ) This is all reverted if average export drops below 200W.
Comments
Cool - great to see people using MQTT!
This is great I was not aware there was an MQTT interface, I must have missed it (or it is new?). I will certainly look into this as I use Node Red as part of my home automation. Thanks sidepipe.
Looking to do this but is there somewhere I can check in regards to Topic? I was also unaware of MQTT interface, I can't remember it being mentioned on any documentation.
Hi all - we've only just launched MQTT support. For the moment, if you please email support@hildebrand.co.uk from the email address of your Bright account, asking for MQTT we will send you description and set it up for you. Thanks.
Thanks Jane, thought it may be the case that it needed to be setup so have already dropped an email to support.
Hi, can anyone confirm if there is anyone dealing with these enquiries at the moment?
Hi Pashek - yes, we are monitoring support tickets - I am sorry but I can't relate your username to a ticket so that I can follow up. Can you please give me the automated ticket reference you would have in the ticket subject line? Format will be XXX-XXXXX-111 - thank you.
Thanks for sharing this @sidepipe - I've got it working with Node Red + Home Assistant + Grafana - I'll share some details soon
No problem @marrold - I see you've posted some Grafana shots in the other thread. Thought I'd share some Domoticz screenshots for my gas meter here https://bit.ly/3bREXcC for those that haven't seen it. This is completely "out of the box" with no particular setup other than the MQTT import I mentioned above. Once you have a year of data, the graphs also have comparisons to the same period last year, which can be handy!
I really like Domoticz - it's simple to set up and extremely flexible. I've got some Z Wave stuff, a HoneyWell EvoHome, various RF temperature/humidity sensors, Daikin aircon etc etc all feeding into it. I've also got a HomeSeer box ( which I mainly use for IR control, though also have another Z Wave network linked to that ) and the two systems share info using their associated APIs and a bit of scripting on my part.
Holy hell, i didnt realise we could get direct MQTT, is this from the CAD directly? If so, how to I got about requesting this? Im currently part way though writing some Node-Red flows to pull this from the API, but MQTT would be much better!
Good morning all - if you drop us an email to support@ from the account associated to your API/device - we will send you instructions on MQTT and set you up.
Happy Saturday everyone! Jane
Thank you for sharing this Sidepipe.
Has anybody had success with the MQTT stream with export electricity? For some reason the MQTT JSON shows negative exporting values as a very high numeric value when parsed.
During net consumption I get:
"04": {
"01": "006432",
"00": "001782"
},
"02": {
"00": "00"
(6018 dec is 1782 hex) which is correctly showing a net consumption of 6018 W.
During net export (approx 2900 W export) I get:
exporting @2916w
"04": {
"01": "00652C",
"00": "FFF49F"
},
"02": {
"00": "00"
(16774303 dec is FFF49F hex) which is showing an untrue value of 16774303 W consumption.
Is this a feature (or problem) with the MQTT data and is there anything I can do with it?
Thanks, Paul
@arundalep - It's a signed 24 bit value, so FFFFFF is -1, FFFFFE is -2 etc. In your case, FFF49F is 2913 ( just take the -ve of the value then and with 0xFFFFFF. )
Adam
Hi Sidepipe
Im trying to adapt your Node-Red flow for import into Home Assistant however I'm a little confused with the current energy readings (that feed into the counter IDX in Domoticz. I am assuming this should be the value in kWh, but the value I have here doesnt seem to be anywhere close to the data I see in the bright app for consumption.
Is this the same thing arundalep is seeing above?
Any ideas?
Josh
@karmacop81 I'm not sure which specific value you mean. The names are a little confusing ( as all a bit of a 5 minute hack to be honest! ) but if you look at the 4 "update" nodes you should be able to see that the inData array contains all the useful values, the various elements are:
"idemand" is current import electricity power in W
"current" is the import energy ( meter reading ) in Wh
"iexport" is the current export electricity power in W ( a positive value )
"export" is the export energy ( export meter reading ) in Wh
"isigned" is the absolute value of power flow in W ( i.e -ve = export )
"cgas" is the gas meter reading in metres cubed * 1000.
The input data is basically the Zigbee Smart Energy Profile, so you can get all sorts of other interesting info if you want to. See https://zigbeealliance.org/wp-content/uploads/2019/11/docs-07-5356-19-0zse-zigbee-smart-energy-profile-specification.pdf
Hi Sidepipe
In the Update Electricity function you have:
"svalue": inData["idemand"] + ";" + inData["current"]
idemand shows the correct value for what im using now, but current doesnt seem to match what I see in Bright...
Josh
@karmacop81 I've just looked at the Bright app ( never use it ) but I can't see anywhere obvious that it shows your current meter reading, only historical usage by day/month/year. In any case, to check the value, "current" is ( or should be ) the actual current meter reading - this should be viewable somewhere on the standard IHD that came with your meters.
It's also possible that there's a bug in there somewhere ( like I say, was a bit of a quick hack! ) but it's fine for me so can't see why yours would be any different.
Adam
Hi Sidepipe,
Thanks for your reply. I have butchered your code to get the desired result. Sadly there is no available export meter via the Glowmarkt platform (though I have been told it may come in the future dependig on permissions etc from suppliers as I understand it).
I have added hex conversion to the Extract Data node and modified the rest for Home Assistant API to allow three separate sensor outputs for instant power figures , it has been a rapid learning curve of Node Red.
I have exported the flow at https://drive.google.com/open?id=1Womy4C4jqZB7ZpExdfQew8RwpvnPwmpr if anyone will find it useful.
Thanks again, Paul
Hi Paul,
Sadly there is no available export meter via the Glowmarkt platform (though I have been told it may come in the future dependig on permissions etc from suppliers as I understand it).
Not sure what you mean here? The export meter reading is available in the MQTT data, it's elecMtr["0702"]["00"]["01"] ( or inData["export"] from my original ExtractData node? )
Hi Adam,
I'm not sure if its a SMETS1 vs SMETS2 issue but mine just has "01": "000000000000", Jane had warned me that export data wouldn't be available on mine.
I have found a workaround for now using Home Assistant's utility meter function.
Hi Sidepipe and Paul,
To clarify - what you get in the API and MQTT is not the meter's register read for Export, consumption just goes negative if exporting.
With SMETS1 meters, if they are Secure Meters, the supplier would need to turn the functionality on for us to pick up the Export Register. For SMETS2 meters - we are investigating how this works as it is early days in the new Smart Export Guarantee scheme.
In either case, we do not currently support capturing and storing the Export register reads. This is on the roadmap but won't be in the near future.
Jane
Hi Jane,
Thanks for all your help, I now have working meters for export using MQTT, Node Red and Home Assistant which was my main issue.
Hi arundalep
I was working on the same thing but using MQTT to get the data into HA. However yours blows mine out of the water so I'm shamelessly stealing it. :)
Are you doing anything particular with the Utility Meter integrations?
Josh
Hi Jane,
With SMETS1 meters, if they are Secure Meters, the supplier would need to turn the functionality on for us to pick up the Export Register. For SMETS2 meters - we are investigating how this works as it is early days in the new Smart Export Guarantee scheme.
Well, for me with SMETS2, the export meter figure is there and appears to be correct, as the CurrentSummationReceived attribute in the Reading Information set ( plus the instantaneous power export of course indicated by InstantaneousDemand in the Historical Consumption set going negative. )
Adam
Hi sidepipe/aka Adam,
Don't quote me on this as we just haven't had time to investigate - what you are picking up technically is the net consumption but not the actual register on the meter which is used to determine SEG payments.... any SEG experts are very welcome to correct me!
Hopefully we will get to SEG one of these days - although it will definitely be in a post Covid/calmer time :)
Jane
Hi Jane,
If that's the case it would be interesting to know where the value comes from - it seems to me that either you must be calculating an approximation before sending from the CAD ( or prior to sending the MQTT data ) OR it's being picked up from the meter data over the HAN. If it's the former, you'd know and can tell me :-). If it's the latter, I'm not sure what it could be other than the meter register - it would seem bizarre if the meter calculates some sort of value from the instantaneous consumption when there's a perfectly good register for it? It would also seem strange that you ( Hildebrand ) would calculate such an approximate value and send it as accurate data?
I've no way to verify 100% of course, but I do still have a current loop sensor, and the export energy calculated from that seems to match the export value from the MQTT, allowing for the relative inaccuracy of the former.
Adam.
@karmacop81 Hi Josh,
To be honest its been a massive leanring curve having never used node red or JS before. I'm glad it is useful to you (or anyone else that wants to use it). I have uploaded a txt file with the entries you need to add to your config.yaml file in home assistant for the meter component at https://drive.google.com/open?id=1u7SHaufTBf-xTwU8rHIbmYTVnfDbJBCg, they are a lazy way for me to get what I wanted to log, thanks need to go to @sidepipe for the bulk of the work and inspiring me to get it working.
@sidepipe
I have just checked again and definately have no values for export with SMETS1 supplied by OVO. I hope you dont mind me using your code as a starting point? Credit to you for all the work you put in, thank you.
Paul
Hi Paul
I have set mine up similarly, but I've not used the integration sensor. I've seen some other examples that use it, but I dont really understand what its for... Is it required?
Mine looks like this:
utility_meter:
daily_electric_usage:
source: sensor.electricity_meter_total
cycle: daily
weekly_electric_usage:
source: sensor.electricity_meter_total
cycle: weekly
monthly_electric_usage:
source: sensor.electricity_meter_total
cycle: monthly
yearly_electric_usage:
source: sensor.electricity_meter_total
cycle: yearly
daily_gas_usage:
source: sensor.gas_meter_total
cycle: daily
weekly_gas_usage:
source: sensor.gas_meter_total
cycle: weekly
monthly_gas_usage:
source: sensor.gas_meter_total
cycle: monthly
yearly_gas_usage:
source: sensor.gas_meter_total
cycle: yearly
Hi Josh,
The integration sensor is only required if you want to meter a value that doesn’t have an accumulating value, in my case electricity export until the data steam includes metered export values. It takes a live sensor reading and creates an accumulation sensor, in this case turns live W into Wh.
@arundalep
I have just checked again and definately have no values for export with SMETS1 supplied by OVO. I hope you dont mind me using your code as a starting point? Credit to you for all the work you put in, thank you.
I've no issues with you using the code... I had a similar learning curve in that I didn't know anything about MQTT, or the SEP, never used Node Red, and not that hot on JSON or Javascript, with only a couple of hours spare to look at it! As long as you don't think it's perfect code that's fine 😂