Prometheus export
Hi all,
In case it is useful to anyone, I wanted to share that I wrote an app that takes the data exported by the IHD (either cloud or local MQTT) and exposes it as Prometheus metrics.
You can find the source code here, or it's also available as a docker container here. Hopefully it's pretty self-explanatory how to run it, and once it's up you just have to point Prometheus to it and you can use Grafana to create lots of lovely graphs 😀
Hope this is useful, and if you have any problems feel free to raise an issue on GitHub or just post here.
Cheers,
Andrew
Comments
I've got most of the way Andrew but am picking up this error on the container? I'm a novice so bear with me please.
'<=' not supported between instances of 'str' and 'int'
Ah, that's interesting. Does it print out a full traceback, including line numbers? I need that to be able to work out where the problem is.
Andrew
Not that I can see, I’ll dig into the logs later and see if there’s anything else. One question, without a topic how does it know where to look on the local MQTT server? That’s running on Hone Assistant, Prometheus and your app are running on my QNAP NAS.
Thanks, I will see if I can workout how the error is being thrown and make it clearer.
It just listens to everything under the glow/ topic, so it should pick everything up.
Andrew
Ta, is the bind variable in the format xx.xx.xx.xx:port? Also, does localhost work? I’ve had problems in the past between the two but usually just bridge the container to my LAN so it’s exposed. I’ll switch it to cloud later and see if it’s the container or my network topology.
Yes, bind is ipaddress:port. I've not tested localhost, but it should work.
Connected with result code 0 using Cloud which I assume is normal? Nothing showing under Prometheus though so I assume I've got something wrong on the binding. Is this the address of the Prometheus server?
The bind address is where it exposes the metric, you then need to add a line to the prometheus configuration file to scrape it.
You should be able to visit http://bindaddress:port/metrics in a web-browser to confirm it's working.
In your prometheus.yml file you can add something like:
scrape_configs: - job_name: 'glowprom' static_configs: - targets: ['bindaddress:port']I'm definitely making a hash of the networking, can't connect to anything! I've tried NAT, Bridge, various ports.
Sounds like you might have bigger docker issues then, not specifically glowprom. You want to try asking somewhere like Stack Overflow to see if they can help get docker working.
Andrew
I'll have a play tomorrow, portainer is working along with gluten and qbittorrent. I think it's entirely a networking issue. Thanks Andrew.
We've made some progress, I removed the BIND option and let it default as per the code then port forwarded 9100 behind the virtual switch. The webpage is showing now but metrics is giving a 404 error. It's also connected to my MQTT server as seen in the logs.
That’s definitely good progress :-) I will have to test the bind parameter to make sure it works correctly.
The metrics will return 404 until it receives some data. Are you able to confirm the Glow IHD is sending data to your mqtt server?
Andrew
Yep, getting a 404 error. The MQTT server receives from the cloud using an MQTT bridge. This could be something to do with storage on the NAS and permissions. That's my next task. I was getting the same error using cloud.
Ah, I hadn’t planned for that use case. The topic names and message formats are different for cloud and local mqtt modes. It won’t work with cloud format via a local server.
If you configure it to point to the cloud directly it should work.
Quick question, it's probably me, server.py, line 39, self.wfile.write(""", is that formatted correctly, no close brackets?
I've tried local and cloud and I still get 404?
That line is ok. Triple quotes are a multi line string in Python. It’s closed on line 46.
If it’s connecting to MQTT ok then it must be listening to the wrong topic. That’s handles in mqtt.py line 47. You probably want to experiment with different topic names.
Andrew
Ta, told you, it was me.
I have:
SMART/HILD/XXXXXXXXXX
I think if you change DEFAULT_MQTT in arguments.py line 22 to your local server, and pass —topic SMART/HILD/+ it might work.
On local the device is at glow/XXXXXXXXXXXX/SENSOR/electricitymeter & glow/XXXXXXXXXXXX/SENSOR/gasmeter
That should match the default topic name, if you pass —mqtt serverip I would expect it to work.
Sorry Andrew, that topic is what I have for cloud, SMART/HILD/XXXXXXXXXX.
Cloud is working, thanks for all your help Andrew. Looks like I was chasing rainbows yesterday as cloud MQTT was down!
Local working too, looks like the bridge isn't an issue. I'll post a docker compose yaml at some point.
Yay! So glad it’s working! I was very confused :-)
My docker compose yaml if anyone is interested? This works on QNAP using portainer stacks. I had to manually import the image, also 'latest' didn't work for some reason.
version: "3.9"
services:
glowprom:
image: andrewjw/glowprom:0.5.0
container_name: glowprom
environment:
- PGID=100
- PUID=1000
# Use below options for local MQTT (comment out whichever option is not needed)
- GLOWPROM_MQTT=X.X.X.X #MQTT server address
- GLOWPROM_PORT=1883 #or whatever local port is used
- GLOWPROM_USER={mqtt local username}
- GLOWPROM_PASSWD={mqtt local password}
# Use below options for cloud MQTT (comment out whichever option is not needed)
- GLOWPROM_USER={cloud username}
- GLOWPROM_PASSWD={cloud password}
- GLOWPROM_TOPIC=SMART/HILD/{glow device ID}
ports:
- 9100:9100 # Prometheus
restart: unless-stopped
Below is the end of the Prometheus yaml file, I added everything from -job_name: 'glowprom' down:
static_configs:
- targets: ["localhost:9090"]
- job_name: 'glowprom'
static_configs:
- targets: ['localhost:9100']
Thanks for this.
A couple of things:
1 - Initially trying to run "docker pull andrewjw/glowprom" didn't work:
so I had to run "docker pull andrewjw/glowprom:0.5.2"
2 - I'd like to run this on a Raspberry Pi in Docker but get this:
It runs OK on my Ubuntu Linux (using Strickers' yml file and I can see the output at http://localhost:9100/metrics) so I expect it's to do with the architecture. I'm a beginner with docker so just wondering how easy it is to make a version for Pi
3 - I have recently set up a Grafana Cloud account but I have no idea how to get this into Grafana Cloud. Wonder if anyone can help please? Or do I have to work out how to install Grafana and Prometheus locally for this to work?
Many thanks.
Hi tarmacalastair,
You're right, I was only building an x86 image and not arm64. I've fixed that, and started pushing an image tagged with latest as well as the version number, so just pulling andrewjw/glowprom without the version number should work too.
Please have a try and let me know how you get on!
I'm afraid I haven't used Grafana Cloud so I'm not sure how you would send metrics there. I presume there must be a way to send metrics that aren't exposed to the internet, but you'll need to read their documentation I'm afraid. I just run prometheus and grafana locally.
Cheers,
Andrew
Thanks Andrew. That's great. I forgot to mention that I'm running my MQTT broker & Domoticz on a Pi3 which runs 24/7. Maybe it's time to use one of my Pi4s for this and Domoticz, as I've been keeping the Pi4 for media stuff so far.
If it's really easy a Pi3 version would be great but no idea how much work that involves.
Just remembered that my NAS is running arm64 so I've just tried on that and glowprom is working nicely and showing output on port 9100 in a browser.
Now to work out how to run Prometheus and Grafana as I've never used them before (except Grafana at work, 7 years ago, which someone else set up!) so a big learning curve!
Anyway, many thanks. Do you have a Grafana dashboard or other useful configuration tips that you could share please? Or anyone else?
Hi,
Version 0.8.0 should now be compatible with Raspberry Pi 3. I don't have one, so I've not been able to test it myself unfortunately.
There's also a Grafana dashboard in the grafana directory in the repository - https://github.com/andrewjw/glowprom. It's quite simple at the moment, so please submit any improvements if you make them :-)
Cheers,
Andrew