I have managed to connect to the localhost so it seems the Mosquitto service is working.
My servers are 23 and 64. I can’t connect though.
I am obviously doing something wrong:
I got this certificate from @plonkster 's Github page:
Any reason why you would want to connect to the VRM mqtt broker? I suspect that you would be better of connecting to the local broker on your Venus Device.
Keep in mind that MQTT Explorer is connecting to a Broker like Mosquitto so you would not need a broker running on Local host for the explorer to work.
I am in another country to my Venus devices.
As for the reason to connect at all, I am just curious as to what I’ll see and what possibilities are available.
If I was to deploy some future system I suppose it would be locally based, but for the time being all I want to do is look under the hood.
OK, got connected.
What a mission!
This is what I did.
For other users’ references:
See “Edit” at the bottom of this post about Mosquitto installation.
Google Mosquitto.
Download Mosquitto from Mosquitto.org & install.
Download | Eclipse Mosquitto
Start the Mosquitto service by navigating to Mosquitto.exe and clicking on.
Downloaded MQTT Explorer from web & installed.
Downloaded server certificate from Github (Hard to find)
A file called: venus-ca.crt - save it somewhere on your PC, you need it later.
(To download from GITHUB, use the green code button and download .zip file, extract this to find the certificate file).
Next, find your own VRM portal ID on your VRM:
You have to know what your VRM MQTT victron server number is:
Go to this site: GDB online Debugger | Code, Compile, Run, Debug online C, C++
Press the “Fork this” button or you won’t be able to enter anything.
Replace the word: PORTAL_ID_HERE with your actual VRM portal ID.
KEEP THE INVERTED COMMAS around your actual VRM portal ID.
Press the “Run” button.
It will produce a number.
That number now replaces “XX” in the following: mqttXX.victronenergy.com
Right so your Mosquitto service is running start MQTT explorer:
And edit the connection like so:
Note the Port number is: 8883 not the default.
Now click the “Advanced” button which takes you to another window with a certificates button. Click that and upload that Github certificate file like so:
Now you should be able to connect.
And see something happening, don’t ask me what I haven’t got that far yet.
EDIT: I don’t know if the initial installation of Mosquitto is necessary because I didn’t need to restart the service to connect when I rebooted my PC.
1 Like
You definitely don’t need to install mosquitto into order to connect to a remote mqtt service.
1 Like
There you go…
OK, what I think I know, so far.
(In my layman’s terms, because software jargon confuses the hell out of me and I am surely not the only one).
This Mosquito server collects relevant data points from the Victron system. These points are pre-defined. The Mosquitto server can in turn resend the data to another service. The other software service of choice is one called NODE-RED.
NODE-RED works on a palette system. (The palettes being a defined set of functional descriptions). Victron it seems has already developed a palette that interprets the data that was passed on via Mosquitto from the Venus device.
Other 3rd party palettes are available for installation in NODE-RED. These can be used in combination with the Venus data to control the charging behaviour and the output relay for example.
One of the palettes available is a modbus palette. So that many other sensors could be read and many output relays could be controlled (in combination with for example the Venus SOC or the weather or something else).
This part seems to have a steep learning curve.
There is also a dashboard palette where you could for example read more meter options than the standard Venus package offers.
This part may have a less steep learning curve.
But wait there’s more…
The Venus OS large comes with the Mosquitto service and NODE-RED pre-installed with the Victron palette.
So this can be done with a Raspberry Pi at a local level.
Ok, my thoughts so far…
The Node-Red functionality has its place as far custom controls of the Venus device.
If you want a different SOC based on tomorrow’s weather forecast this seems the environment to do it.
Replicating the already excellent VRM display is a wasted effort, but I think it can make a graphical display of custom modbus devices easier, where a human can keep an eye on things. It would be great for a set of Modbus energy meters at custom locations, with a nice webpage interface.
I don’t know about data logging and history, but it is also probably good for that.
But…
As far as it being the environment to deploy complex IO automation logic around the farm, I think I prefer a PLC.
A PLC just seems to have a more robust track history. I just couldn’t trust this stuff to an SD card, and just too many software handshakings.
What I don’t like: Venus talks to Mosquitto which talks to Node-RED which relies on palettes that can be made by a host of different contributors. It just seems like too many places for a firmware upgrade to forget about a backslash. And then you’d be hopelessly lost.
I also understand Venus device is also a Modbus gateway and then it just becomes like any other Modbus device. Just read the register of the Node ID you want to know, using ladder logic. Modbus protocol is robust, an industry standard and has been around since the 1970’s.
Ladder logic within a PLC largely consists of soft Normally open/normally closed contacts.
(Something that someone familiar with electrical control circuits should pick up fairly easily).
Get the system working and you don’t have to worry about any of a myriad of software version updates throwing a delicate automation system a wobbler in the future.
So in conclusion, in my opinion so far NODE-RED in Venus offers custom controls and lovely visualisations, within your solar setup. It’s a “nice to have”.
However, if you want wider business automation control extract the limited data you need from the Venus device (which may only be the state of charge, a tank level, generator status etc) and use it with Modbus TCP in a PLC environment.
Not quite, but close. Mosquitto is just a “broker”. The particular protocol it brokers is called MQTT, which is a light-weight protocol for IoT data.
Mosquitto isn’t even the only one. There is HiveMQ as well, several cloud providers have their own (Amazon, Azure), and other pubsub systems like RabbitMQ has MQTT plugins as well.
The broker is a middle man, it sits there and it relays messages between clients. You connect to a broker (aka MQTT server), and then you tell it what kind of data you are interested in by “subscribing” to the topic you are interested in. It is what you call a pub/sub architecture. When you have data to send, you publish it to the relevant topic, and then people subscribed to those topics are notified.
The component that publishes the data from your GX device to the broker is called dbus-mqtt.
On a Victron system, the topics are structured like this, for example: N/badbeef15b4d/system/0/Serial
The topic is a key, and it has an associated value.
To keep traffic down, it doesn’t publish everything. You need to publish a keepalive to tell it what you want. See here.
The first letter of the topic is either R, W or N. R is a read request. If you publish a read request, you will get a notification (starting with N) for that value shortly afterwards. You can also write values (for paths that are writable) with the W topic.
But to begin, you are probably interested only in the notifications. After the N follows the VRM id of the GX device. This is so you can bridge together several installations into one cloud server, and still be able to identify each.
The third component is the service type. This would be something like settings, system, solarcharger, grid, pvinverter, vebus, etc. Without getting into too much detail, settings is where your configuration settings live, system is a service that summarises details from the others, and the others are the individual services each with their information.
The fourth component is the DeviceInstance. That is a unique number Venus uses to identify a service within its group, ie it helps distinguish between two solarchargers.
What follows after that is the “dbus path”, which is the IPC mechanism used by venus and most other Linux versions. It should usually be mostly self-explanatory. But if in doubt, check the modbus XLS document, or the dbus wiki.
Now, node-red. That’s a client. It connects to the MQTT broker, and then you can use that to connect those topic to actions, using some nice graphical interface. And Node-Red can also publish back to MQTT.
You can also bridge brokers. You can tell it to relay some topics (ie copy back and forth) to another server. So for example, if I am running a HomeAssistant server with its own MQTT server (which incidentally is also mosquitto), then I can bridge certain topics from my Venus broker to my HA broker, so those two can talk to each other, but without copying all the traffic.
1 Like
Is the short version “This is all already done as part of Venus OS large”?
OK, slightly shorter summary. A normal Venus installation has:
- its own mqtt client (for publishing information to the broker), this part is called dbus-mqtt.
- It’s own built-in MQTT broker, which is an instance of mosquitto.
- A bridge configured to bridge the relevant information to the cloud brokers, where VRM can get at it.
The large image adds, in addition to this, node-red (and signal-K) to the mix. It is hosted on the GX device itself, so you don’t have to run it separately on another computer.
The work of actually automating something with it (aka set up “flows”) remains exactly the same.
1 Like
OK, another quickie, regarding the node-red package installed with Venus.
Am I correct?
Third party palettes are installable that together with the Venus palette could be used to control Victron behaviour.
How is this dealt with when a firmware update is deployed/done?
You have just ventured outside my area of expertise. I have never used Node-RED 
1 Like
If it’s outside your area of expertise, what chance do we mere mortals have?
Hehe I have a JIT (just in time) policy regarding new stuff. I’ll figure it out when I need to… sorry man.
1 Like
Unless there is some major drivers involved there isn’t much that can go wrong.
Almost all custom nodes is just .js functions that is “neatly” wrapped up. You can even create your own custom node via the sub flow function.
Remember that Node Red is just a graphical interface for Java Script , that is very well supported and has quite a small foot print. In fact I have been playing with running Node red on a Android device and so far it works very well.
Personally I just dont like the WiFi connection and prefer a hard wire connection to the server.
Oh and it should also be noted that it is a fair amount cheaper than PLC’c 
1 Like
Excellent post. As a beginner, I followed your instructions with very few modifications successfully. Here is what I did with VRM already running. Note that I did everything over the internet:
- Installed Venus OS Large on my Cerbo GX through VRM remote console per Victron manual here Venus OS Large image: Signal K and Node-RED [Victron Energy]
2.Activated Node Red on Cerbo through remote console. Mosquito is automatically included and activated on Venus Large; no need to install it separately. - Got my VRM ID as per your instructions.
- Identified my Victron server, using mostly your instructions. I tested your python script as you describe and it works. I used a separate but very similar script that gives the full path (mqttXX.victronenergy.com) but this is not necessary.
- Got the Victron certificate using your Github link. It’s true it is hard to find. It’s included in a directory called “dbus-mttq-master”. See screenshot.
- Downloaded and installed MQTT Explorer on my Windows 10 tablet.
- Connected successfully as per your instructions.
I must say that @plonkster comment was also very useful especially the part explaining the structure of Victron topics/data paths.
