Add Modbus

[b0661]

OpenDTU is extended by a Modbus server.

The Modbus server serves TCP at port 502.

• At Modbus ID 1 the server mimicks the Modbus registers in the original DTUPro.
• At Modbus ID 125 the server serves a SunSpec compatible "total inverter" that
  provides the OpenDTU aggregated data from all registered inverters.
• At Modbus ID 243 the server serves a SunSpec meter that provides AC power and AC yield as if measuring all registered inverters. Same approach as with the "total inverter" but now disguised as a meter.

The webapp is extended by Modbus configuration and info views.

Modbus can be enabled/ disabled. Only a minimal subset of DTUPro/ OpenDTU aggregated data
is currently supported.

Main intention was to have an easy feed to my HomeAssistant installation that already monitors
and controls my other inverters by Modbus.

modbus:
  ##########################################
  # OpenDTU Hub
  # slave: 1   - Hoymiles DTUPro
  # slave: 125 - OpenDTU Total Inverter (SunSpec)
  - type: tcp
    name: opendtu
    delay: 5
    timeout: 10
    host: 192.168.178.xxx
    port: 502

    sensors:
      - name: "OpenDTU#Total AC Power"
        unique_id: "opendtu_total_ac_power"
        slave: 125
        address: 40192
        input_type: holding
        data_type: float32
        unit_of_measurement: W
        device_class: power
        state_class: measurement
        scan_interval: 10
      - name: "OpenDTU#Total AC Yield Total"
        unique_id: "opendtu_total_ac_yield_total"
        slave: 125
        address: 40202
        input_type: holding
        data_type: float32
        unit_of_measurement: Wh
        device_class: energy
        state_class: total
        scan_interval: 20
      - name: "OpenDTU#Total DC Power"
        unique_id: "opendtu_total_dc_power"
        slave: 125
        address: 40208
        input_type: holding
        data_type: float32
        unit_of_measurement: W
        device_class: power
        state_class: measurement
        scan_interval: 10

The OpenDTU Modbus sources were inspired by : https://github.com/ArekKubacki/OpenDTU.
See #582 for the orignal pull request.

The Modbus library used for Modbus communication is: https://github.com/eModbus/eModbus.
Documentation for the library is here: https://emodbus.github.io/.
The library was choosen over full featured libraries to achieve a lower memory footprint.

Memory footprint:

text     data     bss   dec      hex    filename
1160211  532424   42185 1734820  1a78a4 .pio/build/generic_esp32/firmware.elf (Modbus)
1140503  524812   41897 1707212  1a0ccc .pio/build/generic_esp32/firmware.elf

[phol]

Very nice work! This would absolutely be something I could use.

Do you think that, expanding on this commit, making the same modbus data also available through modbus RTU would be possible?

OpenDTU Fusion already includes a Renesas ISL3178E TTL to RS485 transceiver, meaning the hardware is already prepared for this. It's just not used yet at the moment.

EDIT:
A suggestion I just thought of is, perhaps it would be useful to change variable and function names to MODBUS_TCP? That way, a possible future implementation of MODBUS_RTU is not going to cause conflicts with this functionality.

[b0661]

Very nice work!

Thank you.

Do you think that, expanding on this commit, making the same modbus data also available through modbus RTU would be possible?

eModbus supports RTU also, so it should be possible. My Fusionv2 board is used to monitor my system, so I would be a little bit reluctant to use it for testing Modbus RTU. I will have a look into RTU. Maybe there is a way to keep memory footprint low in case one or both servers are not used.

EDIT: A suggestion I just thought of is, perhaps it would be useful to change variable and function names to MODBUS_TCP? That way, a possible future implementation of MODBUS_RTU is not going to cause conflicts with this functionality.

Good suggestion, will work on that.

[ArekKubacki]

I'm very glad that I was an inspiration :) And I'm even more glad that someone took proper care of a problem that I personally didn't have time to deal with.

[ArekKubacki]

Hello,
I compiled the program from your project. Unfortunately, I don't have any tabs for Modbus. Something needs to be defined in the code?

[b0661]

Hello, I compiled the program from your project. Unfortunately, I don't have any tabs for Modbus. Something needs to be defined in the code?

Edit:

You have to build the webapp also, before building the firmware.

Added app.js.gz to the pull request. Now you can build the firmware without building the webapp before.

[b0661]

Added modbus server for minimal SunSpec meter. Only provides AC power and AC yield as if measuring the output of all registered inverters. Same approach as for the "Total Inverter" but now disguised as a meter.

To be used e.g. as a "Fronius Smart Meter TCP" as orginally done in https://github.com/AloisKlingler/OpenDTU-FroniusSM-MB.

[ArekKubacki]

Hello,
Some quick notes I found :)

1. If the inverter is not reachable, the reading should behave as selected in the settings/inverter option in openDTU, i.e. either show the old result or 0.
2. If the inverter is unreachable, Modbus communication does not work on these addresses and throws errors. Maybe it's worth just entering 0 and 1 in the Link Status field?
3. In the original DTU, if we enter the field of an inverter that is not there, it has the address 00000000000. Thanks to this, many libraries check whether all inverters have already been searched. I think you can do something similar here.

And now the strangest thing when it comes to reading from a real DTU. Generally, real DTU has 8-bit registers, not 16-bit registers like real Modbus... The question is whether it would be worth adding a selection option. Especially if openDTU was to be compatible with various DTU libraries.

[b0661]

Hello, Some quick notes I found :)

Thx for testing and advise.

1. If the inverter is not reachable, the reading should behave as selected in the settings/inverter option in openDTU, i.e. either show the old result or 0.

Changed now - just takes the values provided by OpenDTU. Does not check for not reachable anymore.

2. If the inverter is unreachable, Modbus communication does not work on these addresses and throws errors. Maybe it's worth just entering 0 and 1 in the Link Status field?

Added an alarm code in case the inverter is unreachable. Can you advise on the link status? I though it is the DTU communication status (not the inverter communication status).

3. In the original DTU, if we enter the field of an inverter that is not there, it has the address 00000000000. Thanks to this, many libraries check whether all inverters have already been searched. I think you can do something similar here.

Changed - Now 0 is provided for all inverter data list registers that are supported by the DTU if they are unused.

And now the strangest thing when it comes to reading from a real DTU. Generally, real DTU has 8-bit registers, not 16-bit registers like real Modbus... The question is whether it would be worth adding a selection option. Especially if openDTU was to be compatible with various DTU libraries.

Changed - adapted to your original code. It is very strange that the start address for the data (0x1000)/ SN (0x2000) list in the documentation seems to be a "normal" Modbus address, but the following fields are per byte.

I tested with your https://github.com/ArekKubacki/Hoymiles-Plant-DTU-Pro app for HomeAssistant. At least the application starts. This was tested without inverters.

By the way during testing I triggered several Python exceptions due to configuration problems. Especially if the number of panels is > 0 and there are no inverters, the app won't start. Fault indication can only be found in the system log and it is somehow misleading as it is an index out of bounds fault.

I don't own a Hoymiles DTU, so your help is very much appreciated.

[ArekKubacki]

Hi,
Unfortunately, I don't have time to work on it today. But looking at it quickly, something seems to have been moved. I'll look for the problem next week.

[HMSeriesMicroinverterData(data_type=0, serial_number='00000000003c', port_number=17, pv_voltage=Decimal('2494.7'), pv_current=Decimal('165.3'), grid_voltage=Decimal('409.7'), grid_frequency=Decimal('3.96'), pv_power=Decimal('20.1'), today_production=2325, total_production=327551956, temperature=Decimal('0'), operating_status=1384, alarm_code=58, alarm_count=12431, link_status=0, reserved=[107, 0, 3, 0, 0, 0, 0]),

[b0661]

Hello @ArekKubacki ,

I did some more tests using synthetic test data (fixed values for 1 inverter with 6 channels) for the DTUPro data. The test data can be activated in platformio_override.ini by -DOPENDTU_SIM_DTUPRO:

[env:generic_esp32]
build_flags =
    -DPIOENV=\"$PIOENV\"
    -D_TASK_STD_FUNCTION=1
    -D_TASK_THREAD_SAFE=1
    -Wall -Wextra -Wunused -Wmisleading-indentation -Wduplicated-cond -Wlogical-op -Wnull-dereference
    -std=c++17
    -std=gnu++17
    -DLOG_LEVEL=LOG_LEVEL_DEBUG
    -DOPENDTU_SIM_DTUPRO

I used this HomeAssistant configuration.yaml

sensor:
  ### Test ###
  - platform: hoymiles_dtu
    host: 192.168.178.xxx
    name: Hoymiles PV
    dtu_type: 0
    monitored_conditions:
      - "pv_power"
      - "today_production"
      - "total_production"
      - "alarm_flag"
    monitored_conditions_pv:
      - "pv_power"
      - "today_production"
      - "total_production"
      - "pv_voltage"
      - "pv_current"
      - "grid_voltage"
      - "temperature"
      - "operating_status"
      - "alarm_code"
      - "alarm_count"
      - "link_status"
    panels: 3

As a result HomeAssistant shows the following. Test data seems to be correctly displayed.

I this the expected output?

[ArekKubacki]

Unfortunately not yet.
HMSeriesMicroinverterData(data_type=60, serial_number='xxx', port_number=1, pv_voltage=Decimal('37.1'), pv_current=Decimal('2.71'), grid_voltage=Decimal('232.2'), grid_frequency=Decimal('50.03') ), pv_power=Decimal('384.1'), today_production=5356, total_production=3837759

today_production, total_production, pv_power, pv_current, pv_voltage should be for each port (string, panel), not for the entire inverter.

port_number should change depending on the number of ports on the inverter. In my case 1-4. It is currently growing to 24, which is the maximum panels I have.

[b0661]

Thank youu for testing. Will look into this. I hopefully can now understand your original code.

[b0661]

Hello @ArekKubacki

today_production, total_production, pv_power, pv_current, pv_voltage should be for each port (string, panel), not for the entire inverter.

Now switched to values per string.

port_number should change depending on the number of ports on the inverter. In my case 1-4. It is currently growing to 24, which is the maximum panels I have.

Reported port numbers adapted to per inverter.

Would be nice if you could test this version. I tried to make a one to one copy of your code.

What is still open to me is the first byte (data type) at 0x1000. According to the specification it shall be 0x3C, you are using 0x0C. Do you know more? What does the DTUPro use here?

Thanks again for testing.

[ArekKubacki]

Looks very good. As far as register 1 is concerned, it does not matter. According to the specifications, it is 3C in the original DTU, at least in my case it is 0C. It is not used for anything so I think you can leave it as per specifications.

Have you thought about making 2 choices? How is "DTU Clone" and how it should be according to the specification with normal 16-bit registers? This will give users more space to use.