http request - need help getting data from heat pump web gui

[grericht]

Unfortunately, my heat pump cannot be easily integrated into home assistant. But it has a web interface. I read these in Home Assistant using the rest sensor. I have the idea of reading out the data with an esphome board and making it available to the home assistant.
I always retrieve the data in the same way. So far, each sensor individually. I think there would also be a way in Home Assistant to read out all the data regularly with a loop?
The code looks like this:

#configuration.yaml:

rest: !include rest.yaml

#rest.yaml:

   - resource: !secret wp_url_d
     username: !secret wp_usr
     password: !secret wp_pw
     authentication: basic
     scan_interval: 5
     sensor:
     sensor:
       - name: "rest_wp_D005"
         unique_id: "rest_wp_D005"
         json_attributes_path: "$.response.system"
         value_template: "{{ value_json['PCOWEB']['PCO']['DIGITAL']['VARIABLE'][4]['VALUE'] }}"
       - name: "rest_wp_D006"
         unique_id: "rest_wp_D006"
         json_attributes_path: "$.response.system"
         value_template: "{{ value_json['PCOWEB']['PCO']['DIGITAL']['VARIABLE'][5]['VALUE'] }}"

#in esp home I tried with the following code:

http_request:
   id: wp_request
   url: !secret wp_url_d
   method: GET
   username: !secret wp_usr
   password: !secret wp_pw
   authentication: basic

sensor:
   - platform: template
     name: "wp_d005"
     id: wp_d005
     value_template: "{{ state_attr('wp_request', 'system')['PCOWEB']['PCO']['DIGITAL']['VARIABLE'][4]['VALUE'] }}"

#or:

http_request:
   id: wp_request
   url: !secret wp_url_d
   method: GET
   username: !secret wp_usr
   password: !secret wp_pw
   authentication: basic

sensor:
   - platform: template
     name: "wp_d005"
     id: wp_d005

interval:
   - interval: 10s
     then:
       - http_request.get: wp_request
         on_response:
           then:
             - lambda: |-
                 float value = json::parse(id(wp_request).response)["PCOWEB"]["PCO"]["DIGITAL"]["VARIABLE"][4]["VALUE"];
                 id(wp_d005).publish_state(value);

Both result in many errors when validating and do not work. But in the documentation I don't really understand how things should be implemented.

[ssieb]

Did you look at the docs for the http request? What you have there doesn't match at all. The second one is slightly better than the first one because it doesn't try using value_template. Where did you find that in the esphome docs?

[grericht]

Thanks for your answer. I tried reading through the docs. But I can't find my example there. It took me quite a long time to get the sensors accessed in home assistant. Now I thought it would be easier to access these sensors on an ESP32.

I have now discovered that http_request does not support any authentication? But the website needs something to retrieve the data. does that mean I can't use esphome?

[ssieb]

There's a cookbook example for how to do all of what you're trying to do:
https://esphome.io/cookbook/http_request_sensor.html
Basic authentication example:
esphome/feature-requests#709

[grericht]

thank u. http request now works:
HTTP Request completed; URL: ....; Code: 200; Duration: 289 ms

but i dont get the json_parse running.
response looks like:
<PCOWEB> <PCO> <DIGITAL> <VARIABLE> <INDEX>1</INDEX> <VALUE>0</VALUE> </VARIABLE> <VARIABLE> <INDEX>2</INDEX> <VALUE>0</VALUE> </VARIABLE> <VARIABLE> <INDEX>3</INDEX> <VALUE>0</VALUE> </VARIABLE> <VARIABLE> <INDEX>4</INDEX> <VALUE>0</VALUE> </VARIABLE> ... </DIGITAL> </PCO> </PCOWEB>

chatgpt says code like:
on_response: then: - lambda: |- auto response_data = id(wp_request).get_data(); DynamicJsonDocument doc(1024); deserializeJson(doc, response_data.c_str()); float value = doc["PCOWEB"]["PCO"]["DIGITAL"]["VARIABLE"][4]["VALUE"].as<float>(); id(wp_d005).publish_state(value);

validating works but no install.
error: 'class esphome::http_request::HttpRequestComponent' has no member named 'get_data'
auto response_data = id(wp_request).get_data();

best was:
on_response: then: - lambda: |- json::parse_json(id(wp_request).get_data(), [](JsonObject root) { float value = root["PCOWEB"]["PCO"]["DIGITAL"]["VARIABLE"][4]["VALUE"].as<float>(); id(wp_d005).publish_state(value); });

but wp_d005 was na. and logger says: JSON parse error: InvalidInput

[grericht]

now i found:
Content-Type: text/xml

i cant use json parsing?

[ssieb]

Right, the data is xml, not json.

[grericht]

and is there a way to parse the Response?

[ssieb]

esphome doesn't have any xml handling, but you could probably do it with c++ string methods to search for and extract the piece you want.

[grericht]

But for this i have to save the hole Response First as a variable? I dont find a solution for doing this.

I found a solution on arduino to include an XML library. This will nit Work Here?

[ssieb]

This is C++. Just put the response in a local variable. You were already getting it for the json parsing. You really should come ask on discord. It would be a lot easier.

[grericht]

i did a try with discord: https://discord.com/channels/429907082951524364/1215964980399312926

[ssieb]

You asked at a time when very few people are around and then you didn't wait...

[grericht]

ok. was my first try on discord. i now answered again there and will be online again.
what is a goot time?