You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It appears the issue with special characters encoding in the Knowledge API might be influenced by several factors, including the indexing technique, the configuration of the embedding model, and how the model manager interacts with these configurations. Specifically, the indexing_technique set to 'high_quality' and the details of the embedding model (provider and model_type) are crucial areas to investigate. Errors in embedding model availability or provider token initialization could lead to encoding problems.
The Dify v0.6.5 codebase is designed to handle special characters like "ä" properly, as indicated by its use of UTF-8 encoding which supports these characters natively. This is further supported by the direct use of special characters in the source code, such as in the German translations. Therefore, the issue might not stem from the encoding of characters in the source code but rather from how these characters are processed or displayed in different parts of the application.
The headers set for JSON requests (Content-Type: application/json and Authorization) should not impact the encoding of special characters, as JSON inherently supports Unicode. Any encoding issues are more likely related to server-side handling or the JSON encoding process itself.
To better assist with resolving this issue, it would be beneficial to have more details about the specific conditions under which the encoding issue occurs, including the models, techniques, or parameters used. Additionally, any error messages or logs observed when the issue occurs could provide valuable insights into the underlying cause.
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
Self Checks
Dify version
Version 0.6.5
Cloud or Self Hosted
Cloud
Steps to reproduce
✔️ Expected Behavior
String captured in Knowledge dataset reflect String submitted via API
❌ Actual Behavior
Special chars are captured as other (unrelated) special chars
The text was updated successfully, but these errors were encountered: