[BUG] reranker.py 加载模型时输入长度传入错误

[blizzardwj]

应该由1024改为max_length，否则会造成rerank模型predict长度过长的文本时出错。

[dosubot]

你好，@blizzardwj！很高兴遇到你，我是Dosu，一个可以帮助你解决bug、回答问题并成为贡献者的机器人。在人类维护者回复之前，我在这里为你提供帮助。我已经注意到了你提到的问题，将会很快给你一个完整的回答。

[web3aipro]

改了还是报错啊

[web3aipro]

报得错：

配置文件里面配置：

rerank文件得配置：

这么改了，还是报图1错误

[dosubot]

To resolve the issue with reranker.py and ensure it correctly handles texts longer than 1024 characters, you should modify the initialization of the CrossEncoder in the LangchainReranker class. Replace the hardcoded max_length=1024 with the max_length parameter that is passed to the constructor of LangchainReranker. Here's the corrected line of code:

self._model = CrossEncoder(model_name=model_name_or_path, max_length=max_length, device=device)

This adjustment allows the CrossEncoder to dynamically use the max_length value provided upon the LangchainReranker instance creation, accommodating texts of various lengths more flexibly.

Keep in mind, when setting the max_length, that transformer models have a maximum sequence length limit and processing very long sequences can be resource-intensive. This means choosing a max_length value requires balancing between the need to process longer texts and the available computational resources, as well as the potential impact on processing speed and memory usage.

Details

Sources

• server/reranker/reranker.py
• markdown_docs/server/reranker/reranker.md
• server/chat/knowledge_base_chat.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[309283175]

这个原因是reranker模型本身的限制，你这个模型最大支持514个长度，换个支持更长的reranker模型即可

[blizzardwj]

报得错： 配置文件里面配置： rerank文件得配置： 这么改了，还是报图1错误

RERANKER_MAX_LENGTH = 512，这是模型的输入限制，1024超出了。