LLM Spacy NER with custom dataset

[Project-NSE]

I am working on Named Entity Recognition task using large language model. I used SpaCy for that. I want to build a custom dataset. For testing purpose I created a 2 sentences dataset. The dataset is look like (name of the file ner_example1.yml)

- text: "take advil and reduce your fever"
  entities:
    Medicine:
      - advil.
   
- text: "I want to buy tylenol today"
  entities:
    Medicine:
      - tylenol.

Now I configure the config.cfg file as required

[nlp]
lang = "en"
pipeline = ["llm"]

[components]

[components.llm]
factory = "llm"

[components.llm.task]
@llm_tasks = "spacy.NER.v2"
labels = Medicine

[components.llm.task.examples]
@misc = "spacy.FewShotReader.v1"
path = "ner_examples1.yml"

[components.llm.model]
@llm_models = "spacy.Dolly.v1"
# For better performance, use dolly-v2-12b instead
name = "dolly-v2-3b"

The python script to run all those is

!python -m pip install spacy-llm
import spacy
from spacy_llm.util import assemble
nlp = assemble("config.cfg")

doc = nlp("Hi Doctor, give me a advil.")

print([(ent.text, ent.label_) for ent in doc.ents])

I did not get the proper output for the print statement. It return me an empty array.

It should return [('advil', 'Medicine')]

The output is as follows

[svlandeg]

Hi! I think this is simply because the Dolly 7b isn't all that powerful/accurate. When running this with the spacy.GPT-3-5.v1 model, the output is as expected, even without the few-shot learning examples:

[('advil', 'Medicine')]

So basically you'll need to either tune the prompt to see whether you can get Dolly to output good predictions, or use a larger/different model.