TNT-LLM

[hinthornw]

https://arxiv.org/pdf/2403.12173.pdf

Haven't implemented the last steps (applying the taxonomy for labeling + training a log reg classifier ), but this is the more lang-graphy part.

If we want to include that, I think we could add as a second notebook perhaps since this is already sufficiently involved.

The other thing I may consider incorporating later is the steps to subdivide the data to add sub-categories. This would basically add two nodes:

1. Label documents according to the given clusters.
2. split state and recurse on this graph.

And then would make that conditional edge dependent on the depth the user configures (probably default to 0 or 1 levels of recursion)