Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JS SDK traceable is not compatible with langchain JS chain.invoke tracing #598

Closed
mattiaseyram opened this issue Apr 12, 2024 · 5 comments · Fixed by #678
Closed

JS SDK traceable is not compatible with langchain JS chain.invoke tracing #598

mattiaseyram opened this issue Apr 12, 2024 · 5 comments · Fixed by #678

Comments

@mattiaseyram
Copy link

mattiaseyram commented Apr 12, 2024 •
edited

Feature request

Any chains invoked inside a function wrapped with traceable have separate trace ids (or vice versa, with chains calling traceable functions internally).

Ideally the trace id gets propagated automatically.

Related to langchain-ai/langchainjs#5083

Motivation

Increase compatibility with Langchain JS native tracing.

Example

image 
import { traceable } from 'langsmith/traceable';
import { RunnableLambda } from '@langchain/core/runnables';

const exampleRunnable = RunnableLambda.from(async (foo: string) =>
  foo.toUpperCase(),
);

exampleRunnable.name = 'exampleRunnable';

const exampleTraceable = traceable(
  async (foo: string) => {
    const result = await exampleRunnable.invoke(foo);

    console.log(result);

    return result;
  },
  {
    name: 'exampleTraceable',
  },
);

exampleTraceable('bar');
@mattiaseyram mattiaseyram mentioned this issue Apr 12, 2024
Open
5 tasks
@hinthornw
Copy link
Collaborator

Ya we haven't created an automatic handoff yet. Thanks for flagging.

The way around it is to conver tthe run tree into a callback manager

@mattiaseyram
Copy link
Author

Awesome, thanks for the reply. What's the best way to convert the RunTree into a callback manager? I'm mid implementation atm.

@hinthornw
Copy link
Collaborator

hinthornw commented Apr 12, 2024 •
edited

I'm checking with the team, but we do something similar here: https://github.com/langchain-ai/langchainjs/blob/b9f63251fc7d6b2ab5b20189bc5dca2150413b49/langchain/src/smith/runner_utils.ts#L223

We do an auto-handoff in python but I don't believe we have addedit yet in js.

Another option is to use RunnableLambda instead of traceable since then all the tracing happens within langchain code

@mattiaseyram
Copy link
Author

Thanks, ended up using a class similar to RunnableLambda to meet my needs as seen here langchain-ai/langchainjs#5083

@loui7
Copy link

loui7 commented Apr 25, 2024

Same issue with RunTree's, using chain.invoke inside creates a new trace

@dqbd dqbd mentioned this issue May 13, 2024
Merged
5 tasks
@dqbd dqbd closed this as completed in #678 May 17, 2024
dqbd added a commit that referenced this issue May 17, 2024
@dqbd
fix(js): LangChain / traceable handoff (#678)
1f665b4 
- Traceable to LangChain: add the entire tree of RunTrees into the
BaseTracer `runMap`, assign the `_parentRunId` to the callback manager
- LangChain to Traceable: the `RunnableTraceable` invokes the function
without calling `*WithConfig`, avoiding unwanted callback invocation


**Verifying package manager behaviour**

Exposed the LangChain utilities + added `@langchain/core` as an optional
dependency to allow supporting older versions of LangChain.

- [x] Yarn 1.x: does not install the `@langchain/core`, not included in
package-lock.json
- [X] PNPM: does not install `@langchain/core`, included in
package-lock.json
- [X] NPM: does not install `@langchain/core`, included in
package-lock.json
- [X] Bun: does not install `@langchain/core`, included in
package-lock.json
- [X] Deno: does not install `@langchain/core`

Upstream PR which implements similar functionality can be seen in
langchain-ai/langchainjs#5339

Closes #598
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(js): LangChain / traceable handoff langchain-ai/langsmith-sdk
3 participants
@mattiaseyram @hinthornw @loui7