Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support downloading experiment data #1318

Open
oegeshipster opened this issue Dec 10, 2024 · 2 comments
Open

support downloading experiment data #1318

oegeshipster opened this issue Dec 10, 2024 · 2 comments

Comments

@oegeshipster
Copy link

Feature request

Although the langsmith experiment viewing interface is pretty good for basic usecases, for my (quite complicated) usecase I would like to build a custom offline experiment comparison tool (with e.g. streamlit or dash or similar). However I do not see a method in the sdk to download a given experiment run from the platform?

Motivation

In order to do error analysis on runs with complicated nested structured outputs from images, I need an interface that allows me to quickly compare specific metrics for each extracted field and e.g. highlight them in the source pdf, etc. I can build this myself, but need access to the underlying experiment result.

@oegeshipster
Copy link
Author

or I guess this might be covered by this PR? #1280

@hinthornw
Copy link
Collaborator

Agreed that a dedicated endpoint (and button in the UI) would be good.
The dataframe format is supported in the SDK (but it's really just a composition of endpoints rather than a dedicated endpoint) - that PR adds the abiltiy to export as json/other formats.

Semi-related discussion:
#537

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants