How to deploy langgraph using langserve

[luke1212]

I keep getting errors when I try to do this. Does anyone know how to do it?

[luke1212]

I find this answer on Reddit. Still want to know is there any documentation about this. https://www.reddit.com/r/LangChain/comments/1bvy2ws/how_to_deploy_langgraph_using_langserve/

[taochengla]

What exact errors did you get?
Both chain and graph are runnable, which implemented invoke(). I have no problem to use LangGraph in Lang serve. Here's hello world example:
`

Create graph

class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
llm = VertexAI(model_name='gemini-1.5-pro-001')
def chatbot(state: State):
return {"messages": [llm.invoke(state["messages"])]}
graph_builder.add_node("chatbot", chatbot)
graph_builder.set_entry_point("chatbot")
graph_builder.set_finish_point("chatbot")
mygraph = graph_builder.compile()
output = mygraph.invoke({"messages": ["Hello, how are you?"]})
print(output["messages"])

Call graph in langserve

add_routes(
app,
mygraph,
path="/chat",
)

Access API from client.

response = requests.post(
"http://localhost:8000/chat/invoke",
json={'input': {'messages': ["Hi, How are you"]}}
)

`

[lykuan]

from langgraph.graph import END, StateGraph
from graph.GraphState import GraphState
from graph.graphFlow import web_search,retrieve,grade_documents,generate,\
    transform_query,decide_to_generate,grade_generation_v_documents_and_question,route_question

workflow = StateGraph(GraphState)

# Define the nodes
workflow.add_node("web_search", web_search)  # web search
workflow.add_node("retrieve", retrieve)  # retrieve
workflow.add_node("grade_documents", grade_documents)  # grade documents
workflow.add_node("generate", generate)  # generatae
workflow.add_node("transform_query", transform_query)  # transform_query

# Build graph
workflow.set_conditional_entry_point(
    route_question,
    {
        "web_search": "web_search",
        "vectorstore": "retrieve",
    },
)
workflow.add_edge("web_search", "generate")
workflow.add_edge("retrieve", "grade_documents")
workflow.add_conditional_edges(
    "grade_documents",
    decide_to_generate,
    {
        "transform_query": "transform_query",
        "generate": "generate",
    },
)
workflow.add_edge("transform_query", "retrieve")
workflow.add_conditional_edges(
    "generate",
    grade_generation_v_documents_and_question,
    {
        "not supported": "generate",
        "useful": END,
        "not useful": "transform_query",
    },
)

# Compile
app = workflow.compile()
output = app.invoke({"question": "nodejs?"})
print(output["generation"])

add_routes(
    app,
    graph_app,
#    rag_chain,
    path="/openai",
    
)

this is my workflow and langserve, when i request, i got this error,do you know why?
this is my api request body
{
"input": {
"question": "nodejs?"
}

}
and then error reponse
{
"detail": [
{
"loc": [
"root",
"generation"
],
"msg": "field required",
"type": "value_error.missing"
},
{
"loc": [
"root",
"documents"
],
"msg": "field required",
"type": "value_error.missing"
}
]
}

[taochengla]

I am sorry. I am not certain about issue. I guess output = app.invoke({"question": "nodejs?"}) works Ok. If so, it is probably something wrong with FastAPI. The error message looks like from FastAPI as well. You may try to invoke a simple Langserve example, make sure it works in your environment.

[Arslan-Mehmood1]

@taochengla I'm having error when using checkpointer in graph.

| ValueError: Checkpointer requires one or more of the following 'configurable' keys: ['thread_id', 'thread_ts']

.

How to pass configuration with thread_id here to graph?

add_routes(
app,
(graph | RunnableLambda(output_parsing_for_playground)).with_types(input_type=InputChat, output_type=str),
playground_type="chat"
)

[JoshuaC215]

I built a template repo to help with deploying LangGraph agents to FastAPI, in case it's useful. You should be able to drop in your own agent and run it without too much trouble.

https://github.com/JoshuaC215/agent-service-toolkit

[antonioalegria]

@JoshuaC215 this is great - it totally makes sense to create a "langserve-langgraph" bridge anyone can import and setup. Will give your code a look, thanks!

[antonioalegria]

Ah, I see it does not use langserve. Well then, it would be also insteresting to have a langgraph-serve lib, but this code is already great! Kudos. I'll leave more feedback on that project.

[Aillian]

I think we need an open source Langgraph-serve lib at this point, langchain team stopped supporting Langserve to force us to use their paid Langgraph platform.

The current Langserve can work with Langgraph but not with advanced features like human in the loop and others.