How to debug and inspect raw API calls langchain makes

[bluskript]

NOTE: I only figured out a way for ChatOpenAI. Other LLMs probably have a similar structure, but read langchain's code to find what attribute needs to be overridden.

if you want to be able to see exactly what raw API requests langchain is making, use the following code below.

llm = ChatOpenAI(model="gpt-3.5-turbo-0613")

# class that wraps another class and logs all function calls being executed 
class Wrapper:
    def __init__(self, wrapped_class):
        self.wrapped_class = wrapped_class

    def __getattr__(self, attr):
        original_func = getattr(self.wrapped_class, attr)

        def wrapper(*args, **kwargs):
            print(f"Calling function: {attr}")
            print(f"Arguments: {args}, {kwargs}")
            result = original_func(*args, **kwargs)
            print(f"Response: {result}")
            return result

        return wrapper

# overwrite the private `client` attribute inside of the LLM that contains the API client with our wrapped class
llm.client = Wrapper(llm.client)

Ideally this should be a basic Langchain functionality, being able to see raw API responses are SO important sometimes. For example, I faced a KeyError: 'choices' error which was impossible to debug because langchain doesn't provide a way to view raw API responses.

[brandaobrandisborges]

Thanks for the contribution; This is really helpful;

[brendenpetersen]

@bluskript THANK YOU! You are a hero.

I have been at this for many hours. I've tried debug mode, callback functions, etc. Even LangChain traces do not provide all of this information. Traces include part of the raw API call in "invocation_parameters", including "tools" (and within that, "description" of the "parameters"), which is one of the main things I was trying to find. But the traces do not show the "messages" argument anywhere. It shows "prompts", but this is some LangChain-formatted construct.

P.S. This works on AzureChatOpenAI too.

[lucasalvarezlacasa]

Thank you for this contribution!

[sasharosca]

For ChatAnthropic, you need to do
llm._client.messages = Wrapper(llm._client.messages)