[torch.compile] Large cache size limit

[anijain2305]

Describe the bug

sglang/python/sglang/srt/model_executor/cuda_graph_runner.py

Lines 103 to 105 in 2125898

	torch._dynamo.config.accumulated_cache_size_limit = 1024
	if hasattr(torch._dynamo.config, "cache_size_limit"):
	torch._dynamo.config.cache_size_limit = 1024

From torch 2.5 version, we should not need such a large cache size limit. Is it possible for someone to double check and remove the override?

Reproduction

NA

Environment

NA

[zhyncs]

It was added at #2069 As I remember, we should set, otherwise something with FlashInfer will fail

[zhyncs]

We may test whether reducing the size is also compatible (not directly deleting).

[anijain2305]

I see. Maybe a better way is to make FlashInfer kernels torch.compile compatible?