[Request] Compatibility of textual inversion between SD 1.5 and SD 2.1

[suzukimain]

Is your feature request related to a problem? Please describe.
Textual inversions trained on different versions of SD, such as SD 1.5 and SD 2.1, are not compatible.
Related to previously submitted issue #4030

Describe the solution you'd like.
To make textual inversion, which is compatible only with SD 1.5, also compatible with SD 2.1

Describe alternatives you've considered.
I was trying to replace this part with a process that resizes to the expected shape.

Additional context.
Less important, error messages are not displayed correctly.

import torch
from diffusers import StableDiffusionPipeline
from huggingface_hub import hf_hub_download

pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16)

path = hf_hub_download(repo_id="gsdf/EasyNegative", filename="EasyNegative.safetensors", repo_type="dataset")

pipe.load_textual_inversion(path,token="EasyNegative")

# ValueError: Loaded embeddings are of incorrect shape. Expected each textual inversion embedding to be of shape {input_embeddings.shape[-1]}, but are {embeddings.shape[-1]}

cc: @sayakpaul

[sayakpaul]

Currently, my plate is a bit full. So, opening it up for the community. Would be more than happy to help in reviews and tests.