Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dagster_azure: be able to import without fake impls #26733

Open
futurwasfree opened this issue Dec 27, 2024 · 2 comments
Open

dagster_azure: be able to import without fake impls #26733

futurwasfree opened this issue Dec 27, 2024 · 2 comments
Labels
integration: azure Related to dagster-azure type: bug Something isn't working

Comments

@futurwasfree
Copy link

What's the issue?

Currently when I import the following symbols in my production setup:

from dagster_azure.adls2.io_manager import adls2_pickle_io_manager
from dagster_azure.adls2.resources import adls2_resource

I'm also getting all the fake_adls2_resources imported, as it's described in adls2/__init__.py:

from dagster_azure.adls2.fake_adls2_resource import (
    FakeADLS2Resource as FakeADLS2Resource,
    FakeADLS2ServiceClient as FakeADLS2ServiceClient,
    fake_adls2_resource as fake_adls2_resource,

This easily adds 0.3-0.5 seconds to overall import/startup time.

Wondering if it's possible to exclude fakes from init.py or switch to Python namespace subpackages inside (with no init.py on this level)?

What did you expect to happen?

I expect only real/production implementations to be imported when executing these two import lines.

How to reproduce?

No response

Dagster version

dagster, version 1.9.4

Deployment type

Dagster Helm chart

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

@futurwasfree futurwasfree added the type: bug Something isn't working label Dec 27, 2024
@garethbrickman garethbrickman added the integration: azure Related to dagster-azure label Dec 27, 2024
@ion-elgreco
Copy link
Contributor

@futurwasfree you can make a pr for it, probably will be accepted

@futurwasfree
Copy link
Author

@ion-elgreco and I've just did here: #26754. thanks for looking into it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integration: azure Related to dagster-azure type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants