Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override a fixture on a test module level:with "name" key_word_arg, its may not Override #12952

Open
steadyfirmness opened this issue Nov 11, 2024 · 9 comments · May be fixed by #13058
Open
Labels
topic: fixtures anything involving fixtures directly or indirectly type: bug problem that needs to be addressed

Comments

@steadyfirmness
Copy link

steadyfirmness commented Nov 11, 2024

description

I have some fixtures in the same test file, and I need to reload one of them as following:

@pytest.fixture()
def f1():
    return 1

@pytest.fixture(name="f1")
def f0():
    return 0

what i want

When i call “f1” its return 0 because its actually executed “f0” as following:

class TestLib:
    def test_1(self, f1):
        assert f1 == 0

pytest result

But this is fail for test

________________________________ TestLib.test_1 ________________________________

self = <lib.test.test_lib.TestLib object at 0x104b3df40>, f1 = 1

    def test_1(self, f1):
>       assert f1 == 0
E       assert 1 == 0

lib/test/test_lib.py:21: AssertionError
=========================== short test summary info ============================
FAILED lib/test/test_lib.py::TestLib::test_1 - assert 1 == 0

env

os:mac 15.0.1
python:3.12.4
pytest:8.3.2

more detail

When I change name ‘f0’ -> 'f2', as following, it works well:

@pytest.fixture()
def f1():
    return 1


@pytest.fixture(name="f1")
def f2():
    return 2

So I summarized that the overloading of fixtures in the same file is not in the order of bytecode compilation, but in the lexicographic order of functions, resulting in the phenomenon where f0 cannot reload f1 while f2 can reload f1.

I'm not sure if this is a feature or a bug

@RonnyPfannschmidt
Copy link
Member

We should error on such conflicts

@Zac-HD Zac-HD added type: bug problem that needs to be addressed topic: fixtures anything involving fixtures directly or indirectly labels Nov 30, 2024
@bluetech
Copy link
Member

We should error on such conflicts

I'd note that doing this might conceivably make sense if the one requests the other, e.g.:

import pytest

@pytest.fixture(name="fix")
def fix1():
    print('fix1')

@pytest.fixture(name="fix")
def fix2(fix):
    print('fix2')

def test(fix): pass

I'm not necessarily saying we shouldn't make this an error, but it's something to consider.
We can also only raise an error only if the fixture doesn't transitively request itself, but that might be difficult.

So I summarized that the overloading of fixtures in the same file is not in the order of bytecode compilation, but in the lexicographic order of functions, resulting in the phenomenon where f0 cannot reload f1 while f2 can reload f1.

Correct, this is because we use dir (in parsefactories) which sorts. I actually think this sorting is not good, and we'll be better without it. We should use logic similar to what we have in PyCollector.collect().

@dongfangtianyu
Copy link
Contributor

At first glance, it might seem meaningful, but it feels counterintuitive.

In scenarios where the name fix must be retained, and two different implementations (possibly with different scopes or other distinctions) need to coexist, the more common approach looks like this:

import pytest

@pytest.fixture()
def fix1():
    print('fix1')

@pytest.fixture()
def fix2():
    print('fix2')

@pytest.fixture()
def fix(fix1, fix2):
    pass

def test(fix): 
    pass

On the other hand, the following implementation seems overly contrived (especially when both fixtures are in the same file):

@pytest.fixture(name="fix")
def fix1():
    ...

@pytest.fixture(name="fix")
def fix2(fix):
    ...

According to pytest's current implementation, the unique advantage of this approach is that it creates a fixture that cannot be directly request by any test, It can only be request by another fixture with the same name but declared later.


Just my thoughts:

  • This behavior is unexpected.
  • If simple Python code can achieve the goal, there is no need for it to be redundantly implemented with complex pytest rules.

@bluetech
Copy link
Member

This behavior is unexpected.

I definitely agree it feels odd in isolation. However, if we consider that pytest supports to concept of overriding fixtures, the question becomes, should we disallow overriding within a single file (module)?

My feeling on this is that we shouldn't disallow it, mostly because I dislike special cases and I like orthogonality. My feeling would be different if this were a common issue causing confusion, but at least for me, this is the first time I see it come up. And actually the original reporter (@steadyfirmness) seems to dislike only the sorting behavior, not that it's allowed.

WDYT?


BTW @dongfangtianyu, if you feel like it, a PR to remove the sorting behavior of parsefactories (replace dir with code like in PyCollector) would be great, regardless of the outcome of the discussion above.

@RonnyPfannschmidt
Copy link
Member

Same name in Same file should be conflict unless someone demonstrates a valid use case that's not a thedaylywtf submission or esoteric code golfing

@bluetech
Copy link
Member

Well, there isn't really a reason for anyone to write this Python file:

def func():
    pass

def func():
    pass

yet I still don't think Python should disallow it. The downsides of disallowing is added complexity, and preventing any creative usage of it. The upside is to prevent mistakes, but I don't think it's a common mistake.

@steadyfirmness
Copy link
Author

steadyfirmness commented Dec 17, 2024

pytest supports to concept of overriding fixtures

Yes, as he(@bluetech ) said, treating test code as a project and solving scalability issues by adding code is the best approach, as shown below.It can better reflect the process of operation:

import pytest


@pytest.fixture(name="my_list")
def init_list_a():
    print("init_list_a here")
    return ["a"]


@pytest.fixture(name="my_list")
def init_list_b():
    print("init_list_b here")
    return ["b"]


@pytest.fixture(name="my_list")
def add_something(my_list):
    print("add_something here")
    my_list.append("c")
    return my_list

"""
In my intuition,who is "my_list"?:
1. The "add_something" fixture at the bottom of the code, just like Python's method coverage, it is final "my_list".
2. But "add_something" needs "my_list", so find "init_list_b"(Its position in the code is lower than "init_list_a").
3. I got ["b","c"] in test function.

But I got ["b"] in test function(I find it difficult to predict this outcome): 
1. it seems "add_something" and "init_list_a" is not called, called "init_list_b".
2. sorted(["init_list_a", "init_list_b", "add_something"]) == ['add_something', 'init_list_a', 'init_list_b']('init_list_b' coverages others).
"""

def test(my_list):
    print(my_list)

BTW:
Pytest provides another overload and override mechanism compared to Python, which is a feature of Pytest. I don't want to abandon it, I just want it to become intuitive. The current sorting mode requires additional understanding costs and limits function names.

@RonnyPfannschmidt
Copy link
Member

Pytest should warn or error on this as it's always a user error

Linters catch function override
They don't catch name parameters

I consider python allows redefiniton a strawman as it's either used for a valid pattern like overrides/properties or it's incorrect code

As far as I'm concerned in same module name conflicts we know something is wrong and we must make it known to the user

@dongfangtianyu
Copy link
Contributor

dongfangtianyu commented Dec 18, 2024

should we disallow overriding within a single file (module)

@bluetech cool, your description hits the nail on the head.

As described in the documentation, coverage targets different levels. So the question becomes: should we prohibit repeated coverage at the same level?

If such a requirement does exist, it seems more appropriate to implement it using Python code (if after parsefactories can work like PyCollector).


yet I still don't think Python should disallow it

def func():
    pass


"""
The code here can use old functions
"""

def func():
    pass

"""
Function is overwritten, new code uses new functions
"""

Functions can be used completely before being overridden, same-level fixtures don't seem to be like this, they can only be used by the fixture that overwrites them。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: fixtures anything involving fixtures directly or indirectly type: bug problem that needs to be addressed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants