Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linux: Backport fixes for bpfilter and usermode helper to prevent iptable failures with "no child processes" #1495

Open
wants to merge 2 commits into
base: 3.0
Choose a base branch
from

Conversation

randomvariable
Copy link
Member

@randomvariable randomvariable commented Jul 18, 2023

Under some circumstances which are not fully understood, the user mode helper process of the bpfilter module may fail. This causes applications that use iptables (notably Kubernetes kube-proxy and CNIs such as Calico) to not be able to view or change iptables. The error message received under such circumstances is:

  • kube-proxy:
proxier.go:859] "Failed to ensure chain exists" err="error creating chain \"KUBE-EXTERNAL-SERVICES\": exit status 3: iptables v1.8.2 (legacy): can't initialize iptables table `filter': No child processes\nPerhaps iptables or your kernel needs to be upgraded.\n" table=filter chain=KUBE-EXTERNAL-SERVICES
  • calico-node / felix:
[PANIC][8557] felix/table.go 769: iptables-legacy-save command failed after retries ipVersion=0x4 table="nat"
panic: (*logrus.Entry) 0xc000577540

goroutine 294 [running]:
github.com/sirupsen/logrus.Entry.log(0xc00007e1e0, 0xc00061a630, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7f2800000000, ...)
	/go/pkg/mod/github.com/projectcalico/[email protected]/entry.go:128 +0x6a5
github.com/sirupsen/logrus.(*Entry).Panic(0xc0001aa960, 0xc000998b58, 0x1, 0x1)
	/go/pkg/mod/github.com/projectcalico/[email protected]/entry.go:173 +0xfa

Ways to reproduce:

Failure mode 1:

  1. Rename bpfilter.ko.xz to something else
  2. rmmod bpfilter
  3. iptables

Failure mode 2:

  1. grep dmesg for bpfilter_umh, e.g.
bpfilter: Loaded bpfilter_umh pid 1720
  1. echo garbage into /proc/<pid>/fd/0
  2. iptables (this will break until the next reboot)

In some failure modes, this can be worked around with rmmod bpfilter.

The patches here make getsockopt, which is the syscall that iptables expects to succeed return successfully even if the usermode helper process is not working. Additionally, there are pre-requisite backports to improve the robustness of the bpfilter module.

When used together, neither of the failure modes are present.

Signed-off-by: Naadir Jeewa [email protected]

Naadir Jeewa added 2 commits July 20, 2023 01:14
Signed-off-by: Naadir Jeewa <[email protected]>
Change-Id: I488dbd212de7fb36d6ee342054f6c66e086f2dc0
affecting iptables.

Signed-off-by: Naadir Jeewa <[email protected]>
Change-Id: Iac186b69ee40873c30e102400d21ac2241ae4120
@randomvariable randomvariable force-pushed the topic/jeewan/bpfilter-iptables-compat branch from 031e399 to e48406e Compare July 20, 2023 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants