| 19 Apr 2024 |
|  Asinine Monkey joined the room. | 09:48:41 |
|  tee changed their display name from blacky to ttt. | 10:59:35 |
| tee changed their profile picture. | 11:02:00 |
| tee changed their display name from ttt to tee. | 11:02:47 |
|  billisdead joined the room. | 12:29:25 |
 Maranda | In reply to @kenrachynski:matrix.org
guessing Amazon gets to work with very early releases Very early. Good luck high fortune, alas got to understand many things now đ | 16:15:11 |
| Maranda | * Very early. Good luck high fortune, alas understood
many of the EKS shenanigans now đ | 16:16:44 |
| 21 Apr 2024 |
|  simboby joined the room. | 19:28:31 |
| 22 Apr 2024 |
 Feeds | New post in Kubernetes Blog: Kubernetes 1.30: Beta Support For Pods With User Namespaces | 00:30:22 |
| 23 Apr 2024 |
| Feeds | New post in Kubernetes Blog: Kubernetes 1.30: Read-only volume mounts can be finally literally read-only | 00:28:29 |
 Rens Houben | Quick update to earlier: I've managed to get everything working. I did wind up re-enabling default routes because nodes just will not stop complaining otherwise but the firewall doesn't let any of them out onto the internet so that's fine. Using (currently) two control plane nodes mediated via haproxy, ansible playbook mediates the whole thing from bare Debian installs to fully functioning cluster using calico and MetalLB. Thanks for all the help, advice and commiseration, I really appreciate it. | 14:20:30 |
| Rens Houben | .. Now to see what I can actually do with this setup. :D | 14:20:42 |
|  Nick B joined the room. | 22:00:57 |
| Nick B | I'm banging my head on nginx-ingress configuration for websockets. I think I've got the right annotations, but when I exec'd into the nginx pod, it doesn't look like they were applied. https://pastebin.com/iGQWCaf6 | 22:03:43 |
| Nick B | Would anyone mind taking a look? I'd greatly appreciate it | 22:04:49 |
 Sheogorath | Nick B: Do you run ingress-nginx or nginx-ingress? These are different projects with different annotations. (Former from K8s.io the latter from nginx.com.) | 23:45:20 |
| Sheogorath | ingress-nginx claims you don't need to do anything special for websockets: https://kubernetes.github.io/ingress-nginx/user-guide/miscellaneous/#websockets (and that fits with my experience). | 23:46:00 |
| Sheogorath | for nginx-ingress there are explicit websocket annotations: nginx.org/websocket-services | 23:47:27 |
| Sheogorath | * for nginx-ingress there are explicit websocket annotations: nginx.org/websocket-services https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/ | 23:47:39 |
| 24 Apr 2024 |
| Feeds | New post in Kubernetes Blog: Kubernetes 1.30: Validating Admission Policy Is Generally Available | 00:27:52 |
| Nick B | In reply to @sheogorath:shivering-isles.com
Nick B: Do you run ingress-nginx or nginx-ingress? These are different projects with different annotations. (Former from K8s.io the latter from nginx.com.) I use nginx-ingress; Thanks I'll see if I can put something together based on that | 05:21:24 |
 jokeyrhyme | Hmmmm, I've got a weird issue with a DaemonSet that starts 3 Pods, where one of the Pods fails with a status of SysctlForbiddenÂ
But these nodes are all provisioned using the same scripts, and the sibling pods all start properly on the other Nodes
It's just this one Node where it fails consistently
I've even tried deleting the DaemonSet and Pods and starting all over again, but the same thing happens
sudo sysctl --all is the same on all Nodes, and I haven't explicitly used any security contexts or anything like that
| 06:34:34 |
| jokeyrhyme | Hmmmm, I just kubectl describe each Node and compared them and there aren't any significant differences, especially no conspicuous annotations
It's very weird that the same scripts were all working yesterday and now they aren't
| 06:51:06 |
 effendy | Is it always the same node? Have you been able to confirm this? | 12:59:01 |
 lub | does someone know of problems regarding network policies in k3s? I have a namespace with various network policies. everything is alright. then I update one of the container images, deploy the new image and suddenly the network policy doesn't work correctly (i.e. web server can't connect to database anymore). when I remove the network policies everything is fine again.
sometimes it works sometimes it doesn't | 14:45:04 |
| lub | ohh I've just looked again in the issues and there is now this comment https://github.com/k3s-io/k3s/issues/947#issuecomment-2057859416 | 14:46:29 |
|  @fenuks:sibnsk.net left the room. | 15:02:33 |
| jokeyrhyme | In reply to @effendy:matrix.org
Is it always the same node? Have you been able to confirm this? Yep, same node
It also happens to be the --cluster-init node in my k3s cluster, all the others start as --serverÂ
Although, this has all been working for a week or so before suddenly being weird
I did just disable the ServiceLB component, that's the only change I can think of
Maybe I should revert and see if it fixes it | 23:24:59 |
| 25 Apr 2024 |
| Feeds | New post in Kubernetes Blog: Kubernetes 1.30: Structured Authentication Configuration Moves to Beta | 00:36:18 |
| jokeyrhyme | In reply to @effendy:matrix.org
Is it always the same node? Have you been able to confirm this? Aha! Found it: https://github.com/k3s-io/k3s/blob/94e29e2ef5d79904f730e2024c8d1682b901b2d5/pkg/daemons/agent/agent_linux.go#L198
So, yes, disabling the default ServiceLB in k3s removed the allow-listed unsafe sysctls arguments
It's still very odd that this only impacted one node, however, given that I keep what is enabled/disabled the same across the cluster | 02:47:17 |
