!AhUnkatMeayVVxZprp:matrix.org

talos-support

62 Members
Talos.dev product support1 Servers

Load older messages


SenderMessageTime
14 May 2021
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
Very possible it's a PEBKAC/ID10T error. When the Server showed up I ran a patch to set accepted: true but also set bmc user\pass to one I set up manually specifically for sidero... I later saw the patch notes about Auto BMC config, and it's using Secret refs for the creds, but I think my manual setups may have clashed at some point
The sidero/timniverse is the user/pass I set up manually on the IPMI interface for sidero, and the one I put into the Server spec at /spec/bmc/user and /spec/bmc/pass
20:04:29
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
The sidero/timniverse is the user/pass I set up manually on the IPMI interface for sidero, and the one I put into the Server spec at /spec/bmc/user and /spec/bmc/pass
it very well might be it, Sidero is also provisioning sidero user
20:09:29
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
it very well might be it, Sidero is also provisioning sidero user
so you can try removing your bmc user/pass but keeping the reference
20:09:41
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply toundefined
also, any way to control the namespace the Secret is generated in? I don't like stuff in default at all!! πŸ™‚
20:13:35
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply toundefined
(edited) ... at all¬¬ πŸ™‚ => ... at all!! πŸ™‚
20:13:40
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
also, any way to control the namespace the Secret is generated in? I don't like stuff in default at all!! πŸ™‚
rsmitty has more details, but this basically boils down to the cluster-api limitations
20:16:12
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
rsmitty has more details, but this basically boils down to the cluster-api limitations
we can't put it to sidero-system
20:16:21
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
we can't put it to sidero-system
no? shame...
20:16:37
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
no? shame...
user secrets can be anywhere, but auto-generated ones need some namespace
20:16:44
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
user secrets can be anywhere, but auto-generated ones need some namespace
sidero-system is owned by clusterctl install/upgrade process, so it gets cleaned completely on upgrade
20:17:07
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
sidero-system is owned by clusterctl install/upgrade process, so it gets cleaned completely on upgrade
yeah, just wondering how that namespace could be provided to sidero... yet another env var in the templates?
20:17:24
@_slack_taloscommunity_UH9V1FL0K:matrix.orgrsmitty
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
yeah, just wondering how that namespace could be provided to sidero... yet another env var in the templates?
Yeah it’d have to be something like that
20:17:44
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UH9V1FL0K:matrix.org
Yeah it’d have to be something like that
can default to default, but allows override...
20:17:46
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
can default to default, but allows override...
cool
20:17:52
@_slack_taloscommunity_UGL0YU56H:matrix.organdrey
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
cool
yep, please log an issue to GitHub so that we don't forget it please πŸ˜‰
20:17:56
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
yep, please log an issue to GitHub so that we don't forget it please πŸ˜‰
sure thing πŸ‘
20:18:34
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
sure thing πŸ‘
https://github.com/talos-systems/sidero/issues/396
20:21:51
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
https://github.com/talos-systems/sidero/issues/396
Might even be something I can take a look at, but I should be heading to bed already as it is!! Maybe during the little ones nap-time tomorrow πŸ˜‰
20:22:44
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
Might even be something I can take a look at, but I should be heading to bed already as it is!! Maybe during the little ones nap-time tomorrow πŸ˜‰
I also have to ask, is IPMI really so insecure as to allow adding an admin-level user account without prior credentials?? WTF?? πŸ˜†
20:25:11
@_slack_taloscommunity_UH9V1FL0K:matrix.orgrsmitty
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
I also have to ask, is IPMI really so insecure as to allow adding an admin-level user account without prior credentials?? WTF?? πŸ˜†
lol yeah I was kind of surprised that we could just do that
20:25:39
@_slack_taloscommunity_UH9V1FL0K:matrix.orgrsmitty
In reply to@_slack_taloscommunity_UH9V1FL0K:matrix.org
lol yeah I was kind of surprised that we could just do that
but as long as you're on the box itself, it's pretty straightforward
20:25:54
@_slack_taloscommunity_UJYGJM76W:matrix.orgSeΓ‘n C McCord
In reply to@_slack_taloscommunity_UH9V1FL0K:matrix.org
but as long as you're on the box itself, it's pretty straightforward
You can't do unauthenticated except via local access... but yeah, it's kind of crazy-powerful-insecure
20:26:24
@_slack_taloscommunity_UJYGJM76W:matrix.orgSeΓ‘n C McCord
In reply to@_slack_taloscommunity_UJYGJM76W:matrix.org
You can't do unauthenticated except via local access... but yeah, it's kind of crazy-powerful-insecure
Fundamentally, never expose IPMI to an untrusted network.
20:26:42
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UJYGJM76W:matrix.org
Fundamentally, never expose IPMI to an untrusted network.
Guess I should turn off port-forwarding from my public interface then πŸ˜‰
20:27:54
@_slack_taloscommunity_UJYGJM76W:matrix.orgSeΓ‘n C McCord
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
Guess I should turn off port-forwarding from my public interface then πŸ˜‰
Eh, if you want to. It can be fun watching what a hacked box can do, too. Especially since BMC hacks are harder to detect and can do much fancier things than mere OS hacks. πŸ™‚
20:31:39
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UJYGJM76W:matrix.org
Eh, if you want to. It can be fun watching what a hacked box can do, too. Especially since BMC hacks are harder to detect and can do much fancier things than mere OS hacks. πŸ™‚
Yeah, I used to enjoy setting up honeypots open to 22 & the like, but really they were just logging everything peeps triedf to do...
20:39:35
@_slack_taloscommunity_UG8G8UMMG:matrix.orgTim Jones
In reply to@_slack_taloscommunity_UG8G8UMMG:matrix.org
Yeah, I used to enjoy setting up honeypots open to 22 & the like, but really they were just logging everything peeps triedf to do...
But it was only cool when it was real people, when the botnets really started taking off it was just the same script scraping the box and got boring
20:40:21
15 May 2021
@_slack_taloscommunity_URFT9H8MB:matrix.orgAlex Szakaly
In reply to@_slack_taloscommunity_UGL0YU56H:matrix.org
https://www.talos.dev/docs/v0.9/guides/upgrading-kubernetes/#kubelet
I forgot to thank you, sorry Andrey. πŸŽ‰
05:23:20
@_slack_taloscommunity_U01L24AR5MG:matrix.orgborn2bake
In reply to@_slack_taloscommunity_U01L24AR5MG:matrix.org
and then it's added as a worker node
Hi andrey , I ve managed to restore the masters and I can see all of them as etcd members. However, once they were restored, all services/workload are unavailable, nginx-ingress shows 503 errors so it looks like the masters cant see the workers in the cluster. Any suggestions if I can restore worker nodes somehow? The way I restore masters - master-3 was bootstrapped as a worker node, I ve run etcd restore from backup snapshot and then master-3 instantly became a master, then I ran reset ephemeral on that node and restarted it. Then I did restore etcd for master-2 and reset it as well. So all 3 now in a quorum
08:43:59
@_slack_taloscommunity_U01L24AR5MG:matrix.orgborn2bake
In reply to@_slack_taloscommunity_U01L24AR5MG:matrix.org
Hi andrey , I ve managed to restore the masters and I can see all of them as etcd members. However, once they were restored, all services/workload are unavailable, nginx-ingress shows 503 errors so it looks like the masters cant see the workers in the cluster. Any suggestions if I can restore worker nodes somehow? The way I restore masters - master-3 was bootstrapped as a worker node, I ve run etcd restore from backup snapshot and then master-3 instantly became a master, then I ran reset ephemeral on that node and restarted it. Then I did restore etcd for master-2 and reset it as well. So all 3 now in a quorum
I just rebooted worker nodes and all services were restored πŸ™Œ thew
10:27:00

There are no newer messages yet.


Back to Room List