!ksLFyrUxLxeAxpGLNr:matrix.org

Wikimedia Operations

712 Members
2 Servers

Load older messages


SenderMessageTime
28 Nov 2022
@logmsgbot:libera.chatlogmsgbot !log ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P41326 and previous config saved to /var/cache/conftool/dbconfig/20221128-140141-ladsgroup.json 14:01:43
@wikibugs:libera.chatwikibugs (CR) Jbond: "LGTM comments inline" [puppet] - https://gerrit.wikimedia.org/r/861385 (owner: Slyngshede) 14:04:13
@logmsgbot:libera.chatlogmsgbot!log jbond@cumin2002 END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2050.codfw.wmnet with OS bullseye14:06:20
@wikibugs:libera.chatwikibugs SRE-swift-storage, Infrastructure-Foundations, Patch-For-Review: unstable device mapping of SSDs causing installer problems - example reimage with destruction of swift filesystem - https://phabricator.wikimedia.org/T308677 (ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jbond@cum... 14:06:25
@wikibugs:libera.chatwikibugs (PS2) Jaime Nuche: create group for Release Engineering members [puppet] - https://gerrit.wikimedia.org/r/860836 14:07:38
@jinxer-wm:libera.chatjinxer-wm (KubernetesAPILatency) firing: High Kubernetes API latency (LIST secrets) on k8s@eqiad - https://wikitech.wikimedia.org/wiki/Kubernetes - https://grafana.wikimedia.org/d/000000435?var-site=eqiad&var-cluster=k8s - https://alerts.wikimedia.org/?q=alertname%3DKubernetesAPILatency 14:07:59
@logmsgbot:libera.chatlogmsgbot !log ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P41327 and previous config saved to /var/cache/conftool/dbconfig/20221128-140855-ladsgroup.json 14:08:58
@logmsgbot:libera.chatlogmsgbot !log marostegui@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db1149', diff saved to https://phabricator.wikimedia.org/P41328 and previous config saved to /var/cache/conftool/dbconfig/20221128-140939-marostegui.json 14:09:40
@logmsgbot:libera.chatlogmsgbot!log ladsgroup@cumin1001 START - Cookbook sre.hosts.downtime for 12:00:00 on db1157.eqiad.wmnet with reason: Maintenance14:09:57
@logmsgbot:libera.chatlogmsgbot!log ladsgroup@cumin1001 END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1157.eqiad.wmnet with reason: Maintenance14:10:11
@logmsgbot:libera.chatlogmsgbot !log ladsgroup@cumin1001 dbctl commit (dc=all): 'Depooling db1157 (T323907)', diff saved to https://phabricator.wikimedia.org/P41329 and previous config saved to /var/cache/conftool/dbconfig/20221128-141016-ladsgroup.json 14:10:18
@stashbot:libera.chatstashbotT323907: Make fr_user unsigned - https://phabricator.wikimedia.org/T32390714:10:25
@wikibugs:libera.chatwikibugs SRE, LDAP-Access-Requests: Grant Access to wmf for abartov - https://phabricator.wikimedia.org/T323911 (Marostegui) p:TriageMedium @Asaf from what I can see you are already part of the wmf LDAP group. Not sure if you need something else apart - @Ottomata is there anything else required to access... 14:10:30
@cirno:libera.chatcirnoo/14:10:51
@cirno:libera.chatcirnoLucas_WMDE: sorry I missed the ping14:11:05
@wikibugs:libera.chatwikibugs (CR) Muehlenhoff: ldap:management rewrite modify-mfa to use Bitu. (6 comments) [puppet] - https://gerrit.wikimedia.org/r/861385 (owner: Slyngshede) 14:12:05
@wikibugs:libera.chatwikibugs (PS2) Matthias Mullie: Add mediawiki.searchpreview schema [mediawiki-config] - https://gerrit.wikimedia.org/r/845518 (https://phabricator.wikimedia.org/T321069) 14:12:13
@jinxer-wm:libera.chatjinxer-wm (KubernetesAPILatency) resolved: High Kubernetes API latency (LIST secrets) on k8s@eqiad - https://wikitech.wikimedia.org/wiki/Kubernetes - https://grafana.wikimedia.org/d/000000435?var-site=eqiad&var-cluster=k8s - https://alerts.wikimedia.org/?q=alertname%3DKubernetesAPILatency 14:12:59
@wikibugs:libera.chatwikibugs (PS3) Matthias Mullie: Add mediawiki.searchpreview schema [mediawiki-config] - https://gerrit.wikimedia.org/r/845518 (https://phabricator.wikimedia.org/T321069) 14:13:07
@wikibugs:libera.chatwikibugs SRE, ops-eqiad, DC-Ops, Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-05)): Multiple RAID battery failures on hadoop worker hosts - https://phabricator.wikimedia.org/T318659 (Jclark-ctr) @BTullis we just received batteries. When would work best for you I would like to do them this week if... 14:15:53
@logmsgbot:libera.chatlogmsgbot !log ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P41330 and previous config saved to /var/cache/conftool/dbconfig/20221128-141648-ladsgroup.json 14:16:49
@wikibugs:libera.chatwikibugs (CR) Elukey: Add basic rate-limit capabilities to ML clusters (2 comments) [deployment-charts] - https://gerrit.wikimedia.org/r/860925 (https://phabricator.wikimedia.org/T300259) (owner: Elukey) 14:19:10
@wikibugs:libera.chatwikibugs SRE, ops-eqiad, DC-Ops, Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-05)): Multiple RAID battery failures on hadoop worker hosts - https://phabricator.wikimedia.org/T318659 (BTullis) >>! In T318659#8424520, @Jclark-ctr wrote: > @BTullis we just received batteries. When would work best for... 14:19:51
@wikibugs:libera.chatwikibugs SRE, LDAP-Access-Requests: Grant Access to wmf for abartov - https://phabricator.wikimedia.org/T323911 (MoritzMuehlenhoff) If Asaf needs Supetset access to private tables he needs to be added to the analytics-privatedata-users group, but without an SSH key, see https://wikitech.wikimedia.org/wiki/Analyti... 14:20:11
@wikibugs:libera.chatwikibugs SRE, ops-eqiad, DC-Ops, Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-05)): Multiple RAID battery failures on hadoop worker hosts - https://phabricator.wikimedia.org/T318659 (Jclark-ctr) Yea I am on site right now Let me know when they are ready for me 14:20:49
@logmsgbot:libera.chatlogmsgbot !log ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db1157 (T323907)', diff saved to https://phabricator.wikimedia.org/P41331 and previous config saved to /var/cache/conftool/dbconfig/20221128-142107-ladsgroup.json 14:21:13
@stashbot:libera.chatstashbotT323907: Make fr_user unsigned - https://phabricator.wikimedia.org/T32390714:21:18
* @Lucas_WMDE:libera.chatLucas_WMDE back14:22:12
@logmsgbot:libera.chatlogmsgbot !log ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P41332 and previous config saved to /var/cache/conftool/dbconfig/20221128-142402-ladsgroup.json 14:24:10
@logmsgbot:libera.chatlogmsgbot !log marostegui@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db1149', diff saved to https://phabricator.wikimedia.org/P41333 and previous config saved to /var/cache/conftool/dbconfig/20221128-142446-marostegui.json 14:24:49

There are no newer messages yet.


Back to Room List