Robuta

https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/7MUWUJR67X6PK5QVSX327362YNTUE4YP/?sort=date ** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/thread/GRKMKIBC6DUIXRXK47QXMY2HOUIUUHZS/?sort=date selenium-daily-beta-CirrusSearch - Build # 1527 - Failure! - QA-Alerts - lists.wikimedia.org https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/OU6YF4WMU5KQAVXOD45O5EEYDT2VHNWF/?sort=date ** RECOVERY alert - graphite1001/CirrusSearch codfw 95th percentile latency - more_like is OK ** -... https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Updater.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Updater.php code coverageworkspacesrcextensionsincludes https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/KKWZT4LBZ2X7Q7U4U4ZA24JC2ZGOC52K/ ** PROBLEM alert - graphite1001/CirrusSearch codfw 95th percentile latency - more_like is CRITICAL... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/GCLNTNVD3JGXXXJL7EW3VYUKZB2ID4DV/ ** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://www.mediawiki.org/wiki/Topic:Va3ve5wz726qlkfs CirrusSearch suggestion (^ and $ anchors) on User talk:TJones (WMF)/Flow user talksuggestionanchorswmfflow https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/IFM32NMHAX4FCW2BED7UUWU7H7ACVX3N/?sort=thread ** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/C74LQHADYMVPWXJL7IRN27MPLGXT5SNI/ ** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/EQ53UHPPWKBQL3DGMPHAJAFDLRPNUCNK/ ** RECOVERY alert - alert1001/Mediawiki CirrusSearch update rate - eqiad is OK ** -... recoveryalertmediawikiupdaterate https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Maintenance/Validators/index.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Maintenance/Validators code coverageworkspacesrcextensionsincludes https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/UQKAVAXZXQB2PN72IKY2MYYL2LWV5OGX/ ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/D5EIL3KZORDS4KDCZMM66NRXRDG7BAA5/ ** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update lag - eqiad is CRITICAL ** -... problemalertmediawikiupdatelag https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/ITNQ6VH7WKJYJ5IM7XHEOOYDH4VSTDJB/?sort=thread ** RECOVERY alert - graphite2001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/thread/OICEQMSMPGYVC75KA6Z2YDUITAIEPRBB/ selenium-daily-beta-CirrusSearch - Build # 1605 - Failure! - QA-Alerts - lists.wikimedia.org https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/message/OJJYPCATCG35KX6JWHNE6LYXTCQB2SLT/ [QA-Alerts] selenium-daily-beta-CirrusSearch - Build # 1582 - Still Failing! - QA-Alerts -... qaalertsseleniumdailybeta https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/message/4UTTC7YYTJAN4BENO5AZFVCASACZMOVX/ [Discovery-alerts] ** PROBLEM alert - graphite1005/CirrusSearch more_like eqiad 95th percentile... discoveryalertsproblem https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/63BJ7NJCJVUS2UZFJGWEEG7HY4USTDPC/ ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/GMUNCIZLXRNPBU4YSBJTS434LGS4QYAH/ ** RECOVERY alert - graphite1001/Number of backend failures per minute from CirrusSearch is OK ** -... https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Extra/dashboard.html Dashboard for /workspace/src/extensions/CirrusSearch/includes/Extra dashboardworkspacesrcextensionsincludes https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/message/WPP4GSOA7Y32HSGYE6GGNLTJGFUIJWAV/ [Discovery-alerts] ** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is... discoveryalertsrecovery https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/QCP3OFOWD43SUUDJYC3J54DWTXCMBV36/ ** PROBLEM alert - graphite1001/CirrusSearch codfw 95th percentile latency - more_like is CRITICAL... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/Z6VMAQ6UN2GNWKF7X66I3LFODDLWLPE5/ ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/RI36GGSBGDGCMFV4NQY7VH2WBDWSAIA4/ ** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/message/QVRZ37C22P4FB5VFKF7QLNQI5WFNSNZK/ [QA-Alerts] selenium-daily-beta-CirrusSearch - Build # 1528 - Still Failing! - QA-Alerts -... qaalertsseleniumdailybeta https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/thread/HCJSCBTB7EPK63SS7VFOTAZIXFNM4UUM/?sort=thread browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox - Build # 93 - Failure! - QA-Alerts -... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/VSBI57LPWW35YEDUQEMAOIFA3RYZ3GWE/ ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/5VQO4BAGWQNRFCVKZCWMKDK4S2ZXXPA4/ ** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is OK ** -... recoveryalertmediawikiupdaterate https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/WBNRMS4AJTDRPQYAR7ZACTQKKASYYJWP/?sort=thread ** PROBLEM alert - graphite1001/Number of backend failures per minute from CirrusSearch is CRITICAL... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/TFIMTSFXO4XLF76EJN3BWA4RX74DP7SN/ ** RECOVERY alert - graphite1005/CirrusSearch more_like codfw 95th percentile latency is OK ** -... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/OZUNZYYRUG43J5TJ2SPKQYUPCNFM4MAY/ ** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/IMXPAD5OMN6D5WMMBIVMTMXFZQ4Z7YOT/ ** PROBLEM alert - graphite1004/Number of backend failures per minute from CirrusSearch is CRITICAL... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/35QLZW3W2SRKBVSBNSMXTJB73IY26CG5/ ** PROBLEM alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is CRITICAL ** -... problemalertmediawikiupdaterate https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/THYORNO65DEOX4HYBLTNTB3D5E6GPUX2/ ** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update lag - codfw is OK ** -... recoveryalertmediawikiupdatelag https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/6ICTEK6MY2RJB6YDFDCOYOPLYNT2NXP4/ ** PROBLEM alert - graphite1001/CirrusSearch codfw 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Job/IndexArchive.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Job/IndexArchive.php code coverageworkspacesrc https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/YATTIS5NDZ3QQG3RYLOIICQD22XV6X63/ ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/6CJ5GQBZSFUNWAACDVBEFBLLL7WWRCQ4/ ** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/SCY6OANL5WSKXFJ67QQYZ2VM3RXNJB7Z/ ** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is OK ** -... recoveryalertmediawikiupdaterate https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Elastica/index.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Elastica code coverageworkspacesrcextensionsincludes https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/BaseRequestLog.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/BaseRequestLog.php code coverageworkspacesrcextensionsincludes https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/37UPIJWZHTVGHCQACM55FAN5XWW2I7IX/ ** RECOVERY alert - graphite1005/CirrusSearch comp_suggest eqiad 95th percentile latency is OK ** -... https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Search/Fetch/HighlightedField.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Search/Fetch/HighlightedField.php code coverage https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/4UTTC7YYTJAN4BENO5AZFVCASACZMOVX/?sort=date ** PROBLEM alert - graphite1005/CirrusSearch more_like eqiad 95th percentile latency is CRITICAL **... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/7J4VRW2Y4QVIPZG37LN2NMZ6ME276256/ ** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update lag - codfw is OK ** -... recoveryalertmediawikiupdatelag https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/FH36CC4KY6XARL2NYNZC2REEGUDGOAPX/ ** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - eqiad is OK ** -... recoveryalertmediawikiupdaterate https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/ZH2LZCSWQ7NRP5DH2NJSTCOOZJKEEO5Y/?sort=date ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Sanity/CountingRemediator.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Sanity/CountingRemediator.php code coverageworkspacesrc https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/thread/K5OG64DVC4VLOVGUF3QUUTUNGYGX4YXI/?sort=date selenium-daily-beta-CirrusSearch - Build # 1517 - Failure! - QA-Alerts - lists.wikimedia.org https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/SZYBWHUW7IOOITAHFXINV6YD7SDTABX7/ ** RECOVERY alert - icinga1001/Mediawiki Cirrussearch update rate - codfw is OK ** -... recoveryalertmediawikiupdaterate https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/message/HMY6NX67DNA2MNBK2YSOPXBPD5PF5X5F/ [QA-Alerts] selenium-daily-beta-CirrusSearch - Build # 1548 - Failure! - QA-Alerts -... qaalertsseleniumdailybeta https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/thread/4PNUZJZCDRIPZSL2PR6T7HU5D3ZZZEW5/ selenium-CirrusSearch - Build # 172 - Failure! - QA-Alerts - lists.wikimedia.org seleniumbuildfailureqaalerts https://lists.wikimedia.org/hyperkitty/list/discovery@lists.wikimedia.org/thread/YILIHGTWWN4CRXIVRCDTERTHLIYZFLP5/?sort=date CirrusSearch switch to codfw - lessons learned - Discovery - lists.wikimedia.org switch tolessons learneddiscoverylistswikimedia https://wikitech-static.wikimedia.org/wiki/Help_CirrusSearch_OpenSearch_replicas.html Help:CirrusSearch OpenSearch replicas - Wikitech helpopensearchreplicaswikitech https://www.mediawiki.org/wiki/Help_talk:CirrusSearch Help talk:CirrusSearch - MediaWiki helptalkmediawiki https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/Y4S366HGOGUERVWWBXOSG63WAGSJTDQB/?sort=date ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/GX4G2WBVJELI7J7ZPKTR54JXEMUKJMIB/ ** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/SMV2RFWT42A3SBBYGA7MEPS64VFO5ZOM/ ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/qa-alerts@lists.wikimedia.org/message/WDSMJNM44ZPGVZR6BWGTNRHDW763YWPO/ [QA-Alerts] selenium-daily-beta-CirrusSearch - Build # 1581 - Still Failing! - QA-Alerts -... qaalertsseleniumdailybeta https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/2MMKEKJ2GGCG3EV4OC7L7S6GOMYRGJBQ/?sort=date ** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Parser/AST/ParseWarning.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Parser/AST/ParseWarning.php code coverage https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/message/PXTWMRT2ZYXHB6YP5YUGA4J3YWQ4JWWI/ [Discovery-alerts] ** PROBLEM alert - graphite1001/CirrusSearch eqiad 95th percentile latency is... discoveryalertsproblem https://doc.wikimedia.org/cover-extensions/CirrusSearch/includes/Search/Fetch/HighlightingTrait.php.html Code Coverage for /workspace/src/extensions/CirrusSearch/includes/Search/Fetch/HighlightingTrait.php code coverage https://wikitech.wikimedia.org/wiki/Incidents/2025-07-21_Cirrussearch_outage Incidents/2025-07-21 Cirrussearch outage - Wikitech incidentsoutagewikitech https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/ZWEEELVNU7PPHKPXN4PPXZUBHIMVEN7H/ ** PROBLEM alert - graphite1005/CirrusSearch more_like codfw 95th percentile latency is CRITICAL **... https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/BNL2X7H3TQNAQZABDIEPYA6LUE4U7E2I/?sort=thread ** RECOVERY alert - graphite2001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://www.mediawiki.org/wiki/Manual:Hooks/CirrusSearchAddQueryFeatures Extension:CirrusSearch/Hooks/CirrusSearchAddQueryFeatures - MediaWiki extensionhooksmediawiki https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/FRQBN7HKLOOXZCJBNMDFK3MSINTXACQZ/?sort=date ** RECOVERY alert - graphite1001/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/JQYVN47TASTDVEMDRRXNF2RQFH6QINHY/ ** PROBLEM alert - graphite1004/CirrusSearch eqiad 95th percentile latency is CRITICAL ** -... problemalertpercentilelatencycritical https://lists.wikimedia.org/hyperkitty/list/discovery-alerts@lists.wikimedia.org/thread/WZEOS6NBTM6QYD7JLXARG43GJB3XQOQ3/ ** RECOVERY alert - graphite1004/CirrusSearch eqiad 95th percentile latency is OK ** -... recoveryalertpercentilelatencyok