Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 46.741042ms FAIL (2025-08-08 05:09:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 823884 (19h28m25.204926455s) [seg_pos={0 0}, upload=20117]: Done begin upload segment. Aug 8 05:07:50.689845352 (19h28m25.204947931s) [noise=true , address=104.36.82.62:4409 , piece_id=6ETQE3PYGLP2LUKEVYQT5GUYOU4HOXSFSTEMWBE5O4OQQPZ45ROQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.689876284 (19h28m25.20497887s) [noise=true , address=64.49.13.170:4725 , piece_id=MAWII32TCXPSBBIZ6N3UH2SI6GQXXDY5YYMQXEKAAGBXZ3P7EZXA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.690299079 (19h28m25.205401961s) [noise=true , address=104.36.82.58:4273 , piece_id=ZGYB4SSWJNCMKAARDCNSEQMFAQJN7API2HERTCUZP5UYARRSSFPA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.690414644 (19h28m25.205517222s) [noise=true , address=104.36.82.59:4329 , piece_id=TKQYQW3K6TAPCKEMQEBLAMPDWM4ZTKUW2DVQLGO5XAADU5ZVM7MQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.690692455 (19h28m25.205795037s) [noise=true , address=104.36.82.47:4530 , piece_id=OKRF4KVXISXVUTIV4BQGBJOQ62UJK2MA32UI6XM5EYC2Y7PJ4PZQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.690753877 (19h28m25.205856454s) [noise=true , address=104.36.82.46:4503 , piece_id=JZMUA6HDO3UKWDD4OLCN4VEDYOVNRCR7YI2FSFHOGCVFYGTRAVVQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.691078732 (19h28m25.206181312s) [noise=true , address=104.36.82.37:4681 , piece_id=7ZFQFX2Q6C4YZNLIORFAF4QVL7H4QCBZHDVJI2TR2ZRY4HI67TQA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.691274182 (19h28m25.206376764s) [noise=true , address=104.36.82.44:4904 , piece_id=OBWUIWKTMV7WNHLWYMVWNCEDYKVQRTHVSD2QXBIW4Y7R2LHQEX2Q, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.691315491 (19h28m25.206418073s) [noise=true , address=104.36.82.20:4451 , piece_id=CJ57ABBCVOSQPID254BOCGQYV5TCKBX75X3I4TTJPBPDAB3SRGZQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.691540668 (19h28m25.20664325s) [noise=true , address=104.36.82.4:4811 , piece_id=BITGHQSFC2QWI3MT6WW4RQBVEZLGCWZOLEAZDQT5HKKEIS7BQVGQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.691764660 (19h28m25.206867235s) [noise=true , address=104.36.82.12:4163 , piece_id=TLKBIUZ3LCQWCRMHRDZRRI62UKU7ACUDS3YYEOU3K6DASXRCS3VA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.691991240 (19h28m25.207093819s) [noise=true , address=104.36.82.23:4479 , piece_id=OWQGKE2OPGKDKME3ZM5QP7ICZI7DGK64JGDDYZ2Y2XJ7I2SYDDHA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.692208677 (19h28m25.207311255s) [noise=true , address=64.49.13.138:4486 , piece_id=QQQFYIR7LMDJVBCPKRKDYOS5K6PT7V47427ITU33TBHHQG4TOH7Q, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.692483009 (19h28m25.207585591s) [noise=true , address=104.36.82.38:4696 , piece_id=JA4R7FCB2U243LV5LFX4TVIKTKQSXJLM3PMPPCPW45MO75I25XTQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.692631937 (19h28m25.20773452s) [noise=true , address=104.36.82.60:4360 , piece_id=UIOWINYLBEQPYZA4VZIROITCEQNL3Y7VBC5DD6RFUOHYJRRAWGXQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.692956125 (19h28m25.208058727s) [noise=true , address=104.36.82.18:4398 , piece_id=RHEHRWJAGRBFJ7FCBK2AP2QKHRQ6T3SCAU6P6N2676XRX3EIQFQQ, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.693275480 (19h28m25.208378064s) [noise=true , address=104.36.82.43:4839 , piece_id=D3LL37FREL2NLARF7ZNDMIIM5MTIYWT6HKHBGIIJ4WAU3JTSXIMA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.693517734 (19h28m25.208620316s) [noise=true , address=104.36.82.56:4242 , piece_id=VF4TASC7USPMTSXWGYQNNTLDPJE2YXDJBAPGKSA3SYDAPGZKZYGA, seg_pos={0 0}, upload=20117]: Uploading piece... Aug 8 05:07:50.796953768 (19h28m25.312056349s) [upload=20117]: Next returned nil segment Aug 8 05:07:50.796969939 (19h28m25.312072524s) [upload=20117]: Waiting for error group managing segments... Aug 8 05:07:50.807280798 (19h28m25.322383368s) [noise=true , address=104.36.82.44:4904 , piece_id=OBWUIWKTMV7WNHLWYMVWNCEDYKVQRTHVSD2QXBIW4Y7R2LHQEX2Q, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.807499572 (19h28m25.32260214s) [noise=true , address=104.36.82.62:4409 , piece_id=6ETQE3PYGLP2LUKEVYQT5GUYOU4HOXSFSTEMWBE5O4OQQPZ45ROQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.808409851 (19h28m25.323512419s) [noise=true , address=104.36.82.60:4360 , piece_id=UIOWINYLBEQPYZA4VZIROITCEQNL3Y7VBC5DD6RFUOHYJRRAWGXQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.808537380 (19h28m25.32363996s) [noise=true , address=104.36.82.43:4839 , piece_id=D3LL37FREL2NLARF7ZNDMIIM5MTIYWT6HKHBGIIJ4WAU3JTSXIMA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.809142026 (19h28m25.324244594s) [noise=true , address=104.36.82.59:4329 , piece_id=TKQYQW3K6TAPCKEMQEBLAMPDWM4ZTKUW2DVQLGO5XAADU5ZVM7MQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.809727637 (19h28m25.324830208s) [noise=true , address=104.36.82.23:4479 , piece_id=OWQGKE2OPGKDKME3ZM5QP7ICZI7DGK64JGDDYZ2Y2XJ7I2SYDDHA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.810036385 (19h28m25.325138952s) [noise=true , address=104.36.82.20:4451 , piece_id=CJ57ABBCVOSQPID254BOCGQYV5TCKBX75X3I4TTJPBPDAB3SRGZQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.810524084 (19h28m25.325626646s) [noise=true , address=104.36.82.46:4503 , piece_id=JZMUA6HDO3UKWDD4OLCN4VEDYOVNRCR7YI2FSFHOGCVFYGTRAVVQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.811239458 (19h28m25.326342021s) [noise=true , address=104.36.82.18:4398 , piece_id=RHEHRWJAGRBFJ7FCBK2AP2QKHRQ6T3SCAU6P6N2676XRX3EIQFQQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.811747718 (19h28m25.326850288s) [noise=true , address=104.36.82.47:4530 , piece_id=OKRF4KVXISXVUTIV4BQGBJOQ62UJK2MA32UI6XM5EYC2Y7PJ4PZQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.812233806 (19h28m25.327336381s) [noise=true , address=104.36.82.58:4273 , piece_id=ZGYB4SSWJNCMKAARDCNSEQMFAQJN7API2HERTCUZP5UYARRSSFPA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.812577779 (19h28m25.327680346s) [noise=true , address=104.36.82.12:4163 , piece_id=TLKBIUZ3LCQWCRMHRDZRRI62UKU7ACUDS3YYEOU3K6DASXRCS3VA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.813745244 (19h28m25.328847814s) [noise=true , address=104.36.82.38:4696 , piece_id=JA4R7FCB2U243LV5LFX4TVIKTKQSXJLM3PMPPCPW45MO75I25XTQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.813953245 (19h28m25.329055807s) [noise=true , address=104.36.82.37:4681 , piece_id=7ZFQFX2Q6C4YZNLIORFAF4QVL7H4QCBZHDVJI2TR2ZRY4HI67TQA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.813966177 (19h28m25.329068739s) [seg_pos={0 0}, upload=20117]: Segment reached optimal threshold of 14 pieces. Aug 8 05:07:50.814243345 (19h28m25.329345924s) [noise=true , address=104.36.82.56:4242 , piece_id=VF4TASC7USPMTSXWGYQNNTLDPJE2YXDJBAPGKSA3SYDAPGZKZYGA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: <nil> Aug 8 05:07:50.814400035 (19h28m25.329502614s) [noise=true , address=104.36.82.4:4811 , piece_id=BITGHQSFC2QWI3MT6WW4RQBVEZLGCWZOLEAZDQT5HKKEIS7BQVGQ, seg_pos={0 0}, upload=20117]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TxjP4CnxyU4t2DTNBjBMBxnizcHCD42fMrB5Da6Qiqg1Eg17V): protocol: expected piece hash; context canceled; EOF Aug 8 05:07:50.814943535 (19h28m25.330046104s) [noise=true , address=64.49.13.170:4725 , piece_id=MAWII32TCXPSBBIZ6N3UH2SI6GQXXDY5YYMQXEKAAGBXZ3P7EZXA, seg_pos={0 0}, upload=20117]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mit6F4Yz1qKuVeXhhfBpNd5MsfxWLz8dCUAHsBeGzCdLJn8Ne): protocol: read tcp 159.223.143.39:52880->64.49.13.170:4725: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 05:07:50.815037104 (19h28m25.330139685s) [noise=true , address=64.49.13.138:4486 , piece_id=QQQFYIR7LMDJVBCPKRKDYOS5K6PT7V47427ITU33TBHHQG4TOH7Q, seg_pos={0 0}, upload=20117]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126i47Q2ngKd79VRwsSQXZn3QTrJfmvZAQdK6yqFWXkCo5ba26g): protocol: read tcp 159.223.143.39:35090->64.49.13.138:4486: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 05:07:50.815074873 (19h28m25.33017744s) [seg_pos={0 0}, upload=20117]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 8 05:07:50.815158138 (19h28m25.33026072s) [upload=20117]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 05:07:50.815163707 (19h28m25.330266293s) [upload=20117]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 05:07:50.895420076 (19h28m25.410522646s) [upload=20117]: Done uploading segments... Aug 8 05:07:50.895438293 (19h28m25.41054086s) [upload=20117]: Upload finished. err: <nil> Aug 8 05:09:01.786221211 (19h29m36.301323791s) [upload=20144]: Starting upload Aug 8 05:09:01.786280773 (19h29m36.301383344s) [upload=20144]: Done starting upload Aug 8 05:09:01.786606643 (19h29m36.301709213s) [upload=20144]: Uploading segments... Aug 8 05:09:01.786659628 (19h29m36.301762216s) [upload=20144]: Got next segment. Inline: false Aug 8 05:09:01.786720249 (19h29m36.301822836s) [upload=20144]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 05:09:01.786725788 (19h29m36.301828371s) [upload=20144]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 05:09:01.797061485 (19h29m36.312164075s) [upload=20144]: Next segment err: metaclient: Too Many Requests Duration: 46.768609ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.................................X.........................] duration: 204.256225ms FAIL (2025-08-08 05:08:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 204.279116ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.................................X........................] duration: 179.892243ms FAIL (2025-08-08 05:08:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 179.944513ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 537.539945ms FAIL (2025-08-08 05:08:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 537.56365ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X.................................X.......................] duration: 250.905499ms FAIL (2025-08-08 05:08:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 250.929923ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 481.165213ms FAIL (2025-08-08 05:07:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 481.18867ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X.................................X......................] duration: 174.627926ms FAIL (2025-08-08 05:07:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 174.658575ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 500.491549ms FAIL (2025-08-08 05:06:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 500.522492ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 492.703559ms FAIL (2025-08-08 05:06:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 492.724958ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X.................................X.....................] duration: 206.037374ms FAIL (2025-08-08 05:05:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 206.060218ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X.................................X....................] duration: 531.801053ms FAIL (2025-08-08 05:05:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 531.828888ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 479.539593ms FAIL (2025-08-08 05:05:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.565527ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X.................................X...................] duration: 427.777319ms FAIL (2025-08-08 05:05:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 427.806314ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 569.047191ms FAIL (2025-08-08 05:04:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 569.061535ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X.................................X..................] duration: 187.884561ms FAIL (2025-08-08 05:04:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 187.915027ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 521.690133ms FAIL (2025-08-08 05:03:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 521.710044ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X.................................X.................] duration: 429.18844ms FAIL (2025-08-08 05:03:18 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 429.221732ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 502.345197ms FAIL (2025-08-08 05:03:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 502.374936ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X.................................X................] duration: 596.005091ms FAIL (2025-08-08 05:02:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 596.047814ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 501.66533ms FAIL (2025-08-08 05:01:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 501.728035ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X.................................X...............] duration: 403.338083ms FAIL (2025-08-08 05:01:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 403.367394ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 843.386818ms FAIL (2025-08-08 05:01:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 843.409633ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X.................................X..............] duration: 222.937052ms FAIL (2025-08-08 05:00:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 222.966819ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 873.383301ms FAIL (2025-08-08 05:00:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 873.470129ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X.................................X.............] duration: 191.835023ms FAIL (2025-08-08 04:59:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 191.862508ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 733.113591ms FAIL (2025-08-08 04:59:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 733.137574ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 468.68288ms FAIL (2025-08-08 04:58:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 468.716284ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X.................................X............] duration: 200.207365ms FAIL (2025-08-08 04:58:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 200.233963ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X.................................X...........] duration: 236.496329ms FAIL (2025-08-08 04:58:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 236.590727ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.18371279s FAIL (2025-08-08 04:57:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.1837346s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X.................................X..........] duration: 195.064308ms FAIL (2025-08-08 04:57:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 195.097596ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 472.580717ms FAIL (2025-08-08 04:57:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.612515ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X.................................X.........] duration: 191.634469ms FAIL (2025-08-08 04:56:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 191.664746ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 543.902786ms FAIL (2025-08-08 04:55:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 543.931153ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X.................................X........] duration: 751.134245ms FAIL (2025-08-08 04:55:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 751.15658ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 480.250873ms FAIL (2025-08-08 04:54:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 480.278861ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 639.080683ms FAIL (2025-08-08 04:54:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 639.110467ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X.................................X.......] duration: 207.916406ms FAIL (2025-08-08 04:54:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 207.942257ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 475.226205ms FAIL (2025-08-08 04:53:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 475.245631ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X.................................X......] duration: 187.135645ms FAIL (2025-08-08 04:53:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 187.160354ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................X.................................X.....] duration: 191.45911ms FAIL (2025-08-08 04:53:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 191.487575ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 472.088232ms FAIL (2025-08-08 04:53:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.116918ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.................................X....] duration: 170.24473ms FAIL (2025-08-08 04:52:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 170.304087ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 485.429124ms FAIL (2025-08-08 04:52:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 485.461342ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................X.................................X...] duration: 185.249101ms FAIL (2025-08-08 04:51:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 185.279933ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001073176s FAIL (2025-08-08 04:51:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00109352s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.................................X..] duration: 336.145924ms FAIL (2025-08-08 04:51:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 336.176437ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X.................................X.] duration: 184.204849ms FAIL (2025-08-08 04:50:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 184.231927ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.................................X] duration: 206.814593ms FAIL (2025-08-08 04:49:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== uestItem_ObjectCommit Aug 8 04:42:16.180307652 (19h2m50.695410219s) [upload=19671]: Done uploading segments... Aug 8 04:42:16.180326351 (19h2m50.695428913s) [upload=19671]: Upload finished. err: <nil> Aug 8 04:42:41.083579476 (19h3m15.598682039s) [upload=19678]: Starting upload Aug 8 04:42:41.083658806 (19h3m15.598761384s) [upload=19678]: Done starting upload Aug 8 04:42:41.083736126 (19h3m15.598838696s) [upload=19678]: Uploading segments... Aug 8 04:42:41.083798735 (19h3m15.59890132s) [upload=19678]: Got next segment. Inline: true Aug 8 04:42:41.083805351 (19h3m15.598907934s) [upload=19678]: Waiting for error group managing segments... Aug 8 04:42:41.083894019 (19h3m15.598996595s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:42:41.083899702 (19h3m15.599002278s) [upload=19678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:42:41.083903543 (19h3m15.59900612s) [upload=19678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:42:41.147259315 (19h3m15.662361882s) [upload=19678]: Done uploading segments... Aug 8 04:42:41.147276213 (19h3m15.66237879s) [upload=19678]: Upload finished. err: <nil> Aug 8 04:43:54.314265905 (19h4m28.829368481s) [upload=19696]: Starting upload Aug 8 04:43:54.314342572 (19h4m28.829445149s) [upload=19696]: Done starting upload Aug 8 04:43:54.314412786 (19h4m28.829515355s) [upload=19696]: Uploading segments... Aug 8 04:43:54.314461737 (19h4m28.829564323s) [upload=19696]: Got next segment. Inline: true Aug 8 04:43:54.314468588 (19h4m28.829571166s) [upload=19696]: Waiting for error group managing segments... Aug 8 04:43:54.314513835 (19h4m28.829616411s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:43:54.314518357 (19h4m28.829620933s) [upload=19696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:43:54.314521699 (19h4m28.829624275s) [upload=19696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:43:54.374490000 (19h4m28.889592577s) [upload=19696]: Done uploading segments... Aug 8 04:43:54.374507640 (19h4m28.889610205s) [upload=19696]: Upload finished. err: <nil> Aug 8 04:44:37.820610246 (19h5m12.335712815s) [upload=19708]: Starting upload Aug 8 04:44:37.820681721 (19h5m12.335784288s) [upload=19708]: Done starting upload Aug 8 04:44:37.820750998 (19h5m12.335853567s) [upload=19708]: Uploading segments... Aug 8 04:44:37.820847572 (19h5m12.335950158s) [upload=19708]: Got next segment. Inline: true Aug 8 04:44:37.820871197 (19h5m12.335973778s) [upload=19708]: Waiting for error group managing segments... Aug 8 04:44:37.820923988 (19h5m12.336026577s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:44:37.820930149 (19h5m12.33603273s) [upload=19708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:44:37.820950094 (19h5m12.336052671s) [upload=19708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:44:37.891304392 (19h5m12.406406974s) [upload=19708]: Done uploading segments... Aug 8 04:44:37.891324719 (19h5m12.406427302s) [upload=19708]: Upload finished. err: <nil> Aug 8 04:45:49.374287050 (19h6m23.889389635s) [upload=19727]: Starting upload Aug 8 04:45:49.374385480 (19h6m23.889488062s) [upload=19727]: Done starting upload Aug 8 04:45:49.374485186 (19h6m23.889587763s) [upload=19727]: Uploading segments... Aug 8 04:45:49.374553702 (19h6m23.889656288s) [upload=19727]: Got next segment. Inline: true Aug 8 04:45:49.374559271 (19h6m23.889661849s) [upload=19727]: Waiting for error group managing segments... Aug 8 04:45:49.374607801 (19h6m23.889710384s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:45:49.374613021 (19h6m23.889715604s) [upload=19727]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:45:49.374616970 (19h6m23.889719552s) [upload=19727]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:45:49.440532379 (19h6m23.955634956s) [upload=19727]: Done uploading segments... Aug 8 04:45:49.440553942 (19h6m23.955656506s) [upload=19727]: Upload finished. err: <nil> Aug 8 04:46:22.148691181 (19h6m56.663793758s) [upload=19736]: Starting upload Aug 8 04:46:22.148731264 (19h6m56.66383385s) [upload=19736]: Done starting upload Aug 8 04:46:22.148779839 (19h6m56.663882416s) [upload=19736]: Uploading segments... Aug 8 04:46:22.148815522 (19h6m56.663918108s) [upload=19736]: Got next segment. Inline: true Aug 8 04:46:22.148819611 (19h6m56.663922186s) [upload=19736]: Waiting for error group managing segments... Aug 8 04:46:22.148853347 (19h6m56.663955925s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:22.148856905 (19h6m56.663959481s) [upload=19736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:22.148859219 (19h6m56.663961798s) [upload=19736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:22.224160590 (19h6m56.739263173s) [upload=19736]: Done uploading segments... Aug 8 04:46:22.224183380 (19h6m56.739285958s) [upload=19736]: Upload finished. err: <nil> Aug 8 04:46:57.599668727 (19h7m32.114771304s) [upload=19750]: Starting upload Aug 8 04:46:57.599733716 (19h7m32.114836292s) [upload=19750]: Done starting upload Aug 8 04:46:57.599831148 (19h7m32.114933716s) [upload=19750]: Uploading segments... Aug 8 04:46:57.599898194 (19h7m32.115000774s) [upload=19750]: Got next segment. Inline: true Aug 8 04:46:57.599902995 (19h7m32.115005571s) [upload=19750]: Waiting for error group managing segments... Aug 8 04:46:57.599955730 (19h7m32.115058297s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:46:57.599960210 (19h7m32.115062786s) [upload=19750]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:46:57.599963339 (19h7m32.115065916s) [upload=19750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:46:57.661947764 (19h7m32.17705034s) [upload=19750]: Done uploading segments... Aug 8 04:46:57.661964723 (19h7m32.1770673s) [upload=19750]: Upload finished. err: <nil> Aug 8 04:48:07.498218136 (19h8m42.013320702s) [upload=19766]: Starting upload Aug 8 04:48:07.498272608 (19h8m42.013375174s) [upload=19766]: Done starting upload Aug 8 04:48:07.498340076 (19h8m42.013442642s) [upload=19766]: Uploading segments... Aug 8 04:48:07.498377092 (19h8m42.013479678s) [upload=19766]: Got next segment. Inline: true Aug 8 04:48:07.498381514 (19h8m42.013484091s) [upload=19766]: Waiting for error group managing segments... Aug 8 04:48:07.498422115 (19h8m42.013524681s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:07.498426401 (19h8m42.013528979s) [upload=19766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:07.498429474 (19h8m42.01353205s) [upload=19766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:07.573834459 (19h8m42.088937036s) [upload=19766]: Done uploading segments... Aug 8 04:48:07.573849292 (19h8m42.088951861s) [upload=19766]: Upload finished. err: <nil> Aug 8 04:48:24.131081943 (19h8m58.646184521s) [upload=19773]: Starting upload Aug 8 04:48:24.131135304 (19h8m58.646237872s) [upload=19773]: Done starting upload Aug 8 04:48:24.131203832 (19h8m58.646306401s) [upload=19773]: Uploading segments... Aug 8 04:48:24.131242000 (19h8m58.646344585s) [upload=19773]: Got next segment. Inline: true Aug 8 04:48:24.131246356 (19h8m58.646348932s) [upload=19773]: Waiting for error group managing segments... Aug 8 04:48:24.131298903 (19h8m58.646401482s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:48:24.131304610 (19h8m58.646407189s) [upload=19773]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:48:24.131307703 (19h8m58.646410279s) [upload=19773]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:48:24.189165262 (19h8m58.704267839s) [upload=19773]: Done uploading segments... Aug 8 04:48:24.189182600 (19h8m58.704285176s) [upload=19773]: Upload finished. err: <nil> Aug 8 04:49:30.179743426 (19h10m4.69484601s) [upload=19792]: Starting upload Aug 8 04:49:30.179830624 (19h10m4.694933205s) [upload=19792]: Done starting upload Aug 8 04:49:30.179932276 (19h10m4.695034844s) [upload=19792]: Uploading segments... Aug 8 04:49:30.180011264 (19h10m4.69511385s) [upload=19792]: Got next segment. Inline: true Aug 8 04:49:30.180031508 (19h10m4.695134094s) [upload=19792]: Waiting for error group managing segments... Aug 8 04:49:30.180083780 (19h10m4.695186362s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:30.180089451 (19h10m4.695192035s) [upload=19792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:30.180107234 (19h10m4.695209811s) [upload=19792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:30.255066803 (19h10m4.770169372s) [upload=19792]: Done uploading segments... Aug 8 04:49:30.255097834 (19h10m4.770200412s) [upload=19792]: Upload finished. err: <nil> Aug 8 04:49:49.722537438 (19h10m24.237640014s) [upload=19800]: Starting upload Aug 8 04:49:49.722597342 (19h10m24.237699917s) [upload=19800]: Done starting upload Aug 8 04:49:49.722664052 (19h10m24.237766615s) [upload=19800]: Uploading segments... Aug 8 04:49:49.722715327 (19h10m24.23781791s) [upload=19800]: Got next segment. Inline: true Aug 8 04:49:49.722719940 (19h10m24.237822515s) [upload=19800]: Waiting for error group managing segments... Aug 8 04:49:49.722753934 (19h10m24.23785651s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:49:49.722758369 (19h10m24.237860931s) [upload=19800]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:49:49.722815656 (19h10m24.237918234s) [upload=19800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:49:49.824454883 (19h10m24.339557452s) [upload=19800]: Done uploading segments... Aug 8 04:49:49.824470680 (19h10m24.339573247s) [upload=19800]: Upload finished. err: <nil> Duration: 206.868263ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 204.739026ms FAIL (2025-08-08 04:49:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 56a7129b8094d798f6b9adf4d1e343d7ca2c5eccb926d49e3ad3649c93dba5b9 != before 6e0a27ad414b200d077ec4d8c3bf3d61b8f5d661de6ee62df5d306f0dbfcf3d8 ======LOGS====== ]: Done uploading segments... Aug 8 04:14:10.999307357 (18h34m45.514409933s) [upload=19182]: Upload finished. err: <nil> Aug 8 04:15:17.521964380 (18h35m52.037066956s) [upload=19202]: Starting upload Aug 8 04:15:17.522074183 (18h35m52.03717675s) [upload=19202]: Done starting upload Aug 8 04:15:17.522253376 (18h35m52.037355941s) [upload=19202]: Uploading segments... Aug 8 04:15:17.522437932 (18h35m52.037540519s) [upload=19202]: Got next segment. Inline: true Aug 8 04:15:17.522454641 (18h35m52.037557207s) [upload=19202]: Waiting for error group managing segments... Aug 8 04:15:17.522564962 (18h35m52.037667542s) [upload=19202]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:15:17.522570764 (18h35m52.03767334s) [upload=19202]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:15:17.522574164 (18h35m52.03767674s) [upload=19202]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:15:17.633629757 (18h35m52.148732325s) [upload=19202]: Done uploading segments... Aug 8 04:15:17.633646271 (18h35m52.148748847s) [upload=19202]: Upload finished. err: <nil> Aug 8 04:16:02.443190659 (18h36m36.958293238s) [upload=19217]: Starting upload Aug 8 04:16:02.443249545 (18h36m36.958352132s) [upload=19217]: Done starting upload Aug 8 04:16:02.443315574 (18h36m36.958418138s) [upload=19217]: Uploading segments... Aug 8 04:16:02.443359902 (18h36m36.958462489s) [upload=19217]: Got next segment. Inline: true Aug 8 04:16:02.443365091 (18h36m36.958467674s) [upload=19217]: Waiting for error group managing segments... Aug 8 04:16:02.443410418 (18h36m36.958512998s) [upload=19217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:16:02.443415187 (18h36m36.958517767s) [upload=19217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:16:02.443418617 (18h36m36.9585212s) [upload=19217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:16:02.516989645 (18h36m37.032092226s) [upload=19217]: Done uploading segments... Aug 8 04:16:02.517010672 (18h36m37.032113243s) [upload=19217]: Upload finished. err: <nil> Aug 8 04:17:11.242989514 (18h37m45.758092094s) [upload=19237]: Starting upload Aug 8 04:17:11.243059289 (18h37m45.758161868s) [upload=19237]: Done starting upload Aug 8 04:17:11.243149042 (18h37m45.758251607s) [upload=19237]: Uploading segments... Aug 8 04:17:11.243193403 (18h37m45.758295988s) [upload=19237]: Got next segment. Inline: true Aug 8 04:17:11.243198709 (18h37m45.758301288s) [upload=19237]: Waiting for error group managing segments... Aug 8 04:17:11.243246855 (18h37m45.758349434s) [upload=19237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:17:11.243252063 (18h37m45.758354639s) [upload=19237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:17:11.243255475 (18h37m45.758358051s) [upload=19237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:17:11.309718075 (18h37m45.824820651s) [upload=19237]: Done uploading segments... Aug 8 04:17:11.309736445 (18h37m45.824839012s) [upload=19237]: Upload finished. err: <nil> Aug 8 04:18:04.159872362 (18h38m38.674974929s) [upload=19252]: Starting upload Aug 8 04:18:04.159942743 (18h38m38.675045321s) [upload=19252]: Done starting upload Aug 8 04:18:04.160028720 (18h38m38.675131287s) [upload=19252]: Uploading segments... Aug 8 04:18:04.160094115 (18h38m38.675196696s) [upload=19252]: Got next segment. Inline: true Aug 8 04:18:04.160099201 (18h38m38.675201778s) [upload=19252]: Waiting for error group managing segments... Aug 8 04:18:04.160144961 (18h38m38.675247537s) [upload=19252]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:18:04.160163411 (18h38m38.675265988s) [upload=19252]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:18:04.160209450 (18h38m38.675312031s) [upload=19252]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:18:04.220753406 (18h38m38.73585599s) [upload=19252]: Done uploading segments... Aug 8 04:18:04.220803891 (18h38m38.735906471s) [upload=19252]: Upload finished. err: <nil> Aug 8 04:18:55.132226896 (18h39m29.647329481s) [upload=19264]: Starting upload Aug 8 04:18:55.132295630 (18h39m29.647398206s) [upload=19264]: Done starting upload Aug 8 04:18:55.132379135 (18h39m29.647481701s) [upload=19264]: Uploading segments... Aug 8 04:18:55.132421896 (18h39m29.647524482s) [upload=19264]: Got next segment. Inline: true Aug 8 04:18:55.132426929 (18h39m29.647529506s) [upload=19264]: Waiting for error group managing segments... Aug 8 04:18:55.132465898 (18h39m29.647568474s) [upload=19264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:18:55.132471069 (18h39m29.647573645s) [upload=19264]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:18:55.132474554 (18h39m29.647577131s) [upload=19264]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:18:55.186568062 (18h39m29.701670647s) [upload=19264]: Done uploading segments... Aug 8 04:18:55.186594154 (18h39m29.701696734s) [upload=19264]: Upload finished. err: <nil> Aug 8 04:20:04.047614857 (18h40m38.562717438s) [upload=19287]: Starting upload Aug 8 04:20:04.047687566 (18h40m38.562790145s) [upload=19287]: Done starting upload Aug 8 04:20:04.047759610 (18h40m38.562862179s) [upload=19287]: Uploading segments... Aug 8 04:20:04.047809540 (18h40m38.56291213s) [upload=19287]: Got next segment. Inline: true Aug 8 04:20:04.047815214 (18h40m38.562917795s) [upload=19287]: Waiting for error group managing segments... Aug 8 04:20:04.047869869 (18h40m38.562972456s) [upload=19287]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:20:04.047876808 (18h40m38.562979395s) [upload=19287]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:20:04.047881583 (18h40m38.562984172s) [upload=19287]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:20:04.121014295 (18h40m38.636116887s) [upload=19287]: Done uploading segments... Aug 8 04:20:04.121034900 (18h40m38.636137478s) [upload=19287]: Upload finished. err: <nil> Aug 8 04:20:50.282636646 (18h41m24.797739242s) [upload=19297]: Starting upload Aug 8 04:20:50.282705354 (18h41m24.797807937s) [upload=19297]: Done starting upload Aug 8 04:20:50.282816851 (18h41m24.79791943s) [upload=19297]: Uploading segments... Aug 8 04:20:50.282901794 (18h41m24.79800438s) [upload=19297]: Got next segment. Inline: true Aug 8 04:20:50.282909128 (18h41m24.798011713s) [upload=19297]: Waiting for error group managing segments... Aug 8 04:20:50.282978447 (18h41m24.79808103s) [upload=19297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:20:50.282984983 (18h41m24.798087563s) [upload=19297]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:20:50.282990448 (18h41m24.798093036s) [upload=19297]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:20:50.342487201 (18h41m24.857589778s) [upload=19297]: Done uploading segments... Aug 8 04:20:50.342503986 (18h41m24.857606552s) [upload=19297]: Upload finished. err: <nil> Aug 8 04:21:46.493263033 (18h42m21.008365602s) [upload=19319]: Starting upload Aug 8 04:21:46.493347060 (18h42m21.008449628s) [upload=19319]: Done starting upload Aug 8 04:21:46.493476213 (18h42m21.008578796s) [upload=19319]: Uploading segments... Aug 8 04:21:46.493590019 (18h42m21.008692596s) [upload=19319]: Got next segment. Inline: true Aug 8 04:21:46.493596327 (18h42m21.008698904s) [upload=19319]: Waiting for error group managing segments... Aug 8 04:21:46.493671151 (18h42m21.008773729s) [upload=19319]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:21:46.493677303 (18h42m21.008779886s) [upload=19319]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:21:46.493680833 (18h42m21.008783414s) [upload=19319]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:21:46.581182258 (18h42m21.096284836s) [upload=19319]: Done uploading segments... Aug 8 04:21:46.581202826 (18h42m21.096305398s) [upload=19319]: Upload finished. err: <nil> Aug 8 04:22:43.655602697 (18h43m18.170705284s) [upload=19330]: Starting upload Aug 8 04:22:43.655671575 (18h43m18.170774141s) [upload=19330]: Done starting upload Aug 8 04:22:43.655750772 (18h43m18.17085334s) [upload=19330]: Uploading segments... Aug 8 04:22:43.655823942 (18h43m18.170926541s) [upload=19330]: Got next segment. Inline: true Aug 8 04:22:43.655831454 (18h43m18.170934038s) [upload=19330]: Waiting for error group managing segments... Aug 8 04:22:43.655911087 (18h43m18.171013659s) [upload=19330]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:22:43.655917536 (18h43m18.171020106s) [upload=19330]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:22:43.655920811 (18h43m18.171023389s) [upload=19330]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:22:43.753130054 (18h43m18.268232647s) [upload=19330]: Done uploading segments... Aug 8 04:22:43.753151314 (18h43m18.26825389s) [upload=19330]: Upload finished. err: <nil> Aug 8 04:23:30.296555997 (18h44m4.811658568s) [upload=19347]: Starting upload Aug 8 04:23:30.296625817 (18h44m4.811728393s) [upload=19347]: Done starting upload Aug 8 04:23:30.296688473 (18h44m4.811791039s) [upload=19347]: Uploading segments... Aug 8 04:23:30.296726759 (18h44m4.811829344s) [upload=19347]: Got next segment. Inline: true Aug 8 04:23:30.296731122 (18h44m4.811833699s) [upload=19347]: Waiting for error group managing segments... Aug 8 04:23:30.296772865 (18h44m4.811875443s) [upload=19347]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 04:23:30.296778081 (18h44m4.811880659s) [upload=19347]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 04:23:30.296782307 (18h44m4.811884885s) [upload=19347]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 04:23:35.471075270 (18h44m9.986177835s) [upload=19347]: Done uploading segments... Aug 8 04:23:35.471092992 (18h44m9.986195559s) [upload=19347]: Upload finished. err: <nil> Duration: 204.767842ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)