Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X....X.................] duration:
1.894978634s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:18:55 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 1.895013156s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................X...............................] duration:
886.955379ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:18:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 886.984328ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................X..............................] duration:
876.475558ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:18:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 876.493243ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......................................X....X................] duration:
1.955967453s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:17:48 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 1.95600038s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................X.............................] duration:
738.994781ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:17:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 739.023884ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X....X...............] duration:
2.080965697s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:16:53 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.080995876s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [........................................X....X..............] duration:
2.113187526s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:16:37 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.11321303s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................X............................] duration:
833.788808ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:16:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 833.825375ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X....X.............] duration:
2.177935537s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:15:57 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.177971153s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................X...........................] duration:
873.298839ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:15:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 873.330988ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X....X............] duration:
2.27652185s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:15:03 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.276569428s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................X..........................] duration:
863.642574ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:15:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 863.738209ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X....X...........] duration:
1.90767866s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:14:16 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 1.907709332s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................X.........................] duration:
919.651917ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:13:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 919.71162ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [............................................X....X..........] duration:
2.061881732s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:13:48 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.06191136s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................X........................] duration:
841.528558ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:13:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 841.579524ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X....X.........] duration:
2.20572754s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:12:35 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.205775603s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................X.......................] duration:
891.560235ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:12:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 891.630334ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X....X........] duration:
2.160071799s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:12:07 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.16010689s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................X......................] duration:
789.540951ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:11:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 790.930244ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X....X.......] duration:
4.685126475s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:11:26 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 4.685160183s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [................................................X....X......] duration:
21.881386541s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:11:17 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 21.881492012s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................X.....................] duration:
770.98847ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:11:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 771.024816ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................X....................] duration:
805.316373ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:10:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 805.347522ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................X...................] duration:
797.984595ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:10:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 798.048682ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X....X.....] duration:
2.370349493s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:10:13 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.370388025s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X....X....] duration:
1.963616306s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:09:54 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 1.963653106s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................X..................] duration:
879.502411ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:09:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 879.527083ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X....X...] duration:
2.203628951s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:09:01 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.20365587s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X....X..] duration:
26.971748328s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:08:43 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 26.971784633s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................X.................] duration:
766.067382ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:08:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 766.094569ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................X................] duration:
818.741789ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:07:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 818.863305ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration:
2.041391556s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:07:10 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 2.041613659s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......................................................X....X] duration:
1m0.00409846s trace:
https://jaeger.dev.storj.io/trace/5e778fe7a13b60a0
FAIL (2026-05-25 11:07:08 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.096568326 (592h17m50.268686755s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097112562 (592h17m50.269230992s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.097632500 (592h17m50.269750931s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098148476 (592h17m50.270266905s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.098699871 (592h17m50.270818304s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099313777 (592h17m50.271432207s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.099840133 (592h17m50.271958563s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100162049 (592h17m50.272280479s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.100394430 (592h17m50.27251286s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101165405 (592h17m50.273283838s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101833189 (592h17m50.273951618s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.101859975 (592h17m50.273978416s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.102501215 (592h17m50.274619646s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103484544 (592h17m50.275602964s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.103611564 (592h17m50.275729993s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.104895143 (592h17m50.277013568s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.105575195 (592h17m50.277693619s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Uploading piece...
May 25 11:06:38.617322517 (592h17m50.789440943s) [upload=663150]: Next returned nil segment
May 25 11:06:38.617345221 (592h17m50.789463641s) [upload=663150]: Waiting for error group managing segments...
May 25 11:06:38.720245841 (592h17m50.892364266s) [noise=true , address=212.102.52.215:4374 , piece_id=VO2KM6U3AU3BGKVSEXBTI2775GZ4KSZYUZQ5HKQXLP3V7UEQ3ASQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.722028135 (592h17m50.894146559s) [noise=true , address=212.102.52.216:4397 , piece_id=TAHVCE4K2ALUN62UFFOYCAF7VE6OTUEVL7GNBIQNBR2SVT5ODY3Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.723587957 (592h17m50.895706388s) [noise=true , address=212.102.52.213:4339 , piece_id=JATZ2POQ4OP5U6ELUQNV6CAAMJE3633TOMQ3IXZJYTPMTJAGJE3A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.724810236 (592h17m50.89692866s) [noise=true , address=212.102.52.217:4438 , piece_id=6ICU5AYURZ2UM3CCC4ECAMMZX3W3JTBSMUIMGQNVMQVAO7BMKXRA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.725946605 (592h17m50.898065029s) [noise=true , address=212.102.52.217:4430 , piece_id=56MEVCZCL42LAB2H4P5DQ2E5DBNAQDG4BWR356ATM3YRSTHQ5ZHA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.728488369 (592h17m50.900606793s) [noise=true , address=212.102.52.217:4429 , piece_id=3O53KNSNQ2CJCKE57AK67YSWISWCGG7UNMPWACOSZDITSOLSBLWQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.733448559 (592h17m50.905567172s) [noise=true , address=212.102.52.214:4361 , piece_id=MZE3SCRQXKRMTPAHSR6A7RWTD4D477ZJRN3ZSI4IO4HHIJBX6XWA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.734401330 (592h17m50.906519754s) [noise=true , address=212.102.52.216:4406 , piece_id=F3XU7ZHGNOXMWKELLARRXHKQM3W6UGHR7YTWZJJJY7RVK47B7PCA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.735527451 (592h17m50.90764588s) [noise=true , address=212.102.52.215:4390 , piece_id=QX3AZASWXGRINW3R5PM4YSFIVB67JUU5M5PENMDMGTOF4JYHTB6Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.738774235 (592h17m50.910892658s) [noise=true , address=212.102.52.215:4378 , piece_id=WT2PJKH7UJLVGOD3MYKLZ3DFWGNUU6VKK7YURZDV4NCG4DUMO2MA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739476480 (592h17m50.911594889s) [noise=true , address=212.102.52.213:4332 , piece_id=KX7QRN6HEFVA45Q3M6PBGAR2PZPBHN2VMANQ4NGKNMWLFBFIIWOQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.739896594 (592h17m50.91201503s) [noise=true , address=212.102.52.213:4343 , piece_id=6TOF22V4QQZQQMXUNF4DZONHDLVSONSTYAQSQYJAYCVUGEF5OIXQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740257541 (592h17m50.912375965s) [noise=true , address=212.102.52.214:4363 , piece_id=5HDNLRWJB52UX52BWG5JUPBQUJIQXHYKEL2L6X2FP5FE3S3PA27Q, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.740598623 (592h17m50.912717043s) [noise=true , address=212.102.52.214:4364 , piece_id=M5OBC7WVS6RLFRVMCRZPEGWETTIGYQKVS7MOBFSFWNK5TNUFQLUA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741254827 (592h17m50.913373247s) [noise=true , address=212.102.52.217:4431 , piece_id=YZRB2E5L3AS2OKRD56U73BJV67INEK4JPANK3ZBENUZI7TCWNXDA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741939503 (592h17m50.914057909s) [noise=true , address=212.102.52.216:4411 , piece_id=LPJTMKTEEPF3Y4G6WSQ422DSQ53CQMKP5R4FBO6IQIZGPES3GDZQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.741954456 (592h17m50.914072881s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 16 pieces
May 25 11:06:38.742266991 (592h17m50.914385423s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16y12CqwtxrbMoBweh49rb4ZcgKgQqCNn3yApCJLQo3hJPBsoy): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.742278868 (592h17m50.914397275s) [noise=true , address=212.102.52.213:4326 , piece_id=H4SACSGFRP356NBXYLTMZEASDXJJBS6CWS44TXRIJRPRJM5BV7SQ, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.742882211 (592h17m50.915000634s) [noise=true , address=212.102.52.214:4358 , piece_id=HFPKCA3HYHFBXGLBPSZOGXZUYV4RWE5AP7KZ7FOMYPOUVXUMVJQA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.742899489 (592h17m50.915017913s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 17 pieces
May 25 11:06:38.743560409 (592h17m50.915678829s) [noise=true , address=212.102.52.215:4388 , piece_id=ZFKJUACOPRCDKD3MRTVNBL5N4WR3PUOM2FGRZQQKOSC6FYSQPJ4A, seg_pos={0 0}, upload=663150]: Done uploading piece. err: <nil>
May 25 11:06:38.743574497 (592h17m50.915692921s) [seg_pos={0 0}, upload=663150]: Satisfied cohort requirements after 18 pieces
May 25 11:06:38.747671019 (592h17m50.919789675s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 11:06:38.747692770 (592h17m50.919811199s) [noise=true , address=212.102.52.216:4416 , piece_id=NFP6MYIKDIO76Q25VRQHOKVS6KGW6QQI5E5WHITEGBSFM74BYGTA, seg_pos={0 0}, upload=663150]: Long tail cancellation due to optimal threshold
May 25 11:06:38.747725328 (592h17m50.919843735s) [seg_pos={0 0}, upload=663150]: Done waiting for segment. successful: 18 optimal: 16 errs: <nil>
May 25 11:06:38.747827383 (592h17m50.919946043s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:06:38.747874944 (592h17m50.919993376s) [upload=663150]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:06:45.559242654 (592h17m57.731361113s) [upload=663150]: Done uploading segments...
May 25 11:06:45.559267229 (592h17m57.731385653s) [upload=663150]: Upload finished. err: <nil>
Duration: 1m0.004394687s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................................X...............] duration:
844.887066ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:06:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 844.974599ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................................X..............] duration:
864.002472ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:06:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 864.031621ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................................X.............] duration:
855.689981ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:05:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 855.718127ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
23.583459841s trace:
https://jaeger.dev.storj.io/trace/6703bf2b127ea52f
FAIL (2026-05-25 11:05:37 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 11:02:47.489531092 (592h13m59.661649515s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.489912686 (592h13m59.662031107s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490414636 (592h13m59.662533057s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490860469 (592h13m59.662978893s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491303060 (592h13m59.663421482s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491638732 (592h13m59.663757152s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492031688 (592h13m59.664150109s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492404276 (592h13m59.664522699s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492745464 (592h13m59.664863887s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493277626 (592h13m59.665396047s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493863704 (592h13m59.665982123s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494309840 (592h13m59.666428261s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494811214 (592h13m59.666929636s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.505463475 (592h13m59.677581894s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.509323012 (592h13m59.681441454s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.986855613 (592h14m0.158974038s) [upload=663070]: Next returned nil segment
May 25 11:02:47.986876978 (592h14m0.158995404s) [upload=663070]: Waiting for error group managing segments...
May 25 11:02:48.120803830 (592h14m0.292922265s) [noise=true , address=212.102.52.217:4429 , piece_id=HCICITGRIITNGXHYITW7UOJ3NBFQV2UTNAYRL72B554W7KT2U5AA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.120921133 (592h14m0.293039547s) [noise=true , address=212.102.52.216:4408 , piece_id=J2SITJLVU2DUEDQRKL3ENJVQ23BIG4P66QDJHN4FWB3KKSTCVR4Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.121834891 (592h14m0.293953318s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.122003384 (592h14m0.294121814s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125376336 (592h14m0.297494759s) [noise=true , address=212.102.52.214:4366 , piece_id=NMY45QRLPRUISZVURZNDGABGSORYYM2BLUNBTUMB5O5TBNOKCHDQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125904956 (592h14m0.29802338s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126321446 (592h14m0.298439872s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126641497 (592h14m0.298759917s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141143829 (592h14m0.313262252s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141515346 (592h14m0.313633788s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143185356 (592h14m0.315303777s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143284527 (592h14m0.315402937s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.144423996 (592h14m0.31654242s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.145527980 (592h14m0.31764639s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146469812 (592h14m0.31858822s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146794204 (592h14m0.318912613s) [noise=true , address=212.102.52.214:4364 , piece_id=KPYEPDHBSOPGHSNAZEOBBWR46O3GN3FMP7NFGPABKL7F32JEXW6Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146810158 (592h14m0.318928569s) [seg_pos={0 0}, upload=663070]: Satisfied cohort requirements after 16 pieces
May 25 11:02:48.147109800 (592h14m0.319228228s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ey8Bngj9b4FsVddkt7Gy54obDPDoerL8sLAZBmKekPejXhg5M): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147119424 (592h14m0.319237846s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147149753 (592h14m0.319268181s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12259awyjTZXky86DZr2q8ak5V5KQz6f9pWiTbR6KtnS8rhqDDo): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147162311 (592h14m0.319280739s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147383047 (592h14m0.319501477s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RQtuAbsJqQWMogK8y4ZNo6HY4Q3ffS2v6Y2RyPz267WPMxCE8): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147391126 (592h14m0.319509553s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148355954 (592h14m0.320474387s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.148370069 (592h14m0.3204885s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148405711 (592h14m0.320524121s) [seg_pos={0 0}, upload=663070]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 11:02:48.148515177 (592h14m0.320633601s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:02:48.148521550 (592h14m0.320639971s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:02:51.545270058 (592h14m3.71738849s) [upload=663070]: Done uploading segments...
May 25 11:02:51.545296658 (592h14m3.717415086s) [upload=663070]: Upload finished. err: <nil>
Duration: 23.583563996s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
957.310755ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:04:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 957.341132ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.015849363s trace:
https://jaeger.dev.storj.io/trace/6703bf2b127ea52f
FAIL (2026-05-25 11:04:25 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 11:02:47.489531092 (592h13m59.661649515s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.489912686 (592h13m59.662031107s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490414636 (592h13m59.662533057s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490860469 (592h13m59.662978893s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491303060 (592h13m59.663421482s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491638732 (592h13m59.663757152s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492031688 (592h13m59.664150109s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492404276 (592h13m59.664522699s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492745464 (592h13m59.664863887s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493277626 (592h13m59.665396047s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493863704 (592h13m59.665982123s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494309840 (592h13m59.666428261s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494811214 (592h13m59.666929636s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.505463475 (592h13m59.677581894s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.509323012 (592h13m59.681441454s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.986855613 (592h14m0.158974038s) [upload=663070]: Next returned nil segment
May 25 11:02:47.986876978 (592h14m0.158995404s) [upload=663070]: Waiting for error group managing segments...
May 25 11:02:48.120803830 (592h14m0.292922265s) [noise=true , address=212.102.52.217:4429 , piece_id=HCICITGRIITNGXHYITW7UOJ3NBFQV2UTNAYRL72B554W7KT2U5AA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.120921133 (592h14m0.293039547s) [noise=true , address=212.102.52.216:4408 , piece_id=J2SITJLVU2DUEDQRKL3ENJVQ23BIG4P66QDJHN4FWB3KKSTCVR4Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.121834891 (592h14m0.293953318s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.122003384 (592h14m0.294121814s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125376336 (592h14m0.297494759s) [noise=true , address=212.102.52.214:4366 , piece_id=NMY45QRLPRUISZVURZNDGABGSORYYM2BLUNBTUMB5O5TBNOKCHDQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125904956 (592h14m0.29802338s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126321446 (592h14m0.298439872s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126641497 (592h14m0.298759917s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141143829 (592h14m0.313262252s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141515346 (592h14m0.313633788s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143185356 (592h14m0.315303777s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143284527 (592h14m0.315402937s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.144423996 (592h14m0.31654242s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.145527980 (592h14m0.31764639s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146469812 (592h14m0.31858822s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146794204 (592h14m0.318912613s) [noise=true , address=212.102.52.214:4364 , piece_id=KPYEPDHBSOPGHSNAZEOBBWR46O3GN3FMP7NFGPABKL7F32JEXW6Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146810158 (592h14m0.318928569s) [seg_pos={0 0}, upload=663070]: Satisfied cohort requirements after 16 pieces
May 25 11:02:48.147109800 (592h14m0.319228228s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ey8Bngj9b4FsVddkt7Gy54obDPDoerL8sLAZBmKekPejXhg5M): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147119424 (592h14m0.319237846s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147149753 (592h14m0.319268181s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12259awyjTZXky86DZr2q8ak5V5KQz6f9pWiTbR6KtnS8rhqDDo): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147162311 (592h14m0.319280739s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147383047 (592h14m0.319501477s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RQtuAbsJqQWMogK8y4ZNo6HY4Q3ffS2v6Y2RyPz267WPMxCE8): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147391126 (592h14m0.319509553s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148355954 (592h14m0.320474387s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.148370069 (592h14m0.3204885s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148405711 (592h14m0.320524121s) [seg_pos={0 0}, upload=663070]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 11:02:48.148515177 (592h14m0.320633601s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:02:48.148521550 (592h14m0.320639971s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:02:51.545270058 (592h14m3.71738849s) [upload=663070]: Done uploading segments...
May 25 11:02:51.545296658 (592h14m3.717415086s) [upload=663070]: Upload finished. err: <nil>
Duration: 2.015882989s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.972495553s trace:
https://jaeger.dev.storj.io/trace/6703bf2b127ea52f
FAIL (2026-05-25 11:03:34 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 11:02:47.489531092 (592h13m59.661649515s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.489912686 (592h13m59.662031107s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490414636 (592h13m59.662533057s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490860469 (592h13m59.662978893s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491303060 (592h13m59.663421482s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491638732 (592h13m59.663757152s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492031688 (592h13m59.664150109s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492404276 (592h13m59.664522699s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492745464 (592h13m59.664863887s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493277626 (592h13m59.665396047s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493863704 (592h13m59.665982123s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494309840 (592h13m59.666428261s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494811214 (592h13m59.666929636s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.505463475 (592h13m59.677581894s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.509323012 (592h13m59.681441454s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.986855613 (592h14m0.158974038s) [upload=663070]: Next returned nil segment
May 25 11:02:47.986876978 (592h14m0.158995404s) [upload=663070]: Waiting for error group managing segments...
May 25 11:02:48.120803830 (592h14m0.292922265s) [noise=true , address=212.102.52.217:4429 , piece_id=HCICITGRIITNGXHYITW7UOJ3NBFQV2UTNAYRL72B554W7KT2U5AA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.120921133 (592h14m0.293039547s) [noise=true , address=212.102.52.216:4408 , piece_id=J2SITJLVU2DUEDQRKL3ENJVQ23BIG4P66QDJHN4FWB3KKSTCVR4Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.121834891 (592h14m0.293953318s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.122003384 (592h14m0.294121814s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125376336 (592h14m0.297494759s) [noise=true , address=212.102.52.214:4366 , piece_id=NMY45QRLPRUISZVURZNDGABGSORYYM2BLUNBTUMB5O5TBNOKCHDQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125904956 (592h14m0.29802338s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126321446 (592h14m0.298439872s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126641497 (592h14m0.298759917s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141143829 (592h14m0.313262252s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141515346 (592h14m0.313633788s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143185356 (592h14m0.315303777s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143284527 (592h14m0.315402937s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.144423996 (592h14m0.31654242s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.145527980 (592h14m0.31764639s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146469812 (592h14m0.31858822s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146794204 (592h14m0.318912613s) [noise=true , address=212.102.52.214:4364 , piece_id=KPYEPDHBSOPGHSNAZEOBBWR46O3GN3FMP7NFGPABKL7F32JEXW6Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146810158 (592h14m0.318928569s) [seg_pos={0 0}, upload=663070]: Satisfied cohort requirements after 16 pieces
May 25 11:02:48.147109800 (592h14m0.319228228s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ey8Bngj9b4FsVddkt7Gy54obDPDoerL8sLAZBmKekPejXhg5M): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147119424 (592h14m0.319237846s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147149753 (592h14m0.319268181s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12259awyjTZXky86DZr2q8ak5V5KQz6f9pWiTbR6KtnS8rhqDDo): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147162311 (592h14m0.319280739s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147383047 (592h14m0.319501477s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RQtuAbsJqQWMogK8y4ZNo6HY4Q3ffS2v6Y2RyPz267WPMxCE8): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147391126 (592h14m0.319509553s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148355954 (592h14m0.320474387s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.148370069 (592h14m0.3204885s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148405711 (592h14m0.320524121s) [seg_pos={0 0}, upload=663070]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 11:02:48.148515177 (592h14m0.320633601s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:02:48.148521550 (592h14m0.320639971s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:02:51.545270058 (592h14m3.71738849s) [upload=663070]: Done uploading segments...
May 25 11:02:51.545296658 (592h14m3.717415086s) [upload=663070]: Upload finished. err: <nil>
Duration: 1.972527369s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
812.125225ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:03:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 812.18212ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.018272814s trace:
https://jaeger.dev.storj.io/trace/6703bf2b127ea52f
FAIL (2026-05-25 11:03:11 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 11:02:47.489531092 (592h13m59.661649515s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.489912686 (592h13m59.662031107s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490414636 (592h13m59.662533057s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490860469 (592h13m59.662978893s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491303060 (592h13m59.663421482s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491638732 (592h13m59.663757152s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492031688 (592h13m59.664150109s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492404276 (592h13m59.664522699s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492745464 (592h13m59.664863887s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493277626 (592h13m59.665396047s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493863704 (592h13m59.665982123s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494309840 (592h13m59.666428261s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494811214 (592h13m59.666929636s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.505463475 (592h13m59.677581894s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.509323012 (592h13m59.681441454s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.986855613 (592h14m0.158974038s) [upload=663070]: Next returned nil segment
May 25 11:02:47.986876978 (592h14m0.158995404s) [upload=663070]: Waiting for error group managing segments...
May 25 11:02:48.120803830 (592h14m0.292922265s) [noise=true , address=212.102.52.217:4429 , piece_id=HCICITGRIITNGXHYITW7UOJ3NBFQV2UTNAYRL72B554W7KT2U5AA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.120921133 (592h14m0.293039547s) [noise=true , address=212.102.52.216:4408 , piece_id=J2SITJLVU2DUEDQRKL3ENJVQ23BIG4P66QDJHN4FWB3KKSTCVR4Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.121834891 (592h14m0.293953318s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.122003384 (592h14m0.294121814s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125376336 (592h14m0.297494759s) [noise=true , address=212.102.52.214:4366 , piece_id=NMY45QRLPRUISZVURZNDGABGSORYYM2BLUNBTUMB5O5TBNOKCHDQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125904956 (592h14m0.29802338s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126321446 (592h14m0.298439872s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126641497 (592h14m0.298759917s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141143829 (592h14m0.313262252s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141515346 (592h14m0.313633788s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143185356 (592h14m0.315303777s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143284527 (592h14m0.315402937s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.144423996 (592h14m0.31654242s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.145527980 (592h14m0.31764639s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146469812 (592h14m0.31858822s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146794204 (592h14m0.318912613s) [noise=true , address=212.102.52.214:4364 , piece_id=KPYEPDHBSOPGHSNAZEOBBWR46O3GN3FMP7NFGPABKL7F32JEXW6Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146810158 (592h14m0.318928569s) [seg_pos={0 0}, upload=663070]: Satisfied cohort requirements after 16 pieces
May 25 11:02:48.147109800 (592h14m0.319228228s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ey8Bngj9b4FsVddkt7Gy54obDPDoerL8sLAZBmKekPejXhg5M): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147119424 (592h14m0.319237846s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147149753 (592h14m0.319268181s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12259awyjTZXky86DZr2q8ak5V5KQz6f9pWiTbR6KtnS8rhqDDo): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147162311 (592h14m0.319280739s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147383047 (592h14m0.319501477s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RQtuAbsJqQWMogK8y4ZNo6HY4Q3ffS2v6Y2RyPz267WPMxCE8): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147391126 (592h14m0.319509553s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148355954 (592h14m0.320474387s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.148370069 (592h14m0.3204885s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148405711 (592h14m0.320524121s) [seg_pos={0 0}, upload=663070]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 11:02:48.148515177 (592h14m0.320633601s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:02:48.148521550 (592h14m0.320639971s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:02:51.545270058 (592h14m3.71738849s) [upload=663070]: Done uploading segments...
May 25 11:02:51.545296658 (592h14m3.717415086s) [upload=663070]: Upload finished. err: <nil>
Duration: 2.018305598s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.00305109s trace:
https://jaeger.dev.storj.io/trace/6703bf2b127ea52f
FAIL (2026-05-25 11:03:07 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 11:02:47.489531092 (592h13m59.661649515s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.489912686 (592h13m59.662031107s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490414636 (592h13m59.662533057s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.490860469 (592h13m59.662978893s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491303060 (592h13m59.663421482s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.491638732 (592h13m59.663757152s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492031688 (592h13m59.664150109s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492404276 (592h13m59.664522699s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.492745464 (592h13m59.664863887s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493277626 (592h13m59.665396047s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.493863704 (592h13m59.665982123s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494309840 (592h13m59.666428261s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.494811214 (592h13m59.666929636s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.505463475 (592h13m59.677581894s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.509323012 (592h13m59.681441454s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Uploading piece...
May 25 11:02:47.986855613 (592h14m0.158974038s) [upload=663070]: Next returned nil segment
May 25 11:02:47.986876978 (592h14m0.158995404s) [upload=663070]: Waiting for error group managing segments...
May 25 11:02:48.120803830 (592h14m0.292922265s) [noise=true , address=212.102.52.217:4429 , piece_id=HCICITGRIITNGXHYITW7UOJ3NBFQV2UTNAYRL72B554W7KT2U5AA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.120921133 (592h14m0.293039547s) [noise=true , address=212.102.52.216:4408 , piece_id=J2SITJLVU2DUEDQRKL3ENJVQ23BIG4P66QDJHN4FWB3KKSTCVR4Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.121834891 (592h14m0.293953318s) [noise=true , address=212.102.52.216:4406 , piece_id=A6XOY3FAXT3Y64H6D34UGHRBWHVCTCJHKZUSIAHFSQLZTF5WRZ7Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.122003384 (592h14m0.294121814s) [noise=true , address=212.102.52.216:4396 , piece_id=MB5GB43VNJU3SOYLXYVP2FDYPCAZ2QNH7XA7EO6DKXX3SVAYG6IQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125376336 (592h14m0.297494759s) [noise=true , address=212.102.52.214:4366 , piece_id=NMY45QRLPRUISZVURZNDGABGSORYYM2BLUNBTUMB5O5TBNOKCHDQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.125904956 (592h14m0.29802338s) [noise=true , address=212.102.52.215:4389 , piece_id=XKWKUSMNBZJO3ABRMGN3VOATGEVOE5C55GWHDVPMQDY22TA6HXXA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126321446 (592h14m0.298439872s) [noise=true , address=212.102.52.214:4355 , piece_id=T2CODIDT4O2TGX7F4HXMNBEKONWAZCD7TRBQFNWG2SCV3ASGQB2A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.126641497 (592h14m0.298759917s) [noise=true , address=212.102.52.215:4386 , piece_id=NLF2NCM35P7IOAIACAJYO5N4MVK5JES6T5P5U6FPRRN3G5REKENQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141143829 (592h14m0.313262252s) [noise=true , address=212.102.52.215:4392 , piece_id=A4IJF2RFLQY2VG4HZ447KOCYITW763OJPPWNRTKYZKCFRZJQKMBQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.141515346 (592h14m0.313633788s) [noise=true , address=212.102.52.214:4360 , piece_id=6LGODG45O3FCYJZEIRGTLQCCBJG747L44IDJPIGZPQNEMKFC53YQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143185356 (592h14m0.315303777s) [noise=true , address=212.102.52.213:4338 , piece_id=4GBDI4U65JJ7FYE2P6X37I7HDIM3KQEL6Y3UP3LZR22VQ6WYK7WQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.143284527 (592h14m0.315402937s) [noise=true , address=212.102.52.217:4435 , piece_id=KS4M3GUYTIOETQIXTYEMEZIMQ5XCJIKESM5FRGEPE7AWF2V6MPVA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.144423996 (592h14m0.31654242s) [noise=true , address=212.102.52.216:4402 , piece_id=S2JRC2AVSDYJZW4NJS6QORAFUHMI25AKDOGWJLVHBAGANXD57WPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.145527980 (592h14m0.31764639s) [noise=true , address=212.102.52.213:4331 , piece_id=UAEXBFAJJNKDJABPDOBFQIRBMCQXF4WY4BBIJUOCFNWGXA4YF4JA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146469812 (592h14m0.31858822s) [noise=true , address=212.102.52.213:4330 , piece_id=KDSVWFXRKVC5HWDMX2AEWXMQHR3YEXFZAAJ2QSRAIU6QCNGGZ44A, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146794204 (592h14m0.318912613s) [noise=true , address=212.102.52.214:4364 , piece_id=KPYEPDHBSOPGHSNAZEOBBWR46O3GN3FMP7NFGPABKL7F32JEXW6Q, seg_pos={0 0}, upload=663070]: Done uploading piece. err: <nil>
May 25 11:02:48.146810158 (592h14m0.318928569s) [seg_pos={0 0}, upload=663070]: Satisfied cohort requirements after 16 pieces
May 25 11:02:48.147109800 (592h14m0.319228228s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ey8Bngj9b4FsVddkt7Gy54obDPDoerL8sLAZBmKekPejXhg5M): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147119424 (592h14m0.319237846s) [noise=true , address=212.102.52.217:4426 , piece_id=CTFEVLW7MHNO3TJAOHVD4CSRJCFMRJZYPO3JNTCB3PPPRHXSDYCA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147149753 (592h14m0.319268181s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12259awyjTZXky86DZr2q8ak5V5KQz6f9pWiTbR6KtnS8rhqDDo): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147162311 (592h14m0.319280739s) [noise=true , address=212.102.52.215:4376 , piece_id=P4L4CUEDMOEWY73CCDELDQRJ6LCTDIINNDJMUTWTHURVVRIBMVMA, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.147383047 (592h14m0.319501477s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RQtuAbsJqQWMogK8y4ZNo6HY4Q3ffS2v6Y2RyPz267WPMxCE8): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.147391126 (592h14m0.319509553s) [noise=true , address=212.102.52.213:4336 , piece_id=5JBBNOWROX7DFSQWWAHI2LM3AP73N62IDDBPHYSY37M3TFURUZPQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148355954 (592h14m0.320474387s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 25 11:02:48.148370069 (592h14m0.3204885s) [noise=true , address=212.102.52.217:4424 , piece_id=VGGZSY3ASFTJO5UFGQJF7OD2SDP7EC4VHLSK3NE4NNYJM5TXGQAQ, seg_pos={0 0}, upload=663070]: Long tail cancellation due to optimal threshold
May 25 11:02:48.148405711 (592h14m0.320524121s) [seg_pos={0 0}, upload=663070]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 11:02:48.148515177 (592h14m0.320633601s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 11:02:48.148521550 (592h14m0.320639971s) [upload=663070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 11:02:51.545270058 (592h14m3.71738849s) [upload=663070]: Done uploading segments...
May 25 11:02:51.545296658 (592h14m3.717415086s) [upload=663070]: Upload finished. err: <nil>
Duration: 1m0.003103593s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
838.550601ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:03:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 838.609157ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
842.421533ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:01:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 842.44893ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
857.688441ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:01:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 857.721692ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
824.704209ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:01:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 824.848282ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
883.860245ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 11:00:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 883.968827ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
818.810714ms trace:
https://jaeger.dev.storj.io/trace/fb23a289728c702
FAIL (2026-05-25 10:59:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 818.857618ms
core
uplink
us-select-1
us1-placement-20
|