Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 148975 reports (Page 2787 of 2980)
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [......X...X......X.......X...X...X...........XX...X......X..]
duration: 
2.241504528s
FAIL (2025-07-09 00:46:21 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.241527137s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [.X.......X...X................X...........XX...X.XXX...X....]
duration: 
36.197868154s
FAIL (2025-07-09 00:45:56 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 36.197888422s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [.......X...X......X.......X...X...X...........XX...X......X.]
duration: 
24.341577923s
FAIL (2025-07-09 00:45:50 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 24.342199724s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [.............................................X....X.......X.]
duration: 
524.502053ms
FAIL (2025-07-09 00:45:36 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
errs: <nil>
Jul  9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments...
Jul  9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil>
Jul  9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload
Jul  9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload
Jul  9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments...
Jul  9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false
Jul  9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment...
Jul  9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment.
Jul  9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190    , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010     , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662     , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590     , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155    , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299     , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117    , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078    , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804     , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223    , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911     , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805     , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937     , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048     , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150    , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227    , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment
Jul  9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments...
Jul  9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117    , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590     , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010     , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048     , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190    , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299     , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937     , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227    , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662     , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805     , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155    , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911     , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078    , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection
Jul  9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804     , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223    , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection
Jul  9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150    , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection
Jul  9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments...
Jul  9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 524.52973ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
11 out of 60 is failed (threshold=3) [X.......X...X......X.......X...X...X...........XX...X......X]
duration: 
30.001577836s
FAIL (2025-07-09 00:45:21 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001597918s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [..X.......X...X................X...........XX...X.XXX...X...]
duration: 
40.970681845s
FAIL (2025-07-09 00:45:20 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 40.970801621s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
3 out of 60 is failed (threshold=3) [..............................................X....X.......X]
duration: 
30.036759435s
FAIL (2025-07-09 00:45:11 UTC) 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
errs: <nil>
Jul  9 00:43:45.211297211 (375h32m12.137591901s) [upload=300745]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:43:45.211302471 (375h32m12.137597162s) [upload=300745]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:43:45.269341492 (375h32m12.195636178s) [upload=300745]: Done uploading segments...
Jul  9 00:43:45.269358444 (375h32m12.19565313s) [upload=300745]: Upload finished. err: <nil>
Jul  9 00:44:41.129146431 (375h33m8.055441118s) [upload=300759]: Starting upload
Jul  9 00:44:41.129253693 (375h33m8.055548379s) [upload=300759]: Done starting upload
Jul  9 00:44:41.129403652 (375h33m8.05569833s) [upload=300759]: Uploading segments...
Jul  9 00:44:41.129792975 (375h33m8.056087669s) [upload=300759]: Got next segment. Inline: false
Jul  9 00:44:41.129845465 (375h33m8.056140152s) [upload=300759]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:44:41.129851436 (375h33m8.056146123s) [upload=300759]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:44:41.161096648 (375h33m8.087391325s) [seg_pos={0 0}, upload=300759]: Begin upload segment...
Jul  9 00:44:41.161143590 (375h33m8.087438271s) [seg_pos={0 0}, upload=300759]: Done begin upload segment.
Jul  9 00:44:41.161167424 (375h33m8.087462113s) [noise=true , address=45.91.116.14:4190    , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161202049 (375h33m8.087496745s) [noise=true , address=45.91.116.7:4010     , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161568494 (375h33m8.087863181s) [noise=true , address=77.90.31.14:4662     , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161739184 (375h33m8.08803387s) [noise=true , address=77.90.31.12:4590     , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.161926542 (375h33m8.08822123s) [noise=true , address=45.91.116.13:4155    , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.162264603 (375h33m8.088559289s) [noise=true , address=45.91.116.5:4299     , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.162275501 (375h33m8.088570197s) [noise=true , address=45.91.116.11:4117    , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.162710355 (375h33m8.089005047s) [noise=true , address=104.36.82.10:4078    , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163083824 (375h33m8.089378519s) [noise=true , address=104.36.82.4:4804     , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163449862 (375h33m8.089744559s) [noise=true , address=104.36.82.14:4223    , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163765583 (375h33m8.090060273s) [noise=true , address=104.36.82.7:4911     , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.163963749 (375h33m8.090258446s) [noise=true , address=77.90.31.18:4805     , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.164440292 (375h33m8.090734988s) [noise=true , address=45.91.116.4:4937     , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.164860336 (375h33m8.091155033s) [noise=true , address=45.91.116.8:4048     , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.165178752 (375h33m8.091473449s) [noise=true , address=104.36.82.12:4150    , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.165665625 (375h33m8.09196032s) [noise=true , address=45.91.116.12:4227    , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Uploading piece...
Jul  9 00:44:41.266729879 (375h33m8.193024572s) [upload=300759]: Next returned nil segment
Jul  9 00:44:41.266753705 (375h33m8.193048411s) [upload=300759]: Waiting for error group managing segments...
Jul  9 00:44:41.272587189 (375h33m8.198881879s) [noise=true , address=45.91.116.11:4117    , piece_id=W7YOBPGYU72MMDP6QZYVKV2344UGEO4TYQPZOIQ7FDMMYEIQ6I5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.273314775 (375h33m8.199609455s) [noise=true , address=77.90.31.12:4590     , piece_id=UOIGX7N2FC2SBE2RQIFO63XYXKVHSIHOZ5VWJPJSPRU74RYW7B4A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.273750474 (375h33m8.200045167s) [noise=true , address=45.91.116.7:4010     , piece_id=ZA27QVAEZEUUSEBE4V23X2CNK56N5RS5A4QZ2PS7R62DNVVFE6MA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.274331218 (375h33m8.200625895s) [noise=true , address=45.91.116.8:4048     , piece_id=SQM4MHUA7TV3GWFVHLKM4KMDD2NWJ2CJDYDUTORUM2KULD2EYBEQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.274923944 (375h33m8.201218621s) [noise=true , address=45.91.116.14:4190    , piece_id=OY6M4TUDDBUJCJKUP6PBTG37DQTMG3E7HWBQ3AYYVWWM44QDNU5A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.274948104 (375h33m8.201242776s) [noise=true , address=45.91.116.5:4299     , piece_id=JCSR6TFDCYXD6ZV6GCXKX5XEJKFABNI5XTXY3XWA3MKXDAI7HM6A, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.275645664 (375h33m8.201940343s) [noise=true , address=45.91.116.4:4937     , piece_id=6KNGXPXSYEZR2Q56VTZYNLQNIRE227F7HUKU2P5LNBJN73OFKPCA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.275743262 (375h33m8.202037953s) [noise=true , address=45.91.116.12:4227    , piece_id=MPQZGVY25SU6EMRXZYGRGXUKUL4KIYOLEL4Y3WJU7SODRYXFEMKA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.276941414 (375h33m8.203236106s) [noise=true , address=77.90.31.14:4662     , piece_id=7MGLGXOHAUGDF656E3YA5ERRIAJC37MOVLFLFPVSG6Q4KFBV3OAA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.277528968 (375h33m8.203823657s) [noise=true , address=77.90.31.18:4805     , piece_id=FHBHDNQE2H7NROVX3O3LUVFRIIJB3FJBFOBCX52TR5WM54UPI5ZQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:44:41.278121237 (375h33m8.204415924s) [noise=true , address=45.91.116.13:4155    , piece_id=NMJR6PP2WDCRXMTO5SB3NBP75YITP4N7SQC7WJCKA2KYTJIHV6EA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:45:05.004382596 (375h33m31.930677284s) [noise=true , address=104.36.82.7:4911     , piece_id=VDX3TS7WOSSMMBWZVWZOZI7D3C6KY3V2ALHUJ46YPGYO35C33YNQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: <nil>
Jul  9 00:45:11.094109646 (375h33m38.020404335s) [noise=true , address=104.36.82.10:4078    , piece_id=SSDXQ3MFNXPUSKIDU2TK7OHOZCJ7HI7UGN7YFESNT2CVGRXRR46Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:36468->104.36.82.10:4078: use of closed network connection
Jul  9 00:45:11.094166489 (375h33m38.020461179s) [noise=true , address=104.36.82.4:4804     , piece_id=IAUVRTJRNF5RXBVUSSOHX37QJX3KEQP5TSCRDNJ676EHKB34MAFQ, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:45:11.094218109 (375h33m38.020512787s) [noise=true , address=104.36.82.14:4223    , piece_id=NB3I3G56L7KKYY5M7FNLYVFHRCXK3NARKXRFJKR3QW4P7BQXHA4Q, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:46810->104.36.82.14:4223: use of closed network connection
Jul  9 00:45:11.094352041 (375h33m38.020646718s) [noise=true , address=104.36.82.12:4150    , piece_id=BDMXK6RCOK6LTX6FZACCH5PYHR2WRJJMPUILYJS6EMAWHFIQEXWA, seg_pos={0 0}, upload=300759]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:55618->104.36.82.12:4150: use of closed network connection
Jul  9 00:45:11.094400782 (375h33m38.020695476s) [seg_pos={0 0}, upload=300759]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:45:11.128159115 (375h33m38.054453808s) [upload=300759]: Done uploading segments...
Jul  9 00:45:11.128198296 (375h33m38.054492974s) [upload=300759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.036787038s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [...X.......X...X................X...........XX...X.XXX...X..]
duration: 
58.388540804s
FAIL (2025-07-09 00:44:39 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 58.388572623s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
10 out of 60 is failed (threshold=3) [.X.......X...X......X.......X...X...X...........XX...X......]
duration: 
18.009630818s
FAIL (2025-07-09 00:43:55 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.009658535s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...............................................X....X.......]
duration: 
17.507371536s
FAIL (2025-07-09 00:43:47 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 17.507398021s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [....X.......X...X................X...........XX...X.XXX...X.]
duration: 
41.32994411s
FAIL (2025-07-09 00:43:40 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 41.329964119s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [.....X.......X...X................X...........XX...X.XXX...X]
duration: 
1m0.002089262s
FAIL (2025-07-09 00:42:59 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.002117612s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [................................................X....X......]
duration: 
833.335147ms
FAIL (2025-07-09 00:42:58 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 833.37129ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
11 out of 60 is failed (threshold=3) [X.X.......X...X......X.......X...X...X...........XX...X.....]
duration: 
17.424906649s
FAIL (2025-07-09 00:42:57 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 17.42493518s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [XX.X.......X...X......X.......X...X...X...........XX...X....]
duration: 
15.357321094s
FAIL (2025-07-09 00:42:28 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 15.357347254s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.................................................X....X.....]
duration: 
416.437641ms
FAIL (2025-07-09 00:42:16 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 416.463439ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
10 out of 60 is failed (threshold=3) [......X.......X...X................X...........XX...X.XXX...]
duration: 
49.125374318s
FAIL (2025-07-09 00:41:47 UTC) 
context deadline exceeded
Duration: 49.125415216s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [...................................................X....X...]
duration: 
21.051535934s
FAIL (2025-07-09 00:41:45 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 21.051564792s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
12 out of 60 is failed (threshold=3) [.XX.X.......X...X......X.......X...X...X...........XX...X...]
duration: 
16.947373795s
FAIL (2025-07-09 00:41:45 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.947399267s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [X.XX.X.......X...X......X.......X...X...X...........XX...X..]
duration: 
21.34759506s
FAIL (2025-07-09 00:41:01 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.347621362s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
10 out of 60 is failed (threshold=3) [.......X.......X...X................X...........XX...X.XXX..]
duration: 
49.243172292s
FAIL (2025-07-09 00:40:58 UTC) 
context deadline exceeded
Duration: 49.24320921s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [....................................................X....X..]
duration: 
387.095651ms
FAIL (2025-07-09 00:40:10 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 387.127605ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
10 out of 60 is failed (threshold=3) [........X.......X...X................X...........XX...X.XXX.]
duration: 
36.809869597s
FAIL (2025-07-09 00:40:09 UTC) 
context deadline exceeded
Duration: 36.809890327s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [.X.XX.X.......X...X......X.......X...X...X...........XX...X.]
duration: 
7.394279331s
FAIL (2025-07-09 00:39:36 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.394306115s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [.....................................................X....X.]
duration: 
22.046553856s
FAIL (2025-07-09 00:39:33 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 22.046587155s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [X........X.......X...X................X...........XX...X.XXX]
duration: 
1m0.001184029s
FAIL (2025-07-09 00:39:32 UTC) 
context deadline exceeded
Duration: 1m0.001211349s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [..X.XX.X.......X...X......X.......X...X...X...........XX...X]
duration: 
30.000985034s
FAIL (2025-07-09 00:39:26 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001004036s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
2 out of 60 is failed (threshold=3) [......................................................X....X]
duration: 
30.033060493s
FAIL (2025-07-09 00:38:44 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul  9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul  9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul  9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul  9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul  9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul  9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul  9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul  9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul  9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul  9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul  9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul  9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226    , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733     , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191    , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117    , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975     , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045     , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047     , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299     , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843     , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083    , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553     , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293    , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul  9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874     , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969     , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul  9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113    , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul  9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803     , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul  9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul  9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.033088428s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [X..X.XX.X.......X...X......X.......X...X...X...........XX...]
duration: 
13.5052047s
FAIL (2025-07-09 00:38:37 UTC) 
orbiter warning: context deadline exceeded
Duration: 13.505242173s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
11 out of 60 is failed (threshold=3) [XX........X.......X...X................X...........XX...X.XX]
duration: 
1m0.000672503s
FAIL (2025-07-09 00:38:32 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000684958s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration: 
557.098691ms
FAIL (2025-07-09 00:37:53 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul  9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul  9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul  9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul  9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul  9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul  9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul  9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul  9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul  9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul  9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul  9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul  9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul  9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul  9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 557.132998ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
10 out of 60 is failed (threshold=3) [.XX........X.......X...X................X...........XX...X.X]
duration: 
1m0.00091938s
FAIL (2025-07-09 00:37:32 UTC) 
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000941631s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [........................................................X...]
duration: 
867.892022ms
FAIL (2025-07-09 00:37:31 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul  9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul  9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul  9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul  9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul  9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul  9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul  9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul  9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul  9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul  9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul  9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul  9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul  9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul  9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 867.918315ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration: 
11.502681916s
FAIL (2025-07-09 00:37:18 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul  9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul  9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul  9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul  9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul  9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul  9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul  9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul  9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul  9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul  9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul  9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul  9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul  9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul  9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 11.502722529s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [XX..X.XX.X.......X...X......X.......X...X...X...........XX..]
duration: 
1.13078368s
FAIL (2025-07-09 00:37:14 UTC) 
orbiter warning: context deadline exceeded
Duration: 1.130811198s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration: 
20.256139389s
FAIL (2025-07-09 00:36:45 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul  9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul  9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul  9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul  9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul  9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul  9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul  9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul  9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul  9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul  9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul  9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul  9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul  9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul  9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 20.256170132s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
15 out of 60 is failed (threshold=3) [XXX..X.XX.X.......X...X......X.......X...X...X...........XX.]
duration: 
796.265748ms
FAIL (2025-07-09 00:36:45 UTC) 
orbiter warning: context deadline exceeded
Duration: 796.285711ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
9 out of 60 is failed (threshold=3) [..XX........X.......X...X................X...........XX...X.]
duration: 
35.493428341s
FAIL (2025-07-09 00:36:19 UTC) 
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 35.493443143s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
15 out of 60 is failed (threshold=3) [.XXX..X.XX.X.......X...X......X.......X...X...X...........XX]
duration: 
30.001474038s
FAIL (2025-07-09 00:36:05 UTC) 
orbiter warning: context deadline exceeded
Duration: 30.001489612s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [...........................................................X]
duration: 
30.03949796s
FAIL (2025-07-09 00:36:01 UTC) 
orbiter warning: 
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul  9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul  9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul  9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul  9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul  9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul  9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul  9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul  9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul  9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul  9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul  9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul  9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul  9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul  9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul  9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul  9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul  9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520     , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806     , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081    , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045     , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227    , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263     , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940     , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590     , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117    , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910     , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150    , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul  9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077    , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258    , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223    , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802     , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul  9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257    , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul  9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul  9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul  9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 30.039522123s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
9 out of 60 is failed (threshold=3) [...XX........X.......X...X................X...........XX...X]
duration: 
1m0.003061195s
FAIL (2025-07-09 00:35:44 UTC) 
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.003082321s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
14 out of 60 is failed (threshold=3) [..XXX..X.XX.X.......X...X......X.......X...X...X...........X]
duration: 
30.001168111s
FAIL (2025-07-09 00:35:35 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001266989s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [...XXX..X.XX.X.......X...X......X.......X...X...X...........]
duration: 
22.417179797s
FAIL (2025-07-09 00:34:37 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.417206485s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
8 out of 60 is failed (threshold=3) [....XX........X.......X...X................X...........XX...]
duration: 
46.679676638s
FAIL (2025-07-09 00:34:32 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 46.679719697s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [....XXX..X.XX.X.......X...X......X.......X...X...X..........]
duration: 
4.651236251s
FAIL (2025-07-09 00:33:38 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.651257811s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
8 out of 60 is failed (threshold=3) [.....XX........X.......X...X................X...........XX..]
duration: 
31.016017157s
FAIL (2025-07-09 00:33:36 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 31.016038983s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [.....XXX..X.XX.X.......X...X......X.......X...X...X.........]
duration: 
23.059199159s
FAIL (2025-07-09 00:33:07 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 23.059561122s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
8 out of 60 is failed (threshold=3) [......XX........X.......X...X................X...........XX.]
duration: 
19.191477012s
FAIL (2025-07-09 00:32:22 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.19150505s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
13 out of 60 is failed (threshold=3) [......XXX..X.XX.X.......X...X......X.......X...X...X........]
duration: 
19.680181903s
FAIL (2025-07-09 00:31:56 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.680206193s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
8 out of 60 is failed (threshold=3) [.......XX........X.......X...X................X...........XX]
duration: 
1m0.001250813s
FAIL (2025-07-09 00:31:54 UTC) 
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001282615s core uplink us-select-1 us1-placement-20